ReLU - Klyn API

ReLU

classin packageklyn.math.nn.activations

public class ReLU<T extends AbstractFloat = Float> extends Layer<T>

ReLU

Element-wise rectified linear activation for a batch of any sample shape. The derivative at zero is defined as zero. NaN inputs remain NaN.

Example

output = ReLU<Float>().forward(inputs)

Properties

Modifier and Type	Member	Description
public override readonly property	displayName `displayName as String`	Summary label.
public override readonly property	parameterCount `parameterCount as ULong`	ReLU has no trainable parameters.

Methods

Modifier and Type	Member	Description
public	backward `backward(inputs as NDArray<T>, upstream as NDArray<T>) as NDArray<T>`	Propagates an upstream derivative through the activation.
public	build `build(inputSize as Long) as Long`	Validates the width; this activation has no parameters to initialize.
public override	buildShape `buildShape(inputShape as IList<Long>) as IList<Long>`	Validates arbitrary positive sample dimensions.
public override	differentiate `differentiate(inputs as NDArray<T>, upstream as NDArray<T>) as LayerGradients<T>`	Computes input derivatives; there are no parameter derivatives.
public override	forward `forward(inputs as NDArray<T>) as NDArray<T>`	Applies max(0, x) to each batch element without retaining inputs.
public override	parameter `parameter(index as Long) as NDArray<T>`	Rejects every parameter index because ReLU has no parameters.