cellflow.networks.SelfAttention.act_fn

SelfAttention.act_fn()

SiLU (aka swish) activation function.

Computes the element-wise function:

\[\mathrm{silu}(x) = x \cdot \mathrm{sigmoid}(x) = \frac{x}{1 + e^{-x}}\]

swish() and silu() are both aliases for the same function.

Args:

x : input array

Returns:

An array.

See also:

sigmoid()

Return type:

Array

Parameters:

x (Array | ndarray | bool | number | bool | int | float | complex)