cellflow.networks.SeedAttentionPooling

class cellflow.networks.SeedAttentionPooling(num_heads=8, v_dim=64, seed_dim=64, dropout_rate=0.0, transformer_block=False, layer_norm=False, act_fn=<PjitFunction of <function silu>>, parent=<flax.linen.module._Sentinel object>, name=None)

Pooling by multi-head attention with a trainable seed.

Parameters:

• num_heads (int) – Number of heads.

• v_dim (int) – Dimensionality of the value.

• seed_dim (int) – Dimensionality of the seed.

• dropout_rate (float) – Dropout rate.

• transformer_block (bool) – Whether to make it a transformer block (adds FC layer with residual connection).

• layer_norm (bool) – Whether to use layer normalization.

• act_fn (Callable[[Array], Array]) – Activation function.

• parent (Module | Scope | _Sentinel | None)

• name (str | None)

References

[Vaswani et al., 2023]

Methods

act_fn()	SiLU (aka swish) activation function.
apply(variables, *args[, rngs, method, ...])	Applies a module method to variables and returns output and modified variables.
bind(variables, *args[, rngs, mutable])	Creates an interactive Module instance by binding variables and RNGs.
clone(*[, parent, _deep_clone, _reset_names])	Creates a clone of this Module, with optionally updated arguments.
copy(*[, parent, name])	Creates a copy of this Module, with optionally updated arguments.
get_variable(col, name[, default])	Retrieves the value of a Variable.
has_rng(name)	Returns true if a PRNGSequence with name name exists.
has_variable(col, name)	Checks if a variable of given collection and name exists in this Module.
init(rngs, *args[, method, mutable, ...])	Initializes a module method with variables and returns modified variables.
init_with_output(rngs, *args[, method, ...])	Initializes a module method with variables and returns output and modified variables.
is_initializing()	Returns True if running under self.init(...) or nn.init(...)().
is_mutable_collection(col)	Returns true if the collection col is mutable.
lazy_init(rngs, *args[, method, mutable])	Initializes a module without computing on an actual input.
make_rng([name])	Returns a new RNG key from a given RNG sequence for this Module.
module_paths(rngs, *args[, show_repeated, ...])	Returns a dictionary mapping module paths to module instances.
param(name, init_fn, *init_args[, unbox])	Declares and returns a parameter in this Module.
perturb(name, value[, collection])	Add an zero-value variable ('perturbation') to the intermediate value.
put_variable(col, name, value)	Updates the value of the given variable if it is mutable, or an error otherwise.
setup()	Initializes a Module lazily (similar to a lazy __init__).
sow(col, name, value[, reduce_fn, init_fn])	Stores a value in a collection.
tabulate(rngs, *args[, depth, ...])	Creates a summary of the Module represented as a table.
unbind()	Returns an unbound copy of a Module and its variables.
variable(col, name[, init_fn, unbox])	Declares and returns a variable in this Module.
__call__(x[, mask, training])	Apply the pooling by multi-head attention.

Attributes

dropout_rate
layer_norm
name
num_heads
parent
path	Get the path of this Module.
scope
seed_dim
takes_attention_mask	Whether this block's __call__ accepts an attention mask as its second positional argument.
transformer_block
v_dim
variables	Returns the variables in this module.