AbstractNeuralNetworks

AbstractCell

An AbstractCell is a map from $\mathbb{R}^{M}×\mathbb{R}^{N} \rightarrow \mathbb{R}^{O}×\mathbb{R}^{P}$.

Concrete cell types should implement the following functions:

• initialparameters(backend::Backend, ::Type{T}, cell::AbstractCell; init::Initializer = default_initializer(), rng::AbstractRNG = Random.default_rng())
• update!(::AbstractLayer, θ::NamedTuple, dθ::NamedTuple, η::AbstractFloat)

and the functors

• cell(x, st, ps)
• cell(z, y, x, st, ps)

AbstractExplicitCell

Abstract supertype for explicit cells. This type exists mainly for compatibility with Lux.

AbstractExplicitLayer

Abstract supertype for explicit layers. This type exists mainly for compatibility with Lux.

AbstractLayer

An AbstractLayer is a map from $\mathbb{R}^{M} \rightarrow \mathbb{R}^{N}$.

Concrete layer types should implement the following functions:

• initialparameters(backend::Backend, ::Type{T}, layer::AbstractLayer; init::Initializer = default_initializer(), rng::AbstractRNG = Random.default_rng())
• update!(::AbstractLayer, θ::NamedTuple, dθ::NamedTuple, η::AbstractFloat)

and the functors

• layer(x, ps)
• layer(y, x, ps)

Chain

A chain is a sequence of layers.

A Chain can be initialized by passing an arbitrary number of layers

Chain(layers...)

or a neural network architecture together with a backend and a parameter type:

Chain(::Architecture, ::Backend, ::Type; kwargs...)
Chain(::Architecture, ::Type; kwargs...)

If the backend is omitted, the default backend CPU() is chosen. The keyword arguments will be passed to the initialparameters method of each layer.

A supertype for Chain, AbstractCell etc.

apply!(y, cell::AbstractCell, x, ps)

Simply calls cell(y, x, ps)

apply!(y, layer::AbstractLayer, x, ps)

Simply calls layer(y, x, ps)

apply(cayer::AbstractCell, x, ps)

Simply calls cell(x, st, ps)

apply(layer::AbstractLayer, x, ps)

Simply calls layer(x, ps)

initialparameters

Returns the initial parameters of a model, i.e., a layer or chain.

initialparameters(backend::Backend, ::Type{T}, model::Model; init::Initializer = default_initializer(), rng::AbstractRNG = Random.default_rng())
initialparameters(::Type{T}, model::Model; init::Initializer = default_initializer(), rng::AbstractRNG = Random.default_rng())

The init! function must have the following signature:

init!(rng::AbstractRNG, x::AbstractArray)

The default_initializer() returns randn!.