Gradients

The supertype Gradient comprises different ways of taking gradients:

• GradientFunction,
• GradientAutodiff,
• GradientFiniteDifferences.

We first start by showing GradientAutodiff:

f(x::AbstractArray) = sum(x .^ 2)
x = rand(3)
grad = GradientAutodiff(f, x)

GradientAutodiff{Float64, typeof(Main.f), ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3}}}}(Main.f, ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3}}}((Partials(1.0, 0.0, 0.0), Partials(0.0, 1.0, 0.0), Partials(0.0, 0.0, 1.0)), ForwardDiff.Dual{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3}[Dual{ForwardDiff.Tag{typeof(Main.f), Float64}}(1.3e-322,1.33e-322,6.9471834029228e-310,6.94718340292436e-310), Dual{ForwardDiff.Tag{typeof(Main.f), Float64}}(1.4e-322,1.4e-322,6.94718340292594e-310,6.9471834029275e-310), Dual{ForwardDiff.Tag{typeof(Main.f), Float64}}(1.43e-322,1.43e-322,6.9471834029291e-310,6.9471834029307e-310)]))

Instead of calling GradientAutodiff(f, x) we can equivalently do:

grad = Gradient(f, x; mode = :autodiff)

GradientAutodiff{Float64, typeof(Main.f), ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3}}}}(Main.f, ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3}}}((Partials(1.0, 0.0, 0.0), Partials(0.0, 1.0, 0.0), Partials(0.0, 0.0, 1.0)), ForwardDiff.Dual{ForwardDiff.Tag{typeof(Main.f), Float64}, Float64, 3}[Dual{ForwardDiff.Tag{typeof(Main.f), Float64}}(1.425981173916e-311,1.0253483662039e-310,4.074231921973e-312,1.425981173916e-311), Dual{ForwardDiff.Tag{typeof(Main.f), Float64}}(1.0253483662039e-310,4.074231921973e-312,1.425981173916e-311,1.0253483662039e-310), Dual{ForwardDiff.Tag{typeof(Main.f), Float64}}(4.074231921973e-312,1.425981174928e-311,4.0742319196e-312,4.0742319196e-312)]))

When calling an instance of Gradient we can use the functions gradient and gradient!^[1]:

gradient(x, grad)

3-element Vector{Float64}:
 1.042427591070766
 1.1736135149066969
 1.7817573961855622