Objectives

A central object in SimpleSolvers are objectives (see SimpleSolvers.AbstractObjective). They are either SimpleSolvers.AbstractUnivariateObjectives or MultivariateObjectives. The goal of a solver (both LinearSolvers and NonlinearSolvers) is to make the objective have value zero. The goal of an Optimizer is to minimize a MultivariateObjective.

Examples

Univariate Objectives

We can allocate a univariate objective with:

using SimpleSolvers

f(x::Number) = x ^ 2
x = rand()
obj = UnivariateObjective(f, x)

UnivariateObjective:

    f(x)              = NaN 
    d(x)              = NaN 
    x_f               = NaN 
    x_d               = NaN 
    number of f calls = 0 
    number of d calls = 0 

Associated to UnivariateObjective are the following functions (amongst others):

• value
• value!
• value!!
• derivative
• derivative!
• derivative!!

The function value evaluates the objective at the provided input and increases the counter by 1:

y = value(obj, x)

0.2716638206564001

We can check how the function call changed obj:

obj

UnivariateObjective:

    f(x)              = NaN 
    d(x)              = NaN 
    x_f               = NaN 
    x_d               = NaN 
    number of f calls = 1 
    number of d calls = 0 

The stored value for f has not been updated. In order to so we can call the in-place function value!:

y = value!(obj, x)
obj

UnivariateObjective:

    f(x)              = 2.72e-01 
    d(x)              = NaN 
    x_f               = 5.21e-01 
    x_d               = NaN 
    number of f calls = 2 
    number of d calls = 0 

We further note that SimpleSolvers contains another function value!! that forces evaluation. This is opposed to value! which does not always force evaluation:

y = value!(obj, x)
obj

UnivariateObjective:

    f(x)              = 2.72e-01 
    d(x)              = NaN 
    x_f               = 5.21e-01 
    x_d               = NaN 
    number of f calls = 2 
    number of d calls = 0 

So value! first checks if the objective obj has already been called on x. In order to force another evaluation we can write:

y = value!!(obj, x)
obj

UnivariateObjective:

    f(x)              = 2.72e-01 
    d(x)              = NaN 
    x_f               = 5.21e-01 
    x_d               = NaN 
    number of f calls = 3 
    number of d calls = 0 

The function value can also be called without additional input arguments:

value(obj)

0.2716638206564001

But then the associated function is not called again (calling value this way does not increase the counter):

obj

UnivariateObjective:

    f(x)              = 2.72e-01 
    d(x)              = NaN 
    x_f               = 5.21e-01 
    x_d               = NaN 
    number of f calls = 3 
    number of d calls = 0 

An equivalent relationship exists between the functions derivative, derivative! and derivative!!.

In addition to UnivariateObjective, SimpleSolvers also contains a TemporaryUnivariateObjective^[1]:

t_obj = TemporaryUnivariateObjective(obj.F, obj.D)

TemporaryUnivariateObjective{Float64, typeof(Main.f), SimpleSolvers.var"#12#13"{typeof(Main.f)}}(Main.f, SimpleSolvers.var"#12#13"{typeof(Main.f)}(Main.f))

Multivariate Objectives

MultivariateObjectives are used in a way similar to UnivariateObjectives, the difference is that the derivative functions are replaced by gradient functions, i.e.:

• derivative $\implies$ gradient,
• derivative! $\implies$ gradient!,
• derivative!! $\implies$ gradient!!.

f(x::AbstractArray) = sum(x .^ 2)
x = rand(3)

obj = MultivariateObjective(f, x)

MultivariateObjective (for vector-valued quantities only the first component is printed):

    f(x)              = NaN 
    g(x)₁             = NaN 
    x_f₁              = NaN 
    x_g₁              = NaN 
    number of f calls = 0 
    number of g calls = 0 

Every instance of MultivariateObjective stores an instance of Gradient to which we similarly can apply the functions gradient or gradient!:

gradient(obj, x)

3-element Vector{Float64}:
 1.042427591070766
 1.1736135149066969
 1.7817573961855622

The difference to Gradient is that we also store the value for the evaluated gradient, which can be accessed by calling:

gradient(obj)

3-element Vector{Float64}:
 NaN
 NaN
 NaN