What is In-Place and what is Out-Of-Place

In SimpleSolvers we almost always use in-place functions internally for performance, but let the user deal with out-of-place functions for ease of use.

Example

using SimpleSolvers

f(x) = sum(x.^2 .* exp.(-abs.(x)) + 2 * cos.(x) .* exp.(-x.^2))

If we now allocate a MultivariateObjective based on this, we get a series of in-place functions based on this. For example value!^[1]:

x = [0.]
obj = MultivariateObjective(f, x)
y = [0.]
value!(obj, x)
value(obj) == f(x)

true

To compute the derivative we can use gradient!^[2]:

x = [[x] for x in -7.:.1:7.]
y = Vector{Float64}[]
for x_sing in x
    gradient!(obj, x_sing)
    push!(y, copy(gradient(obj)))
end

The idea is however that the user almost never used the in-place versions of these routines directly, but instead functions like solve! and value, gradient etc. as a possible diagnostic.