Module SciPy.optimize.nonlin

Nonlinear solvers

These solvers find x for which F(x)=0. Both x and F is multidimensional.

They accept the user defined function F, which accepts a python tuple x and it should return F(x), which can be either a tuple, or numpy array.

Example:

def F(x):: "Should converge to x=[0,0,0,0,0]" import numpy d = numpy.array([3,2,1.5,1,0.5]) c = 0.01 return -d*numpy.array(x)-c*numpy.array(x)**3

from scipy import optimize x = optimize.broyden2(F,[1,1,1,1,1])

All solvers have the parameter iter (the number of iterations to compute), some of them have other parameters of the solver, see the particular solver for details.

A collection of general-purpose nonlinear multidimensional solvers.

broyden1 -- Broyden's first method - is a quasi-Newton-Raphson

method for updating an approximate Jacobian and then inverting it

broyden2 -- Broyden's second method - the same as broyden1, but

updates the inverse Jacobian directly

broyden3 -- Broyden's second method - the same as broyden2, but

instead of directly computing the inverse Jacobian, it remembers how to construct it using vectors, and when computing inv(J)*F, it uses those vectors to compute this product, thus avoding the expensive NxN matrix multiplication.

broyden_generalized -- Generalized Broyden's method, the same as broyden2,

but instead of approximating the full NxN Jacobian, it construct it at every iteration in a way that avoids the NxN matrix multiplication. This is not as precise as broyden3.

anderson -- extended Anderson method, the same as the

broyden_generalized, but added w_0^2*I to before taking inversion to improve the stability

anderson2 -- the Anderson method, the same as anderson, but

formulated differently

The broyden2 is the best. For large systems, use broyden3. excitingmixing is also very effective. There are some more solvers implemented (see their docstrings), however, those are of mediocre quality.

Utility Functions

norm -- Returns an L2 norm of the vector

Function summary

anderson(F, xin, iter = 10, alpha = 0.10000000000000001, M = 5, w0 = 0.01, verbose = False)

anderson2(F, xin, iter = 10, alpha = 0.10000000000000001, M = 5, w0 = 0.01, verbose = False)

broyden1(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

broyden1_modified(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

broyden2(F, xin, iter = 10, alpha = 0.40000000000000002, verbose = False)

broyden3(F, xin, iter = 10, alpha = 0.40000000000000002, verbose = False)

broyden_generalized(F, xin, iter = 10, alpha = 0.10000000000000001, M = 5, verbose = False)

broyden_modified(F, xin, iter = 10, alpha = 0.34999999999999998, w0 = 0.01, wl = 5, verbose = False)

difference(a, b)

excitingmixing(F, xin, iter = 10, alpha = 0.10000000000000001, alphamax = 1.0, verbose = False)

linearmixing(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

mlog(x)

mul(C, b)

myF(F, xm)

norm(v)

solve(A, b)

sum(a, b)

vackar(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

Functions

anderson(F, xin, iter = 10, alpha = 0.10000000000000001, M = 5, w0 = 0.01, verbose = False)

Extended Anderson method.

Computes an approximation to the inverse Jacobian from the last M interations. Avoids NxN matrix multiplication, it only has MxM matrix multiplication and inversion.

M=0 .... linear mixing M=1 .... Anderson mixing with 2 iterations M=2 .... Anderson mixing with 3 iterations etc. optimal is M=5

anderson2(F, xin, iter = 10, alpha = 0.10000000000000001, M = 5, w0 = 0.01, verbose = False)

Anderson method.

M=0 .... linear mixing M=1 .... Anderson mixing with 2 iterations M=2 .... Anderson mixing with 3 iterations etc. optimal is M=5

broyden1(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

Broyden's first method.

Updates Jacobian and computes inv(J) by a matrix inversion at every
iteration. It's very slow.

The best norm |F(x)|=0.005 achieved in ~45 iterations.

broyden1_modified(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

Broyden's first method, modified by O. Certik.

Updates inverse Jacobian using some matrix identities at every iteration,
its faster then newton_slow, but still not optimal.

The best norm |F(x)|=0.005 achieved in ~45 iterations.

broyden2(F, xin, iter = 10, alpha = 0.40000000000000002, verbose = False)

Broyden's second method.

Updates inverse Jacobian by an optimal formula.
There is NxN matrix multiplication in every iteration.

The best norm |F(x)|=0.003 achieved in ~20 iterations.

Recommended.

broyden3(F, xin, iter = 10, alpha = 0.40000000000000002, verbose = False)

Broyden's second method.

Updates inverse Jacobian by an optimal formula.
The NxN matrix multiplication is avoided. 

The best norm |F(x)|=0.003 achieved in ~20 iterations.

Recommended.

broyden_generalized(F, xin, iter = 10, alpha = 0.10000000000000001, M = 5, verbose = False)

Generalized Broyden's method.

Computes an approximation to the inverse Jacobian from the last M interations. Avoids NxN matrix multiplication, it only has MxM matrix multiplication and inversion.

M=0 .... linear mixing M=1 .... Anderson mixing with 2 iterations M=2 .... Anderson mixing with 3 iterations etc. optimal is M=5

broyden_modified(F, xin, iter = 10, alpha = 0.34999999999999998, w0 = 0.01, wl = 5, verbose = False)

Modified Broyden's method.

Updates inverse Jacobian using information from all the iterations and avoiding the NxN matrix multiplication. The problem is with the weights, it converges the same or worse than broyden2 or broyden_generalized

difference(a, b)

excitingmixing(F, xin, iter = 10, alpha = 0.10000000000000001, alphamax = 1.0, verbose = False)

J=-1/alpha

The best norm |F(x)|=0.005 achieved in ~140 iterations.

linearmixing(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

J=-1/alpha

The best norm |F(x)|=0.005 achieved in ~140 iterations.

mlog(x)

mul(C, b)

myF(F, xm)

norm(v)

Returns an L2 norm of the vector.

solve(A, b)

Solve Ax=b, returns x

sum(a, b)

vackar(F, xin, iter = 10, alpha = 0.10000000000000001, verbose = False)

J=diag(d1,d2,...,dN)

The best norm |F(x)|=0.005 achieved in ~110 iterations.

Local name	Refers to
math	math
numpy	numpy

Local name

Refers to

math

numpy

Module SciPy.​optimize.​nonlin

Nonlinear solvers

Function summary

Functions

Imported Names

Module SciPy.optimize.nonlin