176
7 Solving Nonlinear Algebraic Equations
Here, f (x) is some prescribed formula involving x. For example, the equation
e
−x sin x = cos x
has
f (x) = e
−x sin x − cos x .
Just move all terms to the left-hand side and then the formula to the left of the
equality sign is f (x).
So, when do we really need to solve algebraic equations beyond the simplest
types we can treat with pen and paper? There are two major application areas. One
is when using implicit numerical methods for ordinary differential equations. These
give rise to one or a system of algebraic equations. The other major application type
is optimization, i.e., finding the maxima or minima of a function. These maxima and
minima are normally found by solving the algebraic equation F (x) = 0 if F (x) is
the function to be optimized. Differential equations are very much used throughout
science and engineering, and actually most engineering problems are optimization
problems in the end, because one wants a design that maximizes performance and
minimizes cost.
We first consider one algebraic equation in one variable, for which we present
some fundamental solution algorithms that any reader should get to know. Our
focus will, as usual, be placed on the programming of the algorithms. Systems of
nonlinear algebraic equations with many variables arise from implicit methods for
ordinary and partial differential equations as well as in multivariate optimization.
Our attention will be restricted to Newton’s method for such systems of nonlinear
algebraic equations.
Root Finding
When solving algebraic equations f (x) = 0, we often say that the solution x
is a root of the equation. The solution process itself is thus often called root
finding.
7.1 Brute Force Methods
The representation of a mathematical function f (x) on a computer takes two forms.
One is a Python function returning the function value given the argument, while the
other is a collection of points (x, f (x)) along the function curve. The latter is the
representation we use for plotting, together with an assumption of linear variation
between the points. This representation is also very well suited for equation solving:
we simply go through all points and see if the function crosses the x axis, or for
optimization: we test for local maximum or minimum points. Because there is a lot
of work to examine a huge number of points, and also because the idea is extremely
simple, such approaches are often referred to as brute force methods.
7 Solving Nonlinear Algebraic Equations
Here, f (x) is some prescribed formula involving x. For example, the equation
e
−x sin x = cos x
has
f (x) = e
−x sin x − cos x .
Just move all terms to the left-hand side and then the formula to the left of the
equality sign is f (x).
So, when do we really need to solve algebraic equations beyond the simplest
types we can treat with pen and paper? There are two major application areas. One
is when using implicit numerical methods for ordinary differential equations. These
give rise to one or a system of algebraic equations. The other major application type
is optimization, i.e., finding the maxima or minima of a function. These maxima and
minima are normally found by solving the algebraic equation F (x) = 0 if F (x) is
the function to be optimized. Differential equations are very much used throughout
science and engineering, and actually most engineering problems are optimization
problems in the end, because one wants a design that maximizes performance and
minimizes cost.
We first consider one algebraic equation in one variable, for which we present
some fundamental solution algorithms that any reader should get to know. Our
focus will, as usual, be placed on the programming of the algorithms. Systems of
nonlinear algebraic equations with many variables arise from implicit methods for
ordinary and partial differential equations as well as in multivariate optimization.
Our attention will be restricted to Newton’s method for such systems of nonlinear
algebraic equations.
Root Finding
When solving algebraic equations f (x) = 0, we often say that the solution x
is a root of the equation. The solution process itself is thus often called root
finding.
7.1 Brute Force Methods
The representation of a mathematical function f (x) on a computer takes two forms.
One is a Python function returning the function value given the argument, while the
other is a collection of points (x, f (x)) along the function curve. The latter is the
representation we use for plotting, together with an assumption of linear variation
between the points. This representation is also very well suited for equation solving:
we simply go through all points and see if the function crosses the x axis, or for
optimization: we test for local maximum or minimum points. Because there is a lot
of work to examine a huge number of points, and also because the idea is extremely
simple, such approaches are often referred to as brute force methods.