Quaternion Algebra

A quaternion is a four-dimensional quantity consisting of a scalar, say , and a three-dimensional vector , collectively denoted . Addition of two quaternions is component-wise,

(1)

(we do not need to add quaternions in this article). Their multiplication follows the rule

(2)

It is important to note that such multiplication is associative (even though noncommutative). This can be verified by the following.

When the scalar part of a quaternion is zero, it is called a pure quaternion. Since this constitutes an important special case, we will extend our definition of multiplication accordingly.

The operation of conjugation simply changes the sign of the vector part of a quaternion.

Based on the corresponding Taylor expansion, it is possible to evaluate various functions of a quaternion. Of these, the most important is the exponential function, which, in this article, we need only with pure-quaternion arguments.

Rotation

Rotating a 3D vector with respect to an axis through the origin (with a unit direction of ) by an angle can be achieved by

(3)

where (a vector of length and unit direction ), and and are the parallel and perpendicular (to ) parts of , respectively [1]. Note that and commute, while and anticommute.

One can show that a quaternion has the form of if and only if (a pure scalar). Alternately, the same can be parametrized by the three Euler angles (see the next section).

By taking , where is time, we achieve constant rotation with an uniform angular velocity . Note that

(4)

where , and denotes its time derivative.

The previous formula can be generalized to any time-dependent , as can be seen by differentiating , thus

(5)

where is now the instantaneous angular velocity at time , which may change with time. Note that must be a pure quaternion (take the time derivative of to see that). Now, any rotation can be applied to the axes of the original (inertial) system of coordinates, thus creating the corresponding new, rotating (noninertial) frame. Components of any vector can then be expressed in either the old, or (often and more conveniently) the new coordinates; the latter will be given by . The angular velocity , in the new coordinates, will thus have components given by . This becomes an important and helpful tool: even though the laws of physics are normally valid only in inertial coordinates, the resulting equations often simplify when expressed in the rotating frame.

Spinning Top

Let us consider an axially symmetrical body (a “top”) of mass , and parallel and perpendicular (with respect to the body’s symmetry axis) moments of inertia equal to and , respectively. We make it spin around its axis, place the bottom tip of its (usually tilted) axis on a nonslip horizontal plane (a desk), and let it continue its motion subject to the vertical gravity field. Let us also assume that the top’s center of mass is at a distance from its point of contact, that the gravitational acceleration is , and that there is no friction.

To investigate the top’s motion, we introduce an inertial coordinate system with its origin at the point of contact and its direction pointing vertically upward. First we position a motionless top vertically (its symmetry axis aligned with the direction) on the desk’s surface. We then rotate it using a fully general, time-dependent rotation , parametrized by three Euler angles, thus

(6)

where is a rapidly increasing function of time that represents the actual spinning of the top around its symmetry axis (this rotation is applied first), followed by tilting the spinning top by an angle (potentially a slowly varying function of time) around the inertial direction, further followed by slowly rotating the spinning, tilted top around the inertial direction by an angle , thus creating a so-called precession. Also: , (not used in the last formula), and are unit vectors along the , , and directions, respectively, in their pure-quaternion form.

Equations of Motion

We now use the Lagrange technique to find the corresponding equations for the three Euler angles. The kinetic energy of the top equals one-half of the scalar product of its angular velocity with its angular momentum, as its motion is purely rotational. In the rotating frame (this is when it becomes handy), this equals

(7)

where , , are the rotating-frame coordinates of , and (denoted from now on; similarly, will be called ) is the top’s moment of inertia with respect to a line perpendicular to its symmetry axis, and passing through the origin (which is displaced by from its center of mass). Usually (a thin circular disk needs to be bigger than only one-half of its radius to meet this condition), but our results are valid when as well.

Subtracting the potential energy (by the definition of ) yields the resulting Lagrangian.

Since the result is free of and (it contains only their time derivatives), the corresponding and must be constants of motion [2]. The remaining equation is then obtained from

(8)

This yields the following.

The first two expressions are constants of motion, while the last one must be equal to zero.

Introducing a new variable by

(9)

the last three expressions simplify further. It is also important to realize that the transformation (9) replaces only the parameter (the speed of spinning); if we are interested in the behavior of and only (as is usually the case), we get the same solution regardless of whether we use (in the old set of equations) or (in the new one).

So, let us see what the new results look like.

The equations are now much simpler, and the number of parameters has been reduced from three to only one (namely , whose dimension is ).

We can now easily infer a possibility of what we call a “steady” solution, with a constant value of (say ) and

(10)

where and are two constants whose product must be equal to . To relate the precession speed to the actual spinning speed, say , we have to solve (based on (9))

(11)

for , which yields

(12)

where the negative sign corresponds to the usually observed “slow” precession (assuming, from now on, that ), whereas the positive sign yields a hard-to-achieve “fast” precession. When (spherically symmetrical top), we get only the “slow” solution . For a fast-spinning top, the previous formula yields, to a good approximation, ; the precession speed is practically independent of .

Approximate Solution

The actual motion is usually more complicated than a steady solution of the previous section, as may undergo periodic oscillations, called nutation. Assuming that the amplitude of these is small, we can expand the three dependent variables of our equations around the steady-state solution:

(13)

where the first term of each right-hand side is a constant, the second term remains time dependent, and is small. We substitute these into the three equations, neglecting and higher powers of .

Each of the three expressions must be equal to zero.

We solve the first two equations for and and substitute into the last equation.

Solving for (and, subsequently, for and ) is now quite easy. Note that the coefficient of in the last expression is always non-negative.

Also note that will be always positive (assuming “slow” precession), whenever the following condition is met:

(14)

Reversing the above inequality results in so-called looping orbits (in terms of the path of the axis, when displayed on a unit sphere: see below), with periodically changing direction; making the two sides of (14) equal to each other results in cuspidal orbits (when reaches its smallest value, becomes zero).

If desired, one can extend the approximate solution to achieve (and higher) accuracy.

Exact Solution

One can show that the original differential equation for , namely

(15)

has the following constant of motion.

This can be easily verified.

Making and equal to their initial values, we can solve for and and substitute these into (minus its initial value, so that the resulting expression must be equal to zero).

The resulting differential equation can be simplified by the transformation [3].

To simplify the subsequent solution, we have assumed (without loss of generality) that the initial time has been chosen to coincide with reaching its maximum value (and, consequently, ). The above equation can then be solved, using specific initial conditions, which must satisfy (due to (15), since ); here provides the solution for .

This can be easily transformed back to ; one can then find the corresponding and by analytically integrating the expressions for and (if desired) . Finally, the results can be displayed graphically.

One can now compute and display the top’s motion under various initial conditions, check the accuracy of the approximate solution, and so forth. We will leave it for you to explore. Have fun!

References

About the Author

Jan Vrbik
Department of Mathematics, Brock University
500 Glenridge Ave., St. Catharines
Ontario, Canada, L2S 3A1
jvrbik@brocku.ca

Introduction

Computational quantum chemistry makes extensive use of various integrals (and their derivatives) of the general form [1, 2, 3]

(1)

where is an unnormalized Cartesian Gaussian function centered at :

(2)

where is normally taken at the nucleus, is the orbital exponent, and the polynomial represents the angular part, in that the sum of the Cartesian angular momenta corresponds to functions of type , , , , …. When the operator is 1, one simply has the overlap/density integral; otherwise it can be the energy operator for kinetic energy , electron-nuclear attraction , or electron-electron repulsion (which would involve double integrals). Other molecular properties involving external fields (response functions) or transition moments can also be computed from integrals of this form.

Gaussian-Type Functions

Gaussian-type functions are not the most natural choice for expanding the wavefunction. Slater-type functions, where the exponent is instead, can describe atomic systems more realistically. However, complications quickly arise in molecular calculations, which has led to the use of Gaussian functions in the overwhelming majority of current computational programs. Gaussian functions possess several desirable computational properties [4, 5, 6]. (Much credit is due to S. F. Boys for the introduction of the Cartesian Gaussian function into computational chemistry and its early development in a series of 12 papers under the general title “Electronic Wavefunctions,” the first being [5].) A given Slater function can be approximated by a linear combination of several Gaussians.

The first useful property is that the product of two Gaussian functions located at and is another Gaussian located at a point somewhere between and . (The proof of this can be found in [4].) The product of two Gaussians and is:

(3)

with and .

A second desirable property is that a derivative of a Gaussian can be expressed as a sum of Gaussians of lower and higher Cartesian angular values.

Overlap Integrals

The simplest molecular integral is the overlap integral . We first separate the integral into its orthogonal components:

where the notation expresses its functional dependence on the Cartesian angular components. The component, for instance, is:

(4)

Using a binomial expansion in the polynomial part,

yields:

(5)

Odd values of result in odd functions whose integrals vanish. For even values of , a solution [7] for the integrals is given by , and, in those cases:

(6)

We keep in mind that for the summations only terms of even values of survive. We have thus obtained an expression to evaluate the overlap of two Gaussians with arbitrary Cartesian angular factors. Equation (6) should be sufficient for programming or even for manual evaluation with small Cartesian angular values. One can, however, further reduce the number of operations by using a recurrence relation, one of the most useful techniques of computational chemistry. Recurrence relations let us efficiently calculate molecular integrations of higher angular values using previously obtained results with lower angular values. Recurrence relations are used in most computational chemistry programs and their application to other molecular integrations will be shown here.

Recurrence Relations

The derivation of the following relations involves straightforward algebraic manipulations, but is rather lengthy. Its omission does not impede our understanding of the recurrence relation, but helps maintain a reasonable continuity in our discussion. Interested readers can find a detailed derivation in the Appendix.

We start by defining the function

where we have removed the factor from in (6). If we take the derivative of with respect to the nuclear coordinate using the definition of in equation (4),

(7)

and combining the result with the same derivative of , but using the definition of in equation (6) instead,

(8)

we obtain

(9)

The same approach in which we combine the derivatives of (4) and (6) with respect to the nuclear coordinate gives

(10)

For , we have the index recursion relation

(11)

and combining equations (7) and (8) yields the transfer equation

(12)

Starting with the initial values and , the recurrence relation and the transfer equation, we can build up the overlap of functions of higher Cartesian angular values from lower ones. This is particularly useful for contracted Gaussian basis primitives with different functions sharing the same exponent.

Implementation

The function Ov evaluates the overlap integral of two Gaussian functions; here alpha, beta, RA, RB, LA, and LB are , , , , , and as defined earlier.

Examples

The published contracted Gaussian basis sets (see, for example, [8]) are usually not normalized; in our first example, we will calculate the normalization factor of the and Cartesian Gaussian functions that we will need later on. The normalization factor is simply the inverse square root of the overlap integral. Here we calculate the overlap between two functions with the Cartesian angular vector ,

and observe that . The normalization factor for functions is . Similarly, the overlap between two functions () is

where we have analogously used the recurrence relations to obtain and afterward, . Similar results are obtained for and . The normalization factor for the function is then . These results are special cases of the more general formula of the normalization factor

and we note that this value depends only on the orbital exponent and the Cartesian angular values.

In the second example, we will calculate the overlap matrix of the water molecule (, , geometry optimized at the HF/STO-3G level). The molecule lies in the - plane with Cartesian coordinates in atomic units.

In the STO-3G basis set, each atomic orbital is described by a sum of three Gaussians; here are their primitive contraction coefficients and orbital exponents (taken from [8]).

Here are the centers and Cartesian angular values of the orbitals, in the following order: , , , , , , and .

For instance, the atomic orbital of the hydrogen atom 1 is described as

(13)

Similarly, the orbital of the oxygen atom is

(14)

The overlap integral between the two orbitals expands to nine integrals involving the primitives. Using the function Ov, for example, the overlap integral between the first two primitives of equations (13) and (14) is

And the overlap between and is

The overlap matrix for the entire water molecule in the STO-3G basis set can be calculated in a similar manner.

Since the overlap matrix is symmetrical, we need to calculate only the elements above the main diagonal. The basis functions are normalized, as indicated by the unit diagonal elements. We note that equals zero. This is the overlap between the orbital of the first hydrogen atom and the of the oxygen atom. The molecule lies in the - plane, so this overlap vanishes due to symmetry. Similar cases occur in the overlaps between the and orbitals of () and among the orbitals (), etc. The strongest overlaps are those between the hydrogen orbitals and the oxygen orbital.

We next plot the atomic-orbital overlap between of and of ,

in the - plane, superimposing the molecule structure.

We observe a strong distortion of the positive (lower) lobe of the function toward the hydrogen atom. The negative lobe shows less deformation, whereas the node remains precisely at the atomic position. Note that this is simply the orbital overlap between noninteracting atoms, such as in the case of the “promolecule.”

Conclusion

We have provided an introduction to the evaluation of molecular integrals involving Gaussian-type basis functions both analytically and by use of recurrence relations. The results are general and relatively straightforward; the simple algorithm makes it suitable for implementation in a number of programming languages. Together with the kinetic, nuclear-electron attraction, and electron-electron repulsion energies, this is the first step toward the calculation of molecular energies and electronic properties.

Appendix

Here we provide the derivation of equations (7) and (8). Differentiate using equation (5) with respect to :

Consider the derivative term

the first term is simply

and, using the chain rule, the derivative of the second term is

recalling that . Substitute the results into the expression for ,

Writing in the second term inside the bracket, after expanding we have

Comparing the three integrals with the definitions of and we have the desired equation (7).

To derive equation (8), we differentiate with respect to using equation (6) instead,

We make use of the relationship

to reduce the derivative above to

This is equation (8), which is what was needed to prove.

References

About the Authors

Minhhuy Hô received his Ph.D. in theoretical chemistry at Queen’s University, Kingston, Ontario, Canada in 1998. He is currently a professor at the Centro de Investigaciones Químicas at the Universidad Autónoma del Estado de Morelos in Cuernavaca, Morelos, México.

Julio-Manuel Hernández-Pérez obtained his Ph.D. at the Universidad Autónoma del Estado de Morelos in 2008. He has been a professor of chemistry at the Facultad de Ciencias Químicas at the Benemérita Universidad Autónoma de Puebla since 2010.

Minhhuy Hô
Universidad Autónoma del Estado de Morelos
Centro de Investigaciones Químicas
Ave. Universidad, No. 1001, Col. Chamilpa
Cuernavaca, Morelos, Mexico CP 92010
homh@uaem.mx

Julio-Manuel Hernández-Pérez
Benemérita Universidad Autónoma de Puebla
Facultad de Ciencias Químicas
Ciudad Universitaria, Col. San Manuel
Puebla, Puebla, Mexico CP 72570
jumahernandez@gmail.com

Introduction

This is my first column since Version 6 came out a few months ago. Version 6, after being in the works for many years, provides a wealth of new features that are useful in many numeric and symbolic calculations, advanced programs, visualizations, and elsewhere. With so many exciting new possibilities, it was not easy to decide what to discuss in this column. So instead of delving into a new subject, I think the best way to see some of the new features and resulting capabilities of Version 6 is to compare with some corresponding Version 5 results. That is why today I will come back to a subject discussed in earlier columns and demonstrate how to go quite a bit further with Version 6.

In earlier columns, I started discussing the visualization of Riemann surfaces. In this column, I want to come back to this mathematically (and aesthetically) beautiful subject. So far, we discussed the construction of Riemann surfaces of compositions of elementary functions with finitely many branch points (articles IIa to IId [1-4]). With the new 3D graphics system and interactive features of Version 6, new possibilities open up for visualizing Riemann surfaces; the main ones are:

• Specifying vertex normals to obtain smooth-looking surfaces
• Using opacity to get a better look “inside” a Riemann surface (instead of cutting holes in the polygons of the surface)
• Using adaptive refinement of the built-in graphics functions to better resolve branch points
• Using Manipulate to easily view a function as a 3D projection of the hypersurface (where showing and over the complex plane become special cases)
• Using the Exclusions option to avoid branch cuts in Plot3D, ParametricPlot3D, and others

These features offer a fresh look at the topic of Riemann surfaces (so, if numbered, this would be article IIdII, not yet IIe, which will appear later and deal with special functions).

Starting with this column, I will also introduce a formatting change: Mathematica’s default StandardForm is an ideal editing and reading environment for mathematical expressions and very small program snippets (one to two lines). Convenient automatic line breaking, two-dimensional fraction and radical formatting, and other features make writing and reading easy. But for multiline program-like code pieces, the formatted results are frequently less readable than manually formatting the code to avoid unexpected line breaks, alignments, and nonuniform line spacings. A new style in Version 6 (available as -8) is the "Code" style. It is the default style for packages (the canonical place to store a program) and can, of course, also be used in notebooks. It does not automatically break lines, but allows for special characters and inherits the new syntax coloring feature to maximize code readability. So, function definitions are given in "Code" style (easily identifiable in this notebook by its light blue background) and example uses of the defined functions will be in style "Input". In addition to an easier visual discrimination between actual definitions and example uses, this also allows for a quick selection of all definitions (using RIGHT) for immediate use without having to run all (sometimes time-consuming) examples to experiment with further Riemann surfaces.

The Updated Riemann Surface Package

We start with an updated version of the construction discussed in my earlier columns. Let us quickly recap the construction idea: Starting with a given function , a set of coupled nonlinear meromorphic differential equations is derived from and this set is solved on patches of the plane. These patches arise from a tensor product decomposition of the complex plane in a cylindrical coordinate system and are separated by branch points. The system of differential equations is solved numerically and the resulting function values are displayed. If at the same time we calculate , it is possible to construct the normals to obtain a smooth-looking surface. (Calculating the normals of the 3D embedding of can be done easily using the Cauchy-Riemann conditions.) An updated version of the package, based on the code from The Mathematica GuideBook for Numerics [5], is available on the Mathematica GuideBooks website (www.mathematicaguidebooks.org/V6/downloads/RiemannSurfacePlot3D.m).

This is the main function defined in the package.

The first example is displayed as over the complex plane.

The function has a few new options in addition to the built-in 3D graphics options.

Because 3D graphics in Version 6 are typically no longer made from individual polygons, but rather from GraphicsComplex objects that contain polygons and their adjacency information, it is possible to avoid the small gaps that were used in Version 5 between the patches. The function StitchPatches “stitches” the GraphicsComplex objects together along their boundaries. While in static images, the small (typically 1ppm or smaller) gaps are usually not visible; when interactively rotating graphics, such gaps can potentially become more pronounced.

Note that we do not have to specify a domain of the plane over which to plot the function. The domain is chosen automatically in such a way to include all finite branch points of the function.

Here is another example of a Riemann surface plotted with RiemannSurfacePlot3D.

In the next example, we display the imaginary part of over the complex plane and color the surface according to the real part.

Using a transparent surface makes it easier to see the inner parts of more complicated Riemann surfaces.

Here are some more examples.

RiemannSurfacePlot3D stores some intermediate time-consuming results for the function it has just plotted. As a result, once a function is plotted, calls to RiemannSurfacePlot3D with an identical first argument and mostly identical option settings, but a potentially different second argument, are generally quite fast.

Here is a demonstration that allows moving smoothly from the real to the imaginary part.

By parametrizing the three Cartesian coordinates as a linear form of , , , and , we can implement an even more general view of the surface. We do this through a Manipulate object that makes it easy to change any of the 12 parameters that define the actual function value that is plotted.

We give two examples: the first algebraic and the second inverse trigonometric.

Here is the inverse trigonometric function.

For details of the construction, see the package. Within the new package editor of Version 6, it is quite convenient and straightforward to experiment. Input cells are kept in the package and are easily evaluated; comments and examples can be easily embedded, but are invisible when the package is loaded. When developing or investigating package code with the package editor, we do not have to worry about contexts or saving outputs.

RiemannSurfacePlot3D fails for functions with infinitely many branch points. Here is an example.

The construction given next deals with functions having infinitely many branch points.

The New ContourPlot3D

ContourPlot3D, a package function in Version 5, is now a much faster built-in kernel function. This function comes in handy sometimes for polynomials dealing with Riemann surfaces. Here is a simple example that uses ContourPlot3D to display the real part of the functions defined implicitly through a bivariate polynomial.

Eliminating yields a (more complicated) trivariate polynomial that can be plotted.

Using the Exclusions Options

Now we will use new features of Plot3D and ParametricPlot3D to construct a Riemann surface. Using these 3D plotting functions on a function that has branch cuts, we now automatically get cuts along the branch cuts.

Modulo the mesh, in Version 5, we would have gotten a graphic similar to the following, much less appealing (but still correct) image.

We can also obtain descriptions of the excluded cuts using the function VisualizationDiscontinuities from the context Visualization`.

This neat feature of plotting functions that recognize branch cuts can be used to construct Riemann surfaces by finding parametrizations of all sheets and then plotting them. Instead of the steep vertical walls in earlier versions of Mathematica, we now get small cuts between the sheet patches. The analytic continuation of the logarithm and the power function are straightforward to define through adding multiples of and multiplication by .

For the logarithm, with its infinitely many sheets, we restrict ourselves to (typically) three sheets.

This list can be easily extended to elliptic integrals, Bessel functions, polylogarithms, and others.

The function makeSheetList generates a list of the analytically continued sheets of a multivalued function by forming the outer product of all sheets of all the multivalued functions that occur.

Here is an example. The function results in six parametrized sheets: three from the outer logarithm combined with two from the inner square root.

Here is a quick view on the branch cut structure of these six functions.

And here are the real and imaginary parts of the six sheets shown as 3D plots.

Emphasizing the cuts and making the surface transparent gives an even better looking graphic.

As a second example, we use . This time we have five sheets, all arising from the topmost fifth root.

Here is a plot of the real part over the complex plane.

The function RiemannSheetParametricPlot3D combines the sheet generation and plotting results. All sheets are included in the first argument of ParametricPlot3D, instead of making multiple calls to it. This has the advantage of getting better visual results because the optimal plot range will be calculated by taking all sheets into account.

Here are two examples of this function in action: one algebraic and the other a transcendental function containing .

Using RiemannSheetParametricPlot3D, we can now plot the Riemann surface of the function that failed earlier. We emphasize the vertical connection along the branch cuts through red polygons.

And here are four considerably more complicated examples. Because of the infinite number of branch points, the function RiemannSurfacePlot3D cannot handle them.

We generalize RiemannSheetParametricPlot3D to allow for a matrix as its second argument.

In effect, the next graphic shows over the , plane.

To summarize: Using analytically continued sheets and Plot3D/ParametricPlot3D with the Exclusions option lets you plot Riemann surfaces of compositions of elementary functions. This includes cases where the branch points cannot be calculated in closed form or the case of countably many branch points. Two potential disadvantages of this approach are that calculating the exclusions can be potentially very time consuming and the gaps arising from cutting out thin strips along potential branch cuts cannot be made arbitrarily small with reasonable computational effort.

Higher-Order Polynomial Roots

This is not unexpected. The internal function VisualizationDiscontinuities yields no exclusions for the Root object.

The reason is the quite large computational effort needed to determine the cuts for parametrized roots. While the branch points are relatively easy to determine as the common root of the function and its derivative through one resultant calculation, calculating the cuts is more expensive.

Expanding the first of the roots, we see the existence of three second-order branch points.

A discontinuity of a Root object occurs due to a switch in the root numbering. This in turn results from coinciding real parts of different roots. At the point where two roots have a common real part, the real part must have a double root. So carrying out the two resultant calculations gives a sufficient set of algebraic curves along which the roots will exhibit discontinuities.

Even for just a fourth root, evaluating rootExclusions is quite time consuming and the resulting polynomial is large and complex.

Here is a quick overview of the last polynomial.

Once the set of exclusions has been calculated, we can use it as the setting for the Exclusions option.

Life in Four Dimensions

Up to now, we have seen and over the complex plane (we plotted a more general graphic earlier). The most complete information of a function is the set of pairs of complex numbers or the quadruple of real numbers . Imagining the hypersurface in (isomorphic to ), we are naturally led to Banchoff’s tetraview [6] of a complex-valued function. In this section, we implement such a construction. Starting with a plot of the real and imaginary parts of a complex-valued function over the complex plane, we extract the underlying graphic complex, and re-evaluate the function at the , values. (Using the already discretized surface in the form of a GraphicsComplex gives us the advantage of adaptive subdivision.) Because we will use the values of the derivatives of with respect to for calculating the normals, we store the values of the derivatives at the (complex) vertex points in the option setting for VertexNormals.

We define a function RiemannSheetParametricPlots4D that generates a GraphicsComplex with vertex values in . (Here we use the suboption TimeConstraint in the Exclusions option to allow the calculation of more complicated exclusion sets for more complicated functions.) The sheets are generated using the function analyticallyContinue.

Here is an example. The abbreviated output shows that the actual 4D GraphicsComplex is quite large.

In the absence of a direct 4D viewer in Version 6 (or any support for Graphics4D, for that matter—hopefully Version 7 will have at least some), we must project into a 3D subspace to view the surface as a Graphics3D object. For a given projection matrix, this is straightforward for the vertices and not too difficult for the normals. Fortunately, having the information of the embedded hypersurface in 4D allows a projection with normals in 3D using the Cauchy-Riemann conditions, which will result in a smooth-looking surface. Because we will use the final function interactively, we use Compile to carry out the calculation of the 3D vertex coordinates and the 3D normals as efficiently as possible.

The function to3DGraphicsComplex carries out the projection from 4D to 3D.

Putting the last two functions together, we have TetraviewPlot. We give it the option ColorFunction4D, which operates on the unscaled 4D coordinates to easily color a surface.

Here are a few examples of the function TetraviewPlot.

The next example colors the surface of )) according to values of .

A convenient and natural way to parametrize the projection matrix is to use the six rotation angles between the four coordinate planes in .

In the definition for RiemannSheetParametricPlots4D, we used the line

to cache the result of the most time-consuming step inside TetraviewPlot. This allows us to implement a demonstration that interactively changes the 3D subspace selected.

This demonstration gives us a feeling for the 2D hypersurface. Branch point neighborhoods in correspond to flat regions in , and vice versa. As a result, the initial small gaps along the cuts of widen as we look at the 4D surface from different directions.

This ends our short visit into the world of Riemann surface visualizations with Mathematica 6. We will be coming back to this subject in future columns.

References

Michael Trott
Senior Member, Technical Staff
Wolfram Research, Inc.
mtrott@wolfram.com

Fifteen Classic Puzzles

1. If a package has one of 50 random baseball cards, how many packages do you need to buy to get a complete set?
   (demonstrations.wolfram.com/CouponCollectorProblem)
2. A chess king is at one corner of a chessboard. It makes a series of moves that always take it closer to the opposite corner. How many possible paths are there? (demonstrations.wolfram.com/DelannoyPathExhaustionDiagonal)
3. A hallway has 100 lockers, all closed. 100 students are sent down the hall as follows: student 1 opens all the lockers; student 2 closes every other locker, beginning with the second; student 3 changes the state of every third locker, beginning with the third; and so on. After all the students have marched, which lockers remain open?
   (demonstrations.wolfram.com/TheLockerProblem)
4. In the Florida Lotto, a player picks 6 numbers out of 1 to 53. Matching 3, 4, 5, or 6 numbers from the drawing wins a prize. What are the odds of winning? (demonstrations.wolfram.com/UrnProblem)
5. On a grid, plant 10 trees to make 5 lines each containing 4 trees. (demonstrations.wolfram.com/OrchardPlantingProblem)
6. A grid has 25 “on” buttons. Pressing a button changes the state of that button and the surrounding buttons. Can all the buttons be turned “off”? (demonstrations.wolfram.com/LightsOutPuzzle)
7. Can books similar in size be stacked on the edge of a desk so that the top book is not over the desk at all? (demonstrations.wolfram.com/StackingDominoesToTheLimit)
8. McDonald’s once sold Chicken McNuggets in packs of 6, 9, or 20. What is the highest number of McNuggets you cannot buy?
   (demonstrations.wolfram.com/McNuggetProblemAndFrobeniusNum-bers)
9. Thirty people are in a room. What is the probability that two of them share a birthday? (demonstrations.wolfram.com/TheBirthdayProblem)
10. Wheels of radius 30cm and 40cm are 10cm apart. What is the length of a snug belt that goes around both? (demonstrations.wolfram.com/TwoWheelBelt)
11. You have a 3 gallon jug and a 5 gallon jug at a well. How can exactly 1 gallon be measured? (demonstrations.wolfram.com/JugProblem)
12. A 30-foot statue is on a 10-foot pedestal. Where should you stand so that the statue itself looks as big as possible from your point of view? (demonstrations.wolfram.com/TheStatueOfRegiomontanus)
13. As a contestant on Monty Hall’s game show, you are presented with three doors. Behind one door is a new car and behind each of the other two is a goat. You select a door behind which, you hope, is the new car. Monty Hall then opens another door to reveal a goat and asks if you would like to change your selection. Should you switch?
    (demonstrations.wolfram.com/MontyHallProblem)
14. You have 12 coins, labeled with letters M, I, T, F, O, L, K, D, A, N, C, and E. One of the coins is fake, and is heavier or lighter than the others. Can you find the fake coin in 3 weighings?
    (demonstrations.wolfram.com/The12CoinWeighingProblem)
15. A 55-gallon barrel, with a diameter of 22.5 inches, is lying on its side. 10 inches of grain is in the barrel. How much grain does the barrel contain? (demonstrations.wolfram.com/SagittaApothemAndChord)

Discussion—Coupon Collecting

For each of the problems listed above, a link is given to a Wolfram Demonstration that allows for further exploration [1]. For example, the Coupon Collector Demonstration comes with the following explanatory caption:

If a package has one of 50 random baseball cards, how many packages do you need to buy to get a complete set? The expected answer is packages, where is the harmonic number . Switching from baseball cards to coupons gives the coupon collector problem: if there are different kinds of coupons and many of them are distributed randomly, the expected number of purchases necessary for a complete set is .

Code containing a Manipulate is provided for each Demonstration, allowing for exploration of the problem with either Mathematica or Mathematica Player.

All the classic puzzles listed here have been given a similar Demonstration treatment. Is there another classic problem that should be added to The Wolfram Demonstrations Project? If so, please consider adding it at demonstrations.wolfram.com/participate.html. Drop me a line once your new Demonstration has been published.

References

Ed Pegg Jr
Scientific Information Editor
Wolfram Research, Inc.
edp@wolfram.com

Preliminaries

The most common encounter with the concept of dynamic programming—and here we will be concerned only with continuous time dynamic programming (CTDP)—occurs in the context of the optimal stopping of an observable stochastic process with given termination payoff. This is a special case of stochastic optimal control where the control consists of a simple on-off switch, which, once turned off, cannot be turned on again. A generic example of an optimal stopping problem can be described as this: the investor observes a continuous Markov process with state-space and with known stochastic dynamics and, having observed at time the value , where denotes the observed realization of the random variable , the investor must decide whether to terminate the process immediately and collect the payoff , where is some a priori determined (deterministic) payoff function, or to wait until time for the next opportunity to terminate the process (in the context of CTDP, the time step should be understood as an infinitesimally small quantity). In addition, having observed the quantity , the investor must determine the value of the observable system (associated with that state) at time . In many situations, the stopping of the process cannot occur after some finite deterministic time .

There is a vast literature that deals with both the theoretical and the computational aspects of modeling, analysis, and optimal control of stochastic systems. The works of Bensoussan and Lions [2] and Davis [3] are good examples of classical treatments of this subject.

Many important problems from the realm of finance can be formulated—and solved—as optimal stopping problems. For example, a particular business can be established at the fixed cost of dollars, but the actual market price of that business follows some continuous stochastic process , which is observable. Having observed at time the value , an investor who owns the right to incorporate such a business must decide whether to exercise that right immediately, in which case the investor would collect the payoff of dollars, or to postpone the investment decision until time , with the hope that the business will become more valuable. In addition, having observed the value , the investor may need to determine the market price of the guaranteed right (say, a patent) to eventually incorporate this business—now, or at any time in the future.

Another classical problem from the realm of finance is the optimal exercise of a stock option of American type, that is, an option that can be exercised at any time prior to the expiration date. A stock option is a contract that guarantees the right to buy (a call option) or to sell (a put option) a particular stock (the underlying) at some fixed price (the strike price, stated explicitly in the option contract) at any time prior to the expiration date (the maturity date, also stated in the option contract). Consider, for example, an American style put option which expires at some future date . On date , the holder of the option observes the underlying price and decides whether to exercise the option, that is, to sell the stock at the guaranteed price (and, consequently, collect an immediate payoff of dollars) or to wait, hoping that at some future moment, but no later than the expiration date , the price will fall below the current level . If the option is not exercised before the expiration date, the option is lost if or, ignoring any transaction costs, is always exercised if ; that is, if the option is not exercised prior to the expiration date , on that date the owner of the option collects the amount . In general, stock options are traded in the same way that stocks are traded and, therefore, have a market value determined by the laws of supply and demand. At the same time, the price of such contracts, treated as a function of the observed stock price , can be calculated from general economic principles in conjunction with the principles of dynamic programming.

In general, the solution to any optimal stopping problem has two components: (1) for every moment in time, , one must obtain a termination rule in the form of a termination set so that the process is terminated at the first moment at which the event occurs; and (2) for every moment , one must determine the value function , which measures how “valuable” different possible observations are at time . Clearly, if denotes the termination payoff function (e.g., in the case of an American style put option that would be , ), then for every one must have , while for every one must have ; that is, one would choose to continue only when continuation is more valuable than immediate termination. Consequently, the termination sets , , can be identified with the sets , and the associated optimal stopping time can be described as

Thus, the entire solution to the optimal stopping problem can be expressed in terms of the family of value functions , which may be treated as a single function of the form

It is important to recognize that the solution to the optimal stopping problem, that is, the value function , must be computed before the observation process has begun.

In most practical situations, an approximate solution to the optimal stopping problem associated with some observable process and some fixed termination payoff may be obtained as a finite sequence of approximate value functions

where is the termination date and is some sufficiently large integer number. This approximate solution can be calculated from the following recursive rule, which is nothing but a special discrete version of what is known as the dynamic programming equation: for , we set , that is, at time , the value function coincides with the termination payoff on the entire state-space , and for , , , we set

(1)

where is the instantaneous discount rate at time , and the probability measure , with respect to which the conditional expectation is calculated, is the so-called pricing measure (in general, the instantaneous discount rate may depend on the observed position and the pricing measure may be different from the probability measure that governs the actual stochastic dynamics of the process ). Furthermore, for every , , , the approximate termination set is given by

If, given any , the limit

(2)

( denotes the integer part of a real number) exists, in some appropriate topology on the space of functions defined on the state-space , then one can declare that the function

gives the solution to the optimal stopping problem with observable process , payoff function , and pricing measure .

It is common to assume that under the pricing probability measure the stochastic dynamics of the observable process are given by some diffusion equation of the form

(3)

for some (sufficiently nice) matrix-valued function , vector field , and some -valued Brownian motion process , which is independent from the starting position (note that is Brownian motion with respect to the law ). Given any , let denote the matrix and let denote the second-order field in given by

As is well known, under certain fairly general conditions for the coefficients and , and the termination payoff , the value function

that is, the solution to the associated optimal stopping problem, is known to satisfy the following equation, which is nothing but a special case of the Hamilton-Jacobi-Bellman (HJB) equation

(4)

with boundary condition , . This equation is a more or less trivial consequence of equations (1) and (2), in conjunction with the Itô formula. It is important to recognize that the dynamic programming equation (1) is primary for the optimal stopping problem, while the HJB equation (4) is only secondary, in that it is nothing more than a computational tool. Unfortunately, equation (4) admits a closed form solution only for some rather special choices of the payoff function , the diffusion coefficients and , and the discount rate . Thus, in most practical situations, the HJB equation (4) happens to be useful only to the extent to which this equation gives rise to some numerical procedure that may allow one to compute the solution approximately. Essentially, all known numerical methods for solving equation (4) are some variations of the finite difference method. The most common approach is to reformulate equation (4) as a free boundary value problem: one must find a closed domain with a piecewise differentiable boundary and nonempty interior , plus a continuous function , which is continuously differentiable with respect to the variable and is twice continuously differentiable with respect to the variables , that satisfies the following two relations everywhere inside

and, finally, satisfies the following boundary conditions along the free (i.e., unknown) boundary

The last two conditions are known, respectively, as the value matching and the smooth pasting conditions. It can be shown that under some rather general—but still quite technical—assumptions, the free boundary value problem that was just described has a unique solution (consisting of a function and domain ) which then allows one to write the solution to the optimal stopping problem as follows: for any and any, one has

The drawbacks from formulating an optimal stopping problem as a free boundary value problem for some parabolic partial differential equation (PDE), which—in principle, at least—can be solved numerically by way of finite differencing, are well known. First, the value matching and the smooth pasting conditions are difficult to justify and this makes the very formulation of the problem rather problematic in many situations. Second, just in general, for purely technical reasons, it is virtually impossible to use finite differencing when the dimension of the state-space is higher than 3 and, in fact, in the case of free boundary value problems, even a state-space of dimension 2 is rather challenging. Third, with the exception of the explicit finite difference scheme, which is guaranteed to be stable only under some rather restrictive conditions, the implementation of most finite differencing procedures on parallel processors is anything but trivial.

At the time of this writing, it is safe to say that finite differencing is no longer at the center of attention in computational finance, where most problems are inherently complex and multidimensional. Indeed, most of the research in computational finance in the last five years or so appears to be focused on developing new simulation-based tools—see [4-8], for example (even a cursory review of the existing literature from the last few years is certain to be very long and is beyond the scope of this article). Among these methods, the Longstaff-Schwartz algorithm [8] seems to be the most popular among practitioners.

The attempts to avoid the use of finite differencing are nothing new (there is a section in the book Numerical Recipes in C entitled There Is More to Life than Finite Differencing—see [9], p. 833). Indeed, Monte Carlo, finite element, and several other computational tools have been in use for solving fixed boundary value problems for PDEs for quite some time. Apparently, from the point of view of stochastic optimal control, it is more efficient—and in some ways more natural—to develop numerical procedures directly from the dynamic programming equation (1) and skip the formulation of the HJB equation altogether—at least this is the approach that most recent studies in computational finance are taking.

Since the solution to an optimal stopping problem is given by an infinite family of functions of the form , , for some set , from a computational point of view the actual representation of any such object involves two levels of discretization. First, one must discretize the state-space , that is, functions defined on must be replaced with finite lists of values attached to some fixed set of distinct abscissas

for some fixed . This means that for every list of values , one must be able to construct a function which “extends” the discrete assignment , , to some function defined on the entire state-space . Second, one must discretize time, that is, replace the infinite family of functions not just with the finite sequence of functions

determined by the discrete dynamic programming equation (1), but, rather, with a finite sequence of lists

computed from the following space-discretized analog of the time-discrete dynamic programming equation (1): for , we set

and then define

(5)

consecutively, for , , , where is simply the map that “extends” the discrete assignment .

Of course, in order to compute the conditional expectation in equation (5), the distribution law of the random variable relative to the pricing measure and conditioned to the event must be expressed in computable form. Unfortunately, this is possible only for some rather special choices for the diffusion coefficients and that govern the stochastic dynamics of the process under the pricing measure . This imposes yet another—third—level of approximation. The simplest such approximation is to replace in equation (5) with the random quantity

(6)

where is some standard normal -valued random variable. As a result of this approximation, expression (5) becomes

(7)

where stands for the standard normal probability density function in . Another (somewhat more sophisticated) approximation scheme for the diffusion process in the case is discussed in [1]. Fortunately, in the most widely used diffusion models in finance, the conditional law of , given the event , coincides with the law of a random variable of the form for some function that can be expressed in computable form, so that in this special case the right side of equation (7) is exactly identical to the right side of equation (5) and the approximation given in equation (6) is no longer necessary.

One must be aware that, in general, the distribution law of the random variable is spread on both sides of the point and, even though for a reasonably small time step this law is concentrated almost exclusively in some small neighborhood of the node , when this node happens to be very close to the border of the interpolation region, the computation of the integral on the right side of equation (7) inevitably requires information about the function outside the interpolation domain. However, strictly speaking, interpolating functions are well defined only inside the convex hull of the abscissas used in the interpolation. One way around this problem is to choose the interpolation domain in such a way that along its border the solution to the optimal stopping problem takes values that are very close to certain asymptotic values. For example, if one is pricing an American put option with strike price of , one may assume that the value of the option is practically when the price of the underlying asset is above , regardless of the time left to maturity (of course, this assumption may be grossly inaccurate when the volatility is very large and/or when the option matures in the very distant future). Consequently, for those nodes that are close to the border of the interpolation domain, the respective values will not depend on and will not be updated according to the prescription (7). At the same time, the remaining nodes must be chosen in such a way that the probability mass of the random variable is concentrated almost exclusively inside the interpolation region.

In general, numerical procedures for optimal stopping of stochastic systems differ in the following: First, they differ in the space-discretization method, that is, the method for choosing the abscissas and for “reconstructing” the function from the (finite) list of values assigned to the abscissas . Second, numerical procedures differ in the concrete quadrature rule used in the calculation of the conditional expectation in the discretized dynamic programming equation (5). The method described in this article is essentially a refinement of the method of dynamic interpolation and integration described in [1]. The essence of this method is that the functions are defined by way of spline interpolation—or some other type of interpolation—from the list of values , while the integral in equation (7) is calculated by using some standard procedures for numerical integration. As was illustrated in [1], the implementation of this procedure in Mathematica is particularly straightforward, since NIntegrate accepts as an input InterpolatingFunction objects, and, in fact, can handle such objects rather efficiently.

The key point in the method developed in this article is a procedure for computing a universal list of vectors , , associated with the abscissas , , that depend only on the stochastic dynamics of and the time step , so that one can write

(8)

for any and for any , where stands for the convex hull of the interpolation grid , that is, the domain of interpolation. Essentially, equation (8) is a variation of the well-known cubic spline quadrature rule—see §4.0 in [9] and §5.2, p. 89, in [10]. It is also analogous to the “cubature formula,” which was developed in [11] with the help of a completely different computational tool (following [14] and [11], we use the term cubature as a reference to some integration rule for multiple integrals and the term quadrature as a reference to some integration rule for single integrals). One of the principal differences between the interpolation quadrature/cubature rules and the cubature formula obtained in [11] is that the former allows for a greater freedom in the choice of the abscissas . This means that, in practice, one can choose the evaluation points in a way that takes into account the geometry of the payoff function and not just the geometry of the diffusion . From a practical point of view, this feature is quite important and, in fact, was the main reason for developing what is now known as Gaussian quadratures.

An entirely different method for computing conditional averages of the form

was developed in [12] and [13]. In this method, quantities of this form are approximated by finite products of linear operators acting on the function . One must be aware that the methods described in [11-13] require a certain degree of smoothness for the integrand , which, generally, InterpolatingFunction objects may not have.

The method developed in this article bears some similarity also with the quantization algorithm described in [4, 5]. The key feature in both methods is that one can separate and compute—once and for all—certain quantities that depend on the observable process but not on the payoff function. Essentially, this means that one must solve, simultaneously, an entire family of boundary value problems corresponding to different payoff functions. The two methods differ in the way in which functions on the state-space are encoded in terms of finite lists of values. The advantages in using interpolating functions in general, or splines in particular, are well known: one can “restore” the pricing maps from fewer evaluation points and this feature is crucial for optimal stopping problems in higher dimensions.

Making Quadrature/Cubature Rules with Mathematica

In this section we show how to make quadrature/cubature rules for a fixed set of abscissas that are similar to the well-known Gaussian rules or the cubic spline rules. We consider quadrature rules on the real line first. To begin with, suppose that one must integrate some cubic spline that is defined from the abscissas , , and from some list of tabulated values . Since is a cubic spline, for any fixed there are real numbers , , , and for which we have

As a result, one can write

(9)

which reduces the computation of the integral to the computation of the coefficients from the tabulated values and the abscissas .

Though not immediately obvious, it so happens that when all abscissas are fixed, each coefficient —and, consequently, the entire expression on the right side of equation (9)—can be treated as a linear function of the list (in fact, each is a linear function only of a small portion of that list). This property is instrumental for the quadrature/cubature rules that will be developed in the next section. To see why one can make this claim, recall that (e.g., see §3.3 in [9]) the cubic spline created from the discrete assignment , , is given by

(10)

where

and , (note that the definition implies , ). Usually, one sets (which gives the so-called natural splines) and determines the remaining values from the requirement that the cubic spline has a continuous first derivative on the entire interval (it is a trivial matter to check that with this choice the second derivative of the function defined in equation (10) is automatically continuous on ). Finally, recall that this last requirement leads to a system of linear equations with unknowns and that the right side of each of these equations is a linear function of the list , while the coefficients in the left side are linear functions of the abscissas —see equation (3.3.7) in [9], for example. This means that each quantity in the list can be treated as a linear function of the list . Since the quantities , , , and are all independent from the choice of , one can claim that the entire expression in the right side of equation (10) is a linear function of the list . Thus, for every and every , we can write

where are polynomials of degree at most . More importantly, these polynomials are universal in the sense that they depend on the abscissas but not on the tabulated values . In fact, this representation holds for spline interpolation objects of any degree, except that in the general case the degree of the polynomials may be bigger than 3. An analogous representation is valid for other (non-spline) interpolating functions ; in particular, such a representation is valid for interpolating functions defined by way of divided differences interpolation, which is the method used by ListInterpolation. For example, if denotes the usual interpolating polynomial of degree defined from the discrete assignment , , then is simply the Lagrange polynomial for the abscissas . Thus, when is an interpolating function object (defined by, say, splines, divided differences, or standard polynomial interpolation) from the assignment , , assuming that is some (fixed) continuous and strictly monotone function and that is some (fixed) integrable function and, finally, assuming that the values , , are chosen so that , , one can write

(11)

where

Since the list can be computed once and for all and independently from the interpolated values , and since the calculation of the integral comes down to the calculation of the dot product , the computation of the list may be viewed as a simultaneous integration of all functions of the form for all interpolating functions (of the same interpolation type) that are based on one and the same (forever fixed) set of abscissas . Indeed, the knowledge of the list turns the expression into a trivial function of the list , that is, a trivial function defined on the entire class of interpolating functions . In other words, in equation (11) we have made a quadrature (or cubature) rule.

The preceding remarks contain nothing new or surprising in any way and we only need to find a practical way for computing the weights . Fortunately, Mathematica allows one to define InterpolatingFunction objects even if the interpolated values are defined as symbols that do not have actual numerical values assigned to them. Furthermore, such objects can be treated as ordinary functions that can be evaluated, differentiated, and so on—all in symbolic form. We illustrate this powerful feature next (recall that ListInterpolation has default setting ).

Since, just by definition, the restriction of the function to the interval coincides with a polynomial of degree 3, by Taylor’s theorem, for any fixed , we have

Thus, if we define

then on the interval we can identify the function with the dot product

Mathematica can rearrange this expression as an ordinary polynomial:

We can also do

Essentially, this is some sort of reverse engineering of the InterpolatingFunction object , that is, a way to “coerce” Mathematica to reveal what polynomials the object is constructed of between the interpolation points.

There are other tricks that we can use in order to reverse engineer an InterpolatingFunction object. For example, with the definition of , the entire function can be restored from the values . To do the restoration we must solve a linear system for the four unknown coefficients in some (unknown) cubic polynomial. This can be implemented in Mathematica rather elegantly without the use of CoefficientList. The only drawback in this approach is that one must resort to Solve or LinearSolve, which, generally, are more time-consuming, especially in dimensions 2 or higher. Of course, instead of using reverse engineering, one can simply implement the interpolation algorithm step-by-step, which, of course, involves a certain amount of work.

For example, we have

In the definition of S34, the variable x0 can be given any numerical value inside the interval ; the choice was completely arbitrary.

In the same way, one can reverse engineer InterpolatingFunction objects that depend on any number of independent abscissas. Here is an example.

In this example, for and for , the quantity can be expressed as the following polynomial of the independent variables and .

If one does not suppress the output from the last line (which would produce a fairly long output), it becomes clear that all coefficients in the polynomial of the variables and are linear functions of the array . Consequently, if the variables and are integrated out, the expression will turn into a linear combination of all the entries (symbols) in the array VV.

Now we turn to concrete applications of the features described in this section. For the purpose of illustration, all time-consuming operations in this notebook are executed within the Timing function. The CPU times reflect the speed on Dual 2 GHz PowerPC processors with 2.5 GB of RAM. The Mathematica version is 6.0.1. For all inputs with computing time longer than 1 minute, a file that contains the returned value is made available.

Some Examples of Cubic Cubature Rules for Gaussian Integrals in

First, define the standard normal density in .

In this section, we develop cubature rules for computing integrals of the form

for certain functions . We suppose that the last integral can be replaced by

(12)

without a significant loss of precision, where the value

is chosen so that

The next step is to choose the abscissas , , and , :

For the sake of simplicity, in display equations we will write instead of and instead of . Then we define the associated array of symbolic values attached to the interpolation nodes

and produce the actual InterpolatingFunction object:

Note that V was defined simply as an array of symbols and that, therefore, f can be treated as a symbolic object, too. In any case, we have

Note that each double integral on the right side of the last identity is actually an integral of some polynomial of the variables and multiplied by the standard normal density in . Furthermore, as was pointed out earlier, every coefficient in this polynomial is actually a linear function of the array V. Since the InterpolatingFunction object f was defined with (the default setting for ListInterpolation), we can write

where are linear functions of the tensor that can be computed once and for all by using the method described in the previous section. In addition, the integrals in the right side of the last identity also can be computed once and for all by using straightforward integration—this is the only place in the procedure where actual integration takes place. Once these calculations are completed, the integral (12) can be expressed as an explicit linear function of the array V of the form

for some fixed tensor . Of course, the actual calculation of the tensor is much more involved and time consuming than the direct numerical evaluation of the integral (12) in those instances where f is not a symbolic object, that is, when the symbols are given concrete numeric values. The point is that the calculation of the tensor is tantamount to writing the integral (12) as an explicit function of the symbols that define f; that is, we evaluate simultaneously the entire family of integrals for all possible choices of actual numeric values that can be assigned to the symbols . Once the tensor is calculated, we can write

(13)

Note that this identity is exact if the abscissas are exact numbers. In other words, by computing the tensor we have made a cubature rule. Furthermore, when is some function defined in the rectangle that has already been defined and the symbol V is given the numeric values

then the right side of equation (13) differs from the left side by no more than times the uniform distance between the function and the interpolating function created from the assignment

First, we compute—once and for all—all integrals of the form

for all choices of , , , and , and we represent these integrals as explicit functions of the symbols .

Instead of calculating the tensor integrals, one may load its (precomputed) value from the files or integrals.txt (if available):

or

Next, we replace the symbols with the actual numeric values of the associated abscissas on the grid:

The next step is to create the lists of the midpoints between all neighboring abscissas

which would allow us to produce the Taylor expansion of the InterpolatingFunction object f at the points , for any and any . As noted earlier, for every choice of and , the associated expansion coincides with the object f everywhere in the rectangle

Now we will compute the entire expression

as a linear function of the symbols , , . Note that for every fixed and , the third summation simply gives the dot product between the vectors.

It is important to recognize that the sum of these dot products gives the exact value of the integral

Unfortunately, the exact values of the integrals stored in the list WM are quite cumbersome:

Since working with such expressions becomes prohibitively slow, we convert them to floating-point numbers (we can, of course, use any precision that we wish).

Note that this last operation is the only source of errors in the integration of the interpolating function f.

Finally, we compute the integral

in symbolic form, that is, as an explicit function of the symbols that define the interpolating function f—what we are going to compute are the actual numerical values of the coefficients for the symbols in the linear combination that represents this integral. Consequently, an expression in symbolic form for the integral is nothing but a tensor of actual numeric values. We again stress that the only loss of precision comes from the conversion of the exact numeric values for the integrals stored in WM into the floating-point numbers stored in WMN.

Instead of calculating the object CoeffList in the following, which contains the actual linear combination of the symbols , one may load its (precomputed) value from the files CoeffList.mx or CoeffList.txt (if available):

or

Now we extract the numeric values for the coefficients and store those values in the tensor :

Note that in order to produce the actual cubature rule we need to compute the tensor only once, so that several minutes of computing time is quite acceptable. Note also that (i.e., the cubature rule) can be calculated with any precision. This means that if we approximate an integral of the form

with the dot product

then the upper estimate for the error in this approximation can be made arbitrarily close to the uniform distance between the function and the interpolating function created from the values that u takes on the grid multiplied by the area of the integration domain, which is .

Now we turn to some examples. First, consider the function

and set

Now we have

For the function

we get

One must be aware that NIntegrate is a very efficient procedure and, generally, replacing NIntegrate with a “hand made” cubature rule of the type described in this section may be justified only when the integrand cannot be defined in terms of standard functions, or when a better control of the error is needed, provided that one can control the uniform distance between the integrand and the respective interpolating function created from the grid. In general, such “hand made” cubatures become useful mostly in situations where one has to compute a very large number of integrals for similarly defined integrands—assuming, of course, that the uniform error from the interpolation in all integrands can be controlled (this would be the case, for example, if one has a global bound on a sufficient number of derivatives).

Interpolation quadrature rules for single integrals can be obtained in much the same way. It must be noted that, in general, spline quadrature rules are considerably more straightforward than spline cubature rules. This is because smooth interpolation in can be obtained without any additional information about the derivatives at the grid points and this is not the case in spaces of dimension greater than or equal to. For example, cubic splines in are uniquely determined by the values at the grid points and by the requirement that the second derivative vanishes at the first and last grid points (the so-called natural splines). In contrast, smooth interpolation in two or more dimensions depends on the choice of the gradient and at least one mixed derivative at every grid point, and when information about these quantities is not available—as is often the case—one usually resorts to divided difference interpolation. In general, the efficiency of the interpolation may be increased by using a nonuniform grid and by placing more grid points in the regions where the functions that are being interpolated are more “warped.” We will illustrate this approach in the last section. The use of nonsmooth interpolation, such as bilinear interpolation in , for example, reduces considerably the computational complexity of the problem. In any case, the error from the interpolation must be controlled in one way or another. While the discussion of this issue is beyond the scope of this article, it should be noted that rather powerful estimates for the interpolation error are well known.

Some Examples of Bilinear Cubature Rules for Gaussian Integrals in

In general, just as one would expect, cubature rules based on bilinear interpolation are less accurate than the cubature rules described in the previous section (assuming, of course, one uses the same grid and keeping in mind that this claim is made just in general). However, from a computational point of view, bilinear cubature rules are considerably simpler and easier to implement, especially for more general integrals of the form

for some fixed functions Math and . The greater computational simplicity actually allows one to use a denser grid, which, in many cases, is a reasonable compensation for the loss in smoothness. The dynamic integration procedures in that are discussed in the next section are all based on bilinear cubature rules (in the case of we will use cubic quadratures).

Here we will be working with the same interpolation grid , , , , that was defined in the previous section and, just as before, in all display formulas will write instead of and instead of . Instead of working with the interpolation objects of the form

we will work with bilinear interpolation objects that we are going to “construct from scratch” as explicit functions of the variables and that depend on the respective grid points and tabulated values. This means that for

the bilinear interpolation object can be expressed as

where the function bi is defined as

It is clear from this definition that, treated as a function of the variables and , the expression bi can be written as

where the entire list can be treated as a function of the grid points and the tabulated values . In fact, this function (that is, a list of functions) can be written as

For every fixed and , the integral of the bilinear interpolating object taken over the rectangle

is simply the dot product between the list and the list of integrals (over the same rectangle) of the functions

Now we will calculate—and store—all these lists (of integrals) of dimension 4 for all choices for and for :

and then compute the sum of all dot products:

This sum is a linear function of the symbols , and this linear function represents the integral of the interpolating function over the entire interpolation region (each dot product gives the integral of in the respective sub-region). Finally, we must extract the coefficients for the symbols from the linear combination (of those symbols) that we just obtained.

Now we can use the tensor in the same way in which we used the tensor in the previous section: although contains concrete numeric values, this tensor still has the meaning of “integral of in symbolic form.” For example, if we set

then we obtain

which is reasonably close to

In this case, the cubic cubature rule is considerably more accurate:

Here is another example.

This function is nonsmooth, as the following 3D plot shows.

Finally, we remark that—at least in principle—the procedures developed in this and in the previous sections may be used with any other, that is, nonGaussian, probability density , including nonsmooth densities.

Application to Optimal Stopping Problems and Option Pricing

The main objective in this section is to revisit the method of dynamic interpolation and integration described in [1] and to present a considerably more efficient implementation of this method, which takes advantage of the fact that the objects that are being integrated sequentially are actually interpolating functions. More importantly, we will show how to implement this method in the case of optimal stopping problems for certain diffusions in . Essentially, this section is about the actual implementation in Mathematica of the method encoded in the space-discretized version of the time-discrete dynamic programming equation (7). The solution is approximated in terms of dynamic cubature rules of the form (8) by using the procedure encoded in equation (11). For the sake of simplicity, the examples presented in this section will be confined to the case where the (diffusion type) observation process is computable, in the sense that for any fixed , the random variable can be expressed as

for some computable function (note that the position of the diffusion process at time , , is independent of the future increment of the Brownian motion ). In other words, the conditional law of given the event can be identified with the distribution law of a random variable of the form

where is a standard normal -valued random variable. Of course, when is computable in the sense that we just described, then there is no need to use the approximation (6), and, if is not computable, in principle at least, the procedures described in this section can still be used, provided that the definition of the function is changed according to the recipe (6).

We note that the Mathematica code included in this section is independent of the code included in all previous sections.

To begin with, define the following symmetric probability density function in .

This is nothing but the probability density of a bivariate normal random variable , with standard normal marginals and and with correlation . Let , , and , , be two standard Brownian motions that are independent, so that for any . Next, for some fixed and for , set and for any . Note that the joint distribution law of is the same as the joint distribution law of . It is a trivial matter to check that

is a diffusion process in with generator

In what follows, we study the problem for optimal stopping of the process no later than time , for a given termination payoff . For the sake of simplicity, we will suppose that the discount rate for future payoffs is . In the special case where , , for some “sufficiently nice” function of one variable , the problem comes down to the optimal stopping of the one-dimensional diffusion . We analyze this one-dimensional case first.

The time step in the procedure, that is, the quantity , is set to

Next, define the interpolation grid:

Just as before, in all display equations we will write instead of and instead of . The boundary conditions will be prescribed at the first and last two abscissas on the extended grid . This means that the recursive procedure will update only the values on the grid . Thus, the procedure that we develop can be applied only to situations where the value function does not change at those points in the state-space that are either close to or happen to be very large.

The next step is to define the list of tabulated symbols

and the associated (symbolic) InterpolatingFunction object

Next, extract the coefficients of the cubic polynomials that represent the object g between the interpolation nodes.

For example,

which means that for , the expression can be identified with the polynomial

Next, note that the relation

is equivalent to

For every fixed , we will compute all integrals

and will store these integrals in the matrix (initially filled with zeros):

In particular, the integral

can be replaced by

and we remark that

The entire expression

then becomes a linear function of the symbols . This linear function (i.e., vector of dimension ) depends on the index and represents a quadrature rule for the integral in the left side in the last identity. We stress that we need one such rule for every abscissa , . We will store all these rules (i.e., vectors of dimension ) in the tensor (initially filled with zeroes):

As a first application, we compute the price of an American put option with payoff:

The first iteration will be done by using a direct numerical integration—not interpolation quadrature rules.

Now we will do additional iterations by using interpolation quadratures. The time step is years. This will give us the approximate price of an American put with strike price and one year left to expiry. Note that the list gives the integral associated with the abscissa , which is the same as for . The idea is to keep the value at the abscissa fixed, throughout the iterations, equal to the initial value and, similarly, keep the values at the abscissas and forever fixed at the initial values and .

Note that the first iteration took about 20 times longer to compute than the remaining 19. Now we define the value function after iterations—this is nothing but the option price with one year left to maturity treated as a function defined on the entire range of observable stock prices.

We plot the value function together with the termination payoff .

The range of abscissas at which the value function and the termination payoff coincide is precisely the price range where immediate exercise of the option is optimal, assuming that there is exactly one year left to expiry.

Now we consider the same optimal stopping problem but with a smooth payoff function, namely

Essentially, this is an example of some sort of a generalized obstacle problem.

In this case, there is no need to perform the first iteration by using direct numerical integration.

Now we perform iterations, which will give us the value function with three years left to the termination deadline.

The most common approach to dynamic programing problems of this type is to use the free boundary problem formulation, followed by some suitable variation of the finite difference scheme. However, such an approach is much less straightforward and usually takes much longer to compute—a fraction of a second would be very hard to imagine even on a very powerful computer. In fact, we could double the number of the interpolation nodes and cut the time step in half, which would roughly quadruple the timing, and still complete the calculation within one second.

It is important to point out that, in terms of the previous procedure, there is nothing special about the standard normal density. In fact, we could have used just about any other reasonably behaved probability density function —as long as the associated integrals are computable. In particular, one can use this procedure in the context of financial models in which the pricing process is driven by some nonwhite noise process, including a noise process with jumps. The only requirement is that we must be able to compute the integrals

for any .

Finally, we consider the two-dimensional case. We use the same grid-points on both axes:

Although the lists Y and YY are the same as and , in the notation that we use we will pretend that these lists might be different and will write instead of , instead of , and set

We will use bilinear cubature rules, which are easier to implement—we need one such rule for every point , , . Thus, we must compute all integrals

(14)

for , for , for , for , and for (because of the symmetry in the density , it is enough to consider only the case ). Now we need to operate with substantially larger lists of data and for that reason we need to organize the Mathematica code differently.

Assuming that is an interpolating function constructed by way of bilinear interpolation from the tabulated symbols

we can express each of the integrals

in the form

where is the matrix given by (here we use the definition of cfbi from the previous section)

and locIntegrals is the matrix of the integrals (14) indexed by ( is the first index and is the second). In fact, with our special choice for the bi-variate density , all these integrals can be calculated in closed form:

After fixing the values for the volatility and the time step

the integrals can be written as explicit functions of the integration limits:

Our method depends in a crucial way on the fact that is actually a linear function of the symbols . In fact, we can compute the coefficients in this linear combination explicitly, as we now demonstrate.

The next two definitions are taken from the previous section, so that the code included in this section can be self-contained.

Thus, the entire integral

(15)

can be treated as a linear function of the symbols

This linear function is nothing but a tensor of dimension and this tensor is nothing but a cubature rule for the integral (15), which, obviously, depends on the indices and . Consequently, we have one such cubature rule (tensor) for every choice of and every choice of and all such rules will be stored in the tensor (initially filled with zeros):

The calculation of the tensor is the key step in the implementation of the method of dynamic interpolation and integration in dimension 2. This is the place where we must face the “curse of the dimension.” It is no longer efficient to express the global integral (15) as the sum of local integrals of the form (14) and then extract from the sum the coefficients for the symbols . It turns out to be much faster if one updates the coefficients for the symbols sequentially, as the local integrals in (14) are calculated one by one. Just as one would expect, dealing with the boundary conditions in higher dimensions is also trickier. Throughout the dynamic integration procedure we will be updating the values of the value function at the grid points , , , or, which amounts to the same, the grid points , , . The values on the remaining grid points with , or , or , or , or , or , will be kept unchanged, or, depending on the nature of the problem, will be updated according to some other—one-dimensional, perhaps—dynamic quadrature rule.

Now we turn to the actual computation of the tensor . On a single “generic” processor, this task takes about hour (if more processors are available the task can be distributed between several different Mathematica kernels in a trivial way). The key point is that this is a calculation that we do once and for all. As will be illustrated shortly, once the tensor is calculated, a whole slew of optimal stopping problems can be solved within seconds. Of course, the “slew of optimal stopping problems” is limited to the ones where the termination payoff and the value functions obtained throughout the dynamic integration can be approximated reasonably well by way of bilinear interpolation from the same interpolation grid. In general, the set of these “quickly solvable” problems can be increased by choosing a denser grid and/or a grid that covers a larger area. However, doing so may become quite expensive. For example, if we were to double the number of grid points in each coordinate, the computing time for the tensor would increase roughly 16 times—again, everything is easily parallelizable.

It is important to recognize that the calculation of the list involves only the evaluation of standard functions and accessing the elements of a fairly large tensor ( has a total of 1,245,621 entries). This is close to the “minimal number of calculations” that one may expect to get away with: if the number of discrete data from which the value function can be restored with a reasonable degree of accuracy is fairly large, then there is a fairly large number of values that will have to be calculated no matter what. Furthermore, the evaluation of standard functions is very close to the fastest numerical procedure that one may hope to get away with in this situation. From an algorithmic point of view, the efficiency of this procedure can be improved in two ways: (1) find a new way to encrypt the value function with fewer discrete values; and/or (2) represent all functions involved in the procedure (i.e., all combinations of standard functions) in a form that allows for an even faster evaluation. In terms of hardware, the speed of the procedure depends not only on the speed of the processor but also on the speed and the organization of the memory.

Instead of calculating the list , one may load its (precomputed) value from the files Xi.mx or Xi.txt (if available):

or

Now we can turn to some concrete applications. First, we will consider an American put option with strike price on the choice of one of two uncorrelated assets (the owner of the option can choose one of the two assets when the option is exercised). The termination payoff from this option is

The two underlying assets are uncorrelated and follow the processes , , and , , where and are the Brownian motions described earlier in this section. Let be the value function with years left to expiry, that is, if the time left to expiry is years and the prices of the two assets are, respectively, and , then the value of the option is and we remark that what is meant here as “the value of the option” is actually a function defined on the entire range of prices, that is, the range of prices covered by the interpolation grid. Clearly, we must have . Furthermore, when one of the assets has a very large value, then the option is very close to a canonical American put option on the other asset. Consequently, the values at the grid points , where or , will never be updated and will remain forever fixed at the initial value . When is fixed to either or , the values at the grid points , , will be updated exactly as we did earlier in the case of an American put on a single asset with a strike price . Similarly, when is fixed to either or , the values at the grid points , , will be updated in the same way, that is, as if we were dealing with an American put on a single asset with strike price 40. Of course, since the payoff is symmetric and so is the density , we only need to update the values at the grid points for and .

Now we turn to the actual calculation. By using the method of dynamic integration of interpolating functions, we will compute—approximately—the pricing function that maps the prices of the underlying assets in the range covered by the grid into the price of the option with one year left to maturity. With time step , we need to perform 20 iterations in the dynamic integration procedure. Note that if we choose to perform the first iteration by direct numerical integration of the termination payoff, then that would require the calculation of 741 integrals.

We initialize the procedure by tabulating the termination payoff over the interpolation grid.

Then we do 20 iterations—at each iteration we update the tensor

Now we can produce the pricing map with one year left to maturity.

Calculating the derivatives of the value function (treated as functions, too) is straightforward as we now illustrate, and before we do we remark that in the realm of finance, information about these derivatives (known as the deltas of the option) is often more important than the pricing function itself.

In our second example, we consider an American put option on the more expensive of two given underlying assets (in contrast, in our previous example we dealt with an American put option on the less expensive of the two underlying assets). In this case, the termination payoff function is given by

This time the boundary conditions must take into account the fact that when one of the assets becomes too expensive, the option becomes worthless, and when one of the assets is worthless, the option may be treated as a standard American option on the other asset. For the purpose of illustration we will compute the value of the option with three years left to maturity.

This is a good example of a value function which is neither everywhere smooth nor convex.

In our final example we consider optimal stopping of the same diffusion in but with smooth termination payoff of the form

With this termination payoff the boundary conditions must reflect the fact that the option is worthless when one of the assets is worthless or when one of the assets is too expensive. We will execute iterations, which will give us the value function with three years left to expiry.

Conclusions

Most man-made computing devices can operate only with finite lists of numbers or symbols. Any use of such devices for modeling, analysis, and optimal control of stochastic systems inevitably involves the encoding of inherently complex phenomena in terms of finite lists of numbers or symbols. Thus one can think of two general directions that may lead to expanding the realm of computable models. First, one may try to construct computing devices that can handle larger and larger lists, faster and faster. Second, one may try to develop “smarter” procedures with which more complex objects can be encrypted with shorter lists of numbers. With this general direction in mind, the methodology developed in this article is entirely logical and natural. Indeed, the approximation of functions with splines or other types of interpolating functions is a familiar, well developed, and entirely natural computing tool. The use of special quadrature/cubature rules—as opposed to general methods for numerical integration—in the context of dynamic integration of interpolating functions is just as logical and natural. However, only recently have computer languages in which one can implement such procedures in a relatively simple and straightforward fashion become widely available. This article is an attempt to demonstrate how the advent of more sophisticated computing technologies may lead to the development of new and more efficient algorithms. Curiously, new and more efficient algorithms often lead to the design of new and more efficient computing devices. In particular, all procedures described in this article can be implemented on parallel processors or on computing grids in essentially a trivial way. There is a strong incentive to build computing devices that can perform simultaneously several numerical integrations, or can compute dot products between very large lists of numbers very fast.

Finally, we must point out that most of the examples presented in the article are only prototypes. They were meant to be executed on a generic (and slightly out of date) laptop computer with the smallest possible number of complications. Many further improvements in terms of higher accuracy and overall efficiency can certainly be made.

Acknowledgments

The author thanks all three anonymous referees for their extremely helpful comments and suggestions.

References

Additional Material

Lyasoff.zip contains:
integrals.mx
integrals.txt
CoeffList.mx
CoeffList.txt
Xi.mx
Xi.txt

Available at www.mathematica-journal.com/data/uploads/2011/12/Lyasoff.zip.

About the Author

Andrew Lyasoff is director of the Graduate Program in Mathematical Finance at Boston University. His research interests are mainly in the areas of Stochastic Analysis, Optimization Theory, and Mathematical Finance and Economics.

Andrew Lyasoff
Boston University
Mathematical Finance Program
Boston, MA
alyasoff@bu.edu

Introduction

Given a 2D or 3D object, it is useful to view its cutaway image along an intersecting curve or surface. For example, we could cut an object along planes where , , or are constant by adjusting the PlotRange parameters. For cutting along a general curve or outside a surface, we introduce makehole, a program that removes vertices in Polygon, Line, and Point graphics primitives whose coordinates satisfy a stated condition (e.g., an equation, inequality, or compound inequality). We also introduce transparent and makewindow, which make an opaque surface into a transparent mesh by replacing each Polygon primitive with line segments around the boundary of the polygon. Viewing a region of integration for a triple integral in rectangular coordinates is one application of these programs.

Making Surfaces Transparent

We first introduce transparent. To outline the boundary of a polygon, it repeats the first vertex at the end of each list of vertices and replaces all Polygon heads with Line. Unlike WireFrame in the Shapes package, transparent has adjustable PlotStyle options and works on graphics with either Graphics or Graphics3D heads.

We use Plot3D to draw the plane and to convert the SurfaceGraphics head to Graphics3D. Then we apply transparent and show the result.

Making Holes

Before introducing makehole, we demonstrate its main ideas by removing portions of the following Graphics3D object that lie on or above the plane. The object consists of two polygons.

The condition is expressed as in the following. Also, is used to represent the vertices with either two or three coordinates that satisfy condition removes vertices from the polygons that satisfy the condition, as well as polygons with no remaining vertices.

We view the output in InputForm and observe that and all vertices of the second polygon were removed from the original object because they satisfied .

We note that Polygon primitives with fewer than three remaining vertices will not be displayed by Show.

To demonstrate that this strategy works for more complicated objects, we generate a torus (suppressing the plot using ), then use the same commands to remove the portions on and above .

We show the results with the transparent plane.

The complete makehole program, which follows, removes vertices with two and three coordinates that satisfy a stated condition from within Polygon, Line, and Point primitives.

To demonstrate makehole on graphics with Graphics heads, we use it to remove regions inside three ellipses from the graph of a spiral and three points. In order to remove a union of regions, we use || to represent “or,” so that points that satisfy any of the three inequalities are removed.

Making Windows in Surfaces

The makewindow program replaces Polygon with Line primitives if one of the vertices satisfies a stated condition.

To show how it works, we use the torus and the condition to create a mesh window on portions lying above the corresponding plane. We use PlotStyle options to control the color and thickness of the mesh and to introduce dashing.

Showing the Region of Integration for a Triple Integral

We use makehole and makewindow to draw a region corresponding to the triple integral, . To visualize the region, we draw many of the bounding surfaces and then impose the three conditions , , and from the limits of integration.

We first draw bounding surfaces for the region of integration. The surfaces drawn with Plot3D are converted to a Graphics3D head using . The blue bounding surface and the purple surface are suppressed.

To sketch the surface , we view it as a parametrized surface between cross-sectional curves on planes and . We color the surface red and suppress the plot.

The region of integration is enclosed by the surfaces. We omit the surfaces , , and from the image because the present region does not cross these planes.

To isolate the region of integration, we use makehole to remove all polygons that lie outside the region.

The condition removes all points whose coordinates do not satisfy each of , , and .

To display the region inside the original image, we use makewindow to make everything outside the region transparent.

Conclusion

We have demonstrated programs for making holes and windows in portions of two- and three-dimensional objects which satisfy one or more specified inequalities. Applications include slicing an opaque surface along an intersecting surface and viewing the region of integration for a triple integral.

Editor’s Note

This article describes techniques for working with legacy graphics in Mathematica. The code would need to be modified to work with GraphicsComplex and other extensions to the Mathematica graphics language in Version 6. Mathematica 6 also contains new features that overlap with the functionality described in this article.

to generate wireframe surfaces.

Use Opacity to create partially transparent surfaces.

Use RegionFunction to remove portions of a surface.

Use RegionPlot3D to show a region defined by inequalities.

References

About the Author

Alan Horwitz received a B.S. from Ohio State University in 1980 and an M.A. in 1985 and a Ph.D in 1988 from SUNY at Stonybrook. He is currently a faculty member in the Mathematics Department at Marshall University.

Alan Horwitz
Dept. of Math, Marshall University
Huntington, West Virginia 25755
horwitz@marshall.edu

Introduction

Graphing a curve in the Cartesian plane can be done only in a restricted “window” . If the function to be plotted has a large domain or range, it is practically impossible to get a global view of the curve. This makes it difficult to understand the asymptotic behavior of complicated curves with various kinds of infinities. Furthermore, for most functions (e.g., polynomials of degree greater than four), graphing in a large window loses important details, while graphing in a small window loses the global features.

The remedy is to compactify the plane and represent graphs on the Riemann sphere. The usual method is to map the plane graph to the sphere using the inverse stereographic projection. We prefer a slightly modified version: we smoothly wrap the plane on the sphere using the inverse stereographic projection from the pole . The origin maps to the blue point on the sphere, and the point at infinity maps to the red point .

Benefits of the Method

Asymptotic Behavior

Although the point at infinity cannot be reached, the mapping gives points so close to that it is as if we had reached it. As an illustration, here are the graphs of a polar curve first in the plane (with asymptotes) and then on the sphere.

The default view point shows , the image of the point at infinity.

Here is an animation where the view point goes once around the equator. The blue point zero is the image of the origin in the plane.

To see through the sphere we use the Opacity option.

Sensitivity and Faithfulness

This modified inverse stereographic projection is very sensitive close to the origin: two points near the origin in the plane will appear far apart on the sphere. On the other hand, points near infinity will appear close together, thus showing the asymptotic behavior that is the qualitative property of the curve far away from the origin.

Faithfulness means that mapping the plane curve to the sphere does not alter the shape of the curve. In particular, at , the slopes of the different branches are exactly what they should be. We illustrate this by plotting the function .

Making the interval somewhat larger loses details close to the origin.

Here is the same curve on the sphere, viewed to show zero and then .

The derivatives at are , and, indeed, we see that the curve is vertical near .

Unveiling the True Nature of a Curve

Geometers know that plane geometry is incomplete. We have to look at a curve in the projective plane (i.e., the Riemann sphere) to get complete results.

For example, students in high school are told that hyperbolas have two branches.

In this animation, a hyperbola appears as a one-branched figure-eight curve on the sphere.

The Mapping

We define the mapping using the following picture.

The projection pole is , and is the point corresponding to on . Then and to find the mapping , we solve for .

One solution gives the point and the other solution determines the mapping to the sphere.

The Program: Parametric Curves on the Sphere

The modified inverse stereographic function turns out to be fairly simple.

We define the sphere and the equator, two meridians, a blue zero, and a red for orientation.

To avoid infinities, SphereParametricPlot is based on ScatterPlot3D rather than ParametricPlot3D. In analogy with Plot and PolarPlot, we define the functions SpherePlot and SpherePolarPlot, both in terms of SphereParametricPlot.

The first argument to SphereParametricPlot is a pair of functions. The first argument to SpherePlot or SpherePolarPlot is a single function. The second argument to all three functions is a range specification. A ViewPoint option may be given.

Here is one final example.

Additional Material

ImplicitEquOnSphere.nb

Available at www.mathematica-journal.com/data/uploads/2011/12/ImplicitEquOnSphere.nb.

References

About the Author

Djilali Benayat received a B.Sc. in mathematics (1968) and a Diploma of Advanced Studies (DEA) in functional analysis (1969) from Algiers University, a DEA in algebraic topology (1970) and a “Doctorat de 3e cycle” in -theory (1972) from Strasbourg University, and a D.Sc. in algebraic topology (Theory of Adams Algebras in Stable Homotopy) (1987) from Metz and Algiers universities. Presently he is a professor of mathematics at the École Normale Supérieure of Algiers, Algeria. He has supervised a master’s thesis and three D.Sc.’s in algebraic topology and number theory.

Djilali Benayat
École Normale Supérieure
Vieux Kouba
Algiers, Algeria
dbenayat@yahoo.fr

Introduction

Fluid dynamics is a branch of mechanics concerned with the motion of a fluid continuum under the action of applied forces. The motion and general behavior of a fluid is governed by the fundamental laws of classical mechanics and thermodynamics and plays an important role in such diverse fields as biology, meteorology, chemical engineering, and aerospace engineering. An introductory text on fluid mechanics, such as [1], surveys the basic concepts of fluid dynamics and the various mathematical models used to describe fluid flow under different restrictive assumptions. Advances in computational power and in modeling algorithms during the past few decades have enabled industry to use increasingly realistic models to solve problems of practical geometric complexity. Alternatively, these advances make it feasible to adapt some of the older, simpler models to inexpensive desktop computers.

Aerodynamics is a branch of fluid dynamics concerned primarily with the design of vehicles moving through air. In the not-so-distant past, a collection of relatively simple numerical models, known as panel methods, was the primary computational tool for estimating some of the aerodynamic characteristics of airplanes and their components for cruise conditions. For example, Hess [2] commented in 1990 that at Douglas Aircraft Company, a major design calculation was performed using panel methods approximately 10 times per day.

Panel methods are numerical models based on simplifying assumptions about the physics and properties of the flow of air over an aircraft. The viscosity of air in the flow field is neglected, and the net effect of viscosity on a wing is summarized by requiring that the flow leaves the sharp trailing edge of the wing smoothly. The compressibility of air is neglected, and the curl of the velocity field is assumed to be zero (no vorticity in the flow field). Under these assumptions, the vector velocity describing the flow field can be represented as the gradient of a scalar velocity potential, , and the resulting flow is referred to as potential flow. A statement of conservation of mass in the flow field leads to Laplace’s equation as the governing equation for the velocity potential, . Laplace’s equation is a widely studied linear partial differential equation and is discussed in detail in classical books on applied mathematics such as [3]. It also plays an important role in the theoretical development of several fields, including electrostatics and elastic membranes as well as fluid dynamics.

To solve the problem of potential flow over a solid object, Laplace’s equation must be solved subject to the boundary condition that there be no flow across the surface of the object. This is usually referred to as the tangent-flow boundary condition. Additionally, the flow far from the object is required to be uniform. The results of solving Laplace’s equation subject to tangent-flow boundary conditions provide an approximation of cruise conditions for an airplane.

Using a vector identity, the solution to this linear partial differential equation can be written in terms of an integral over the surface of the object. This boundary integral contains expressions for surface distributions of basic singular solutions to Laplace’s equation. A linear combination of relatively simple singular solutions is also a solution to the differential equation. This superposition of simple solutions provides the complexity needed for satisfying boundary conditions for flow over objects of complex geometry. Panel methods are based on this approach and are described in detail in [4]. Commonly used singular solutions for panel methods are referred to as source, vortex, and doublet distributions. Analogies can be made to other fields of study. The velocity field induced by a point source is analogous to the electrostatic field induced by a point charge. A doublet would be positive and negative charges of equal strength in close proximity. The velocity induced by a line vortex is analogous to the magnetic field induced by a current-carrying wire.

The basic solution procedure for panel methods consists of discretizing the surface of the object with flat panels and selecting singularities to be distributed over the panels in a specified manner, but with unknown singularity-strength parameters. Since each singularity is a solution to Laplace’s equation, a linear combination of the singular solutions is also a solution. The tangent-flow boundary condition is required to be satisfied at a discrete number of points called collocation points. This process leads to a system of linear algebraic equations to be solved for the unknown singularity-strength parameters. Details of the procedure vary depending on the singularities used and other details of problem formulation, but the end result is always a system of linear algebraic equations to be solved for the unknown singularity-strength parameters.

Panel methods are applicable to two- and three-dimensional flows. For flow over a two-dimensional object, the flat panels become straight lines, but can be thought of as infinitely long rectangular panels in the three-dimensional interpretation. For two-dimensional potential flow, the powerful technique of conformal mapping can also be used as a solution procedure. Conformal mapping provides exact solutions for certain airfoil shapes and is useful for validating numerical models.

This article introduces a collection of three packages providing computational tools for the formulation and solution of steady potential flow over an airfoil. In addition to the packages and associated online help for functions defined, examples of model implementation and use are included. Each package is discussed briefly. This is followed by an example of step-by-step implementation of a particular model for a small discretization number with intermediate results displayed. Finally, the steps are assembled into a module representing a particular model, and the lift and pressure distribution on an airfoil are computed. The current version of the software collection is available form the Wolfram Library Archive as Aerodynamics 1.2.

Load Required Packages

Set your working directory and then load the application package needed for this notebook.

Test that packages are loaded.

Nomenclature and Basic Equations for Airfoil Aerodynamics

Much of the nomenclature associated with the theory of lift on an airfoil has made its way into everyday vocabulary, but some terms may be unfamiliar or have more specific meanings than occur in common usage. Also, there are some basic equations used in the example problem that should be mentioned. An introductory book on aerodynamics such as [5] or [6] presents the basic nomenclature and concepts associated with the theory of flight.

The term airfoil is used to denote the cross section, or profile, of a three-dimensional wing (see inset in Figure 2). The chord line of an airfoil is the straight line from the leading edge of the airfoil to the sharp trailing edge; the length of this line is referred to as the chord of the airfoil and is denoted by . The camber line of an airfoil is the locus of points midway between the upper and lower surfaces of the airfoil, measured perpendicular to the camber line. When describing an airfoil in dimensionless variables and in a local coordinate system, the chord of the airfoil is the segment of the axis from 0 to 1. The angle of attack is the angle between the chord line of the airfoil and the uniform onset velocity, and is denoted by .

In incompressible potential flow, the pressure is related to the fluid speed by Bernoulli’s equation, , where , , and are the density, speed, and pressure at a point in the flow field, and the subscript refers to conditions far from the airfoil. The dimensionless measure of pressure is the pressure coefficient, defined by . Combining Bernoulli’s equation and the definition for the pressure coefficient yields a simple equation for the pressure coefficient in terms of the local speed of the fluid, .

Using the aerodynamic sign convention, the circulation of the velocity field around a closed contour is defined by the line integral . The lift force per unit length on an airfoil can be related to the circulation around the airfoil by the Kutta-Joukowski lift theorem . The aerodynamic sign convention used in the definition of circulation is chosen so that positive circulation leads to positive lift. The dimensionless measure for lift on an airfoil is the two-dimensional lift coefficient, . If a dimensionless circulation is defined by , then the lift coefficient is simply twice the dimensionless circulation. The Kutta condition summarizes the primary viscous effect of the flow on the airfoil and establishes the circulation around the airfoil by the simple statement that the flow leaves the sharp trailing edge of the airfoil smoothly.

Input Parameters

The example airfoil for illustrating the implementation of a panel method in this article is a member of the NACA four-digit family of airfoils. Specify the identification number for the airfoil and the angle of attack in degrees.

The first of the four digits in the identification number gives the maximum camber in percent chord, the second digit gives the location of maximum camber in tenths of chord, and the last two digits give the thickness in percent chord.

The discretization process is determined by a discretization number and one of three layout options (ConstantSpacing, CosineSpacing, or HalfCosineSpacing) providing two alternatives to constant spacing of discretization points. Specify small (to illustrate a step-by-step implementation of the example) and large (for computing results) discretization numbers, and a layout option.

Finally, specify a small number used to ensure that collocation points are computed to be outside of the discretization panels representing the airfoil.

If you have installed the software discussed in this article, you can change the input parameters and rerun the notebook for additional results.

Packages

Data Type for Handling Collections of Vectors

The package CartesianVectors defines a data type to simplify the manipulation of large collections of n-dimensional vectors while maintaining packed arrays for efficient computation using machine numbers. The data type CartesianVectors is represented in the format , where vx, vy, and vz are simple or nested lists of the components of the collection of vectors. The data type is designed to enable the manipulation of collections of vectors with notation commonly used for a single vector or for lists. Using a data type also simplifies pattern matching for valid input arguments for exported functions developed in other packages. The properties of the data type are defined by overloading existing Mathematica functions whenever possible. The package contains some exported functions including several functions specific to two-dimensional vectors.

As an example of using the data type, specify two collections of two-dimensional vectors, and compute the collection of displacement vectors from each vector in one group to every vector in the other group. This is a computation common to many n-body problems. The constructor for the data type is MakeCartesianVectors.

Compute the displacement vectors from each point in rB to all points in rA.

Count the number of vectors in the collection of displacement vectors.

Airfoil Geometry

The package AirfoilGeometry provides functions to compute the geometry and discretization of airfoils in support of the construction of numerical models for potential flow over an airfoil. A list of values used for discretization can be specified directly by the user or generated by the function NDiscretizeUnitSegment, which accepts a discretization number, , as its input argument and divides the unit segment into pieces. A layout option for this function allows constant spacing (default), cosine spacing, or half-cosine spacing. Cosine spacing provides finer discretization near the leading and trailing edges of the airfoil compared to constant spacing, and half-cosine spacing provides even finer discretization near the leading edge, but coarser discretization near the trailing edge compared to constant spacing. The function NACA4DigitAirfoil computes a list of thickness and camber properties at the values, and the function AirfoilSurfacePoints computes the collection of vectors locating points on the surface of the airfoil from the list of thickness and camber properties. These points on the surface of the airfoil serve as panel end points for the discretized airfoil. Note that the result is expressed as the data type CartesianVectors as indicated by the arrow separating the lists of components.

Compute a list of panel lengths. Note the use of Mathematica functions that have been overloaded for use with the data type CartesianVectors.

Count the number of panels describing the discretized airfoil.

The functions PanelPoints and PanelNormals are used to locate collocation points at midpanel and outward facing unit normals to the panels. Note that collocation points are displaced a small distance, proportional to the panel length, in the direction of the outward unit normal to ensure that these points are outside the discretized airfoil. This is done in preparation for applying the tangent-flow boundary condition at collocation points.

Figure 1 shows the geometry of the discretized airfoil and the numbering convention for panels. The panels are straight-line segments joining points on the airfoil contour, and panel normals are shown at panel midpoints with the panel number near the head of the arrow. For an airfoil with thickness, the number of panels describing the airfoil is twice the discretization number. This numbering of panels is referred to as the clockwise convention. For a reference airfoil with no thickness (camber line), the number of panels is equal to the discretization number, and the convention is to number panels from leading edge to trailing edge. The airfoil shape is plotted in a local coordinate system with the origin at the leading edge of the airfoil and the axis coincident with the chord line. Lengths are nondimensionalized using the chord of the airfoil, .

Figure 1. Panels and panel normals for NACA 4412 airfoil discretized to six panels.

The online help for this package also includes examples of importing data files for individual airfoils. These are defined by specifying points on the airfoil contour and rearranging the imported data for use with this software. The UIUC Airfoil Data Site, maintained by Michael Selig of the University of Illinois at Urbana-Champaign, contains specifications for over 1500 airfoils [7].

Two-Dimensional Influence Coefficients

When computing the velocity field induced at a point due to a singularity located elsewhere, the velocity can be written as the product of a geometric term (called an influence coefficient) and a measure of the strength of the singularity. For example, consider the velocity induced at an arbitrary field point due to a point source located at the origin of the coordinate system.

Compute the velocity, velocity-potential, and stream-function influence coefficients at the field point due to the singularity using the package functions ICSourcePoint, ICPhiSourcePoint, and ICPsiSourcePoint.

The velocity, velocity-potential, or stream-function at would be obtained by multiplying the appropriate influence coefficient by the strength of the source. Influence coefficients can also be thought of as the velocity, velocity-potential, or stream-function induced by a singularity of unit strength.

The package InfluenceCoefficients contains over thirty functions for velocity, velocity-potential, and stream-function influence coefficients for source, vortex, and doublet singularities commonly used in two-dimensional panel methods. They serve as a tool box for constructing numerical models for two-dimensional potential flow.

Potential-Flow Model Using Vortex Panels of Linearly Varying Strength

Step-by-Step Model Formulation Using Coarse Discretization

The singularity element chosen for this model is the vortex panel of linearly varying strength, which provides a circulation density along the panel of the form, in local coordinates, where is the distance from the “beginning” of the panel. Each singularity panel involves two unknown constants, and .

The boundary condition that the velocity be everywhere tangent to the airfoil contour is discretized to require that the velocity component normal to each panel at the collocation point be zero. Since each vortex panel introduces two unknown strength parameters, application of the tangent-flow boundary condition provides equations and unknowns, where is the number of panels describing the geometry of the discretized airfoil. Continuity of circulation density from one panel to the next and the Kutta condition provide additional equations to complete a system of linear algebraic equations and unknowns. The system of equations can be put into standard form. The terms involving unknowns are collected on the left-hand side of the system of equations and the known quantities are collected on the right-hand side. The result can be written in block-matrix form as

The symbols and represent lists of the unknown constant and linear strength parameters for the vortex panels: represents the projection of the panel influence coefficients associated with on the unit normal vectors, represents the projection of the panel influence coefficients associated with on the unit normal vectors, Math and represent terms in the equations imposing continuity of circulation density between panels and the Kutta condition, and is the projection of the free-stream velocity on unit normals at collocation points.

Use the block-matrix form to write the system of equations as and . Solve the latter system for the list of slope strengths, . Substitute this into the former system of equations to eliminate the slope-strength parameters. The resulting system of equations can be written as . This system of equations can be solved for the list of strength parameters , and then the transformation is used to compute the list of slope parameters . All variables in the following formulation and solution are dimensionless.

Compute the matrix of velocity influence coefficients and project them on the panel normals.

Write the equations expressing the continuity of circulation density between panels and the Kutta condition. The equation expressing the Kutta condition is written to accommodate the different numbering conventions for airfoils with thickness and reference airfoils without thickness.

Form the coefficient matrix for the system of linear algebraic equations to be solved for unknown strength parameters.

Display the matrix in reduced precision to illustrate the coefficient matrix for the full system of equations.

The upper half of the matrix represents normal-component influence coefficients. The first row of the lower half of the matrix represents terms in an equation implementing the Kutta condition and sums the circulation density at the beginning of the first panel and the circulation density at the end of the last panel. Setting this sum to zero imposes zero circulation at the trailing edge of the airfoil. The remaining rows in the lower half of the matrix are coefficients of terms in the equations requiring that the circulation density at the end of one panel be equal to the circulation density at the beginning of the next panel, , where denotes the length of the panel.

Define the transformation matrix to compute the list of slope parameters () from the list of constant parameters ().

Compute the free-stream velocity at collocation points.

Compute the components of the uniform flow normal to panels at collocation points.

Solve the system of equations for the list of constant parameters.

Use the transformation matrix to compute the list of slope parameters.

The lift coefficient for the airfoil can be computed using the Kutta-Joukowski theorem. Recall that the distribution of circulation on a panel in local panel coordinates can be written as , where denotes the distance from the leading edge of the panel. The contribution of each panel to the lift is computed and the results summed over all panels.

In terms of the dimensionless variables used in this example, the contribution by each panel to the lift coefficient is just twice the net circulation associated with the panel, which is obtained by integrating the linear circulation density function, . Compute contributions of each panel to the airfoil lift coefficient.

Sum the two terms for each panel to obtain the list of contributions of each panel to the lift coefficient.

Sum the panel contributions to obtain the airfoil lift coefficient.

The computations in this section illustrate the process of model implementation using a coarse discretization so that intermediate results can be viewed; however, the discretization is too coarse to provide useful results.

Remove names from computer memory, except those with values needed in the subsequent section, which presents an example computation of the pressure distribution and lift coefficient for a specified airfoil using a larger discretization number.

Numerical Model for Fine Discretization

In the following expression, the individual steps for implementing the model in the previous section are collected into a module. Most names for variables have been shortened for conciseness, but should be recognizable.

The results of this computation are the singularity strength parameters for all panels.

This model implementation has been validated by computing the results for a van de Vooren airfoil for which an exact solution is known by the method of conformal mapping. Also, convergence and timing studies have been performed and are available as online help documents in the software collection.

Pressure and Lift Coefficients

Use previously computed influence coefficients to determine the pressure coefficient at collocation points using Bernoulli’s equation.

Lift and pitching moments can be computed from the pressure distribution. For example, the lift coefficient is computed by approximating the integral, , where the integral is over the airfoil contour, is the pressure coefficient, is the outward unit normal to the airfoil surface, and is a unit vector perpendicular to the free-stream velocity in the direction of positive lift. The integral is approximated by considering the pressure coefficient constant over each panel, computing the contribution to lift of each panel, and summing the results.

The lift can also be computed from the circulation distribution as described in the section on step-by-step model formulation.

Figure 2 shows the surface pressure distribution on the airfoil in the conventional manner for such plots. Useful information from such plots include the locations of the stagnation point and the point of minimum pressure, and the severity of the positive pressure gradient on the upper surface.

Figure 2. Pressure coefficient for a NACA 4412 airfoil, discretization 200 panels with HalfCosineSpacing.

Conclusions

A brief summary of some features of a collection of packages that provide computational tools for formulating numerical models for two-dimensional potential flow over an airfoil using panel methods is presented. An example of solving the problem of steady flow over a specific airfoil is given using vortex panels of linearly varying strength and tangent-flow boundary conditions. This example includes the computation of surface pressure distribution and lift coefficient.

Session time for a typical PC indicates the practicality of such computations on low-cost computing systems and suggests the feasibility of going to the next level of modeling. This could include unsteady two-dimensional potential flow, steady three-dimensional potential flow, or including an integral boundary-layer method with the steady two-dimensional potential flow model presented in this article.

References

Additional Material

Fearn.zip

Available at library.wolfram.com/infocenter/MathSource/7785/.

About the Author

While teaching for thirty years at the University of Florida, I often wished for effective computational tools to help students learn aerodynamics. Since retirement, I have started developing software for that purpose. When not playing with Mathematica, I can usually be found summers hiking in the Canadian Rockies or, spring and fall, walking or canoeing in the northern part of Florida with family and friends.

Richard L. Fearn
Professor Emeritus
Department of Mechanical and Aerospace Engineering
University of Florida
Gainesville, FL 32611-6250
rlf@ufl.edu

Introduction

MathCode is a Mathematica add-on that translates a Mathematica program into C++ or Fortran 90. The subset of Mathematica that MathCode is able to translate involves purely numerical operations, and no symbolic operations. In the following sections we provide a variety of examples that show precisely what we mean. The code that is generated can be called and run from within Mathematica, as if you were running a Mathematica function.

There are two important purposes that are served by MathCode. Firstly, the C++/Fortran 90 code runs faster, typically by a factor of about a few hundreds (or about 50 to 100) over interpreted (compiled) Mathematica code, resulting in considerable performance gains, while still requiring hardly any knowledge of C++/Fortran 90 on the part of the user. Secondly, the generated code can also be executed as a stand-alone program outside Mathematica, offering a portability otherwise not possible. You should note, however, that these advantages come at some loss of generality since integer and floating point overflow are not trapped and switched to arbitrary precision as in standard Mathematica code. Here the user is responsible for ensuring an appropriate choice of scaling and input data to avoid such problems. The measurements in this article were made using Mathematica 6.

There are situations in which having a system such as MathCode can be particularly helpful and effective, like when a certain calculation involves a symbolic phase followed by a numerical one. In such a hybrid situation, Mathematica can be employed for the symbolic part to give a set of expressions involving only numerical operations that can be made part of a Mathematica function, which can then be translated into C++/Fortran 90 using MathCode.

In this article, we describe some of the more important features of MathCode. For a more detailed discussion the reader is referred to [1]. For brevity, we simply say C++ when we actually mean C++ or Fortran 90: MathCode can generate code in both C++ and Fortran, although we illustrate C++ code generation in this article.

In Section 2, we show how to quickly get started with MathCode using a simple example of a function to add integers.

Section 3 presents many useful features of MathCode. In Section 3.1, we discuss the way the system works, the various auxiliary files generated and what to make of them, and how to build C++ code and install the executable. We then compare the execution times of the interpreted Mathematica code and the compiled C++ code. This section also illustrates how MathCode works with packages.

Section 3.2 briefly makes a few points about types and type declarations in MathCode. There are two ways to declare argument types and return types of a function mentioned in this section.

In Section 3.3, we show how to generate a stand-alone C++ executable. This executable can be run outside of Mathematica. We illustrate how to design a suitable main program that the executable runs.

It should be emphasized that MathCode can generate C++ for only that subset of Mathematica functions referred to as the compilable subset. Section 3.4 gives a sample of this subset, while Section 3.5 presents three ways to extend it with the already-available features of MathCode: Sections 3.5.1 through 3.5.3 discuss, respectively, symbolic expansion of function bodies, callbacks to Mathematica, and handling external functions. Each of these extensions has its own strengths and limitations.

Section 3.6 discusses common subexpression elimination, a feature that is aimed at enhancing the efficiency of generated code.

Section 3.7 presents some shortcuts available in MathCode to extract and make assignments to elements of matrices and submatrices, while Section 3.8 is about array declarations.

In Section 4, we present several examples of effectively using MathCode. Section 4.1 provides a summary of the examples.

Section 4.2 discusses an essentially simple example, that of computing the function over a grid in the - plane, but done in a somewhat roundabout manner so as to illustrate various features of MathCode.

Section 4.3 discusses an implementation of the Gaussian elimination algorithm [2] to solve matrix systems of the type , where is a square matrix of size and (the solution vector) and are vectors of size . In this section, we make a detailed performance study by computing the solution of a matrix system by turning on a few compilation options available in MathCode, and also make comparisons with LinearSolve.

In Section 4.4, we show how to call external libraries and object files from a C++ program that is automatically generated by MathCode. We take the example of a well-known matrix library called SuperLU [3], and demonstrate how to solve, using one of its object modules, a sparse matrix system arising from a partial differential equation.

The MathCode User Guide that is available online discusses more advanced aspects, like a detailed account of types and declarations, the numerous options available in MathCode with the aid of which the user can control code generation and compilation, and other features. We refer interested readers to [1].

In Section 5, we summarize the salient aspects of MathCode and discuss the kinds of applications for which MathCode is particularly useful. We conclude the article with a brief summary of various points made. The first version of MathCode, released in 1998, was partly developed from the code generator in the ObjectMath environment [4, 5]. The current version is almost completely rewritten and very much improved.

2. Getting Started with MathCode

2.1. An Example Function

In this section we take the reader on a quick tour of MathCode using the simple example of a function to add integers.

The following command loads MathCode.

MathCode works by generating a set of files in the current directory (see Section 3.1). We can set the directory in the standard way as follows; here, $MCRoot is the MathCode root directory. The user can, however, use any other directory to store the files.

Let us now define a Mathematica function sumint to add the first natural numbers.

Note that the body of this function has purely numerical operations, like incrementing the loop index , adding two numbers, and assigning the result to a variable.

2.2. Declaration of Types

We must now declare the data types of the parameter and the local variables and ; we must also specify the return type of the function. We do this using the function Declare that MathCode provides.

Note that does not mean ; the function Declare creates an environment in which this is interpreted as a type declaration, that is, an integer variable is being declared in the example. The type Integer is translated to a native C int type, and the type Real to a native C double type.

2.3. C++ Code

To generate and compile the C++ code, we execute the following command.

Since we have not specified the context of sumint, its default context is Global. We could, therefore, have simply executed the following command instead.

With the following command, we seamlessly integrate an external program with Mathematica.

We can now run the external program in the same way that we would execute a Mathematica command.

If we want to run the Mathematica code (and not the generated C++ code) for sumint, we must first uninstall the C++ executable.

Now the Mathematica code for sumint will run.

3. A Tour of MathCode

3.1. How the MathCode System Works

MathCode works by generating a set of files in the home directory. In the example of sumint, the default context is Global and the files generated by MathCode are: Global.cc (the C++ source file), Global.h and Global.mh (the header files), Globaltm.c, Global.tm and Globalif.cc (the -related files that enable transparently calling C++ versions of the function sumint from Mathematica), and Globalmain.cc, which contains the function main( ) needed when building a stand-alone executable.

We can also create a package (let us call it foo) that defines its own context foo instead of the default context Global. See Figure 1 for a block diagram of the way the overall system works. The MathCode code generator translates the Mathematica package to a corresponding C++ source file foo.cc. Additional files are automatically generated: the header file foo.h, the MathCode header file foo.mh, the MathLink-related files footm.c, foo.tm, foo.icc, and fooif.cc, which enable calling the C++ versions from Mathematica, and foomain.cc, which contains the function main that is needed when building a stand-alone executable for foo (see Section 3.3). The generated file foo.cc created from the package foo, the header file foo.h, and additional files are compiled and linked into two executables. In the case of MathCode F90, Fortran 90 is generated and a file foo.f90 is created. No header file is generated in that case since Fortran 90 provides directives for the use of module. External numerical libraries may be included in the linking process by specifying their inclusion (Sections 3.5.3 and 4.5). The executable produced, foo.exe, can be used for stand-alone execution, whereas fooml.exe is used when calling on the compiled C++ functions from Mathematica via MathLink.

Figure 1. Generating C++ code with MathCode for a package called foo.

Let us see how to work with a package again using the same sumint example.

If we are compiling the package foo using MathCode, we also need to mention MathCodeContexts within the path of the package.

We define the function sumint

and close the context foo.

We next declare the types, and then build and install as before.

Again, since the package foo has been defined, it is the default context, and so we could simply have executed the following command.

To run the executable from the notebook, we must install it.

Now the following command runs the C++ executable fooml.exe. The call to sumint via MathLink is executed 1000 times. The timing measurement includes MathLink overhead, which typically for small functions is much more than the execution time for the compiled function. This can be avoided if the loop is executed within the external function itself, as in the example in Section 4.2.5.

Here is the C++ code that was generated.

Note that the function sumint appears as foo_Tsumint in the generated code. This is because the full name of the function is in fact foo`sumint, and MathCode replaces the backquote “” by “_T” in the C++ code.

To run the Mathematica function (and not its C++ equivalent) sumint, we must use the following command to uninstall the C++ code.

Now it is the Mathematica code that runs when you execute sumint.

You can see that the C++ executable together with the MathLink overhead runs about 15 times faster than the Mathematica code. The factor by which the performance is enhanced is problem dependent, however. The performance of the Mathematica code could also have been improved by using the built-in Compile function. In Section 4 we will see many more examples, some quite involved, where we get a range of performance enhancements, also including usage of the Compile function.

We clean up the current directory by removing the files automatically generated by MathCode.

3.2. Types and Declarations

To be able to generate efficient code, the types of function arguments and return values must be specified, as we have seen in the preceding examples. The basic types used by MathCode are

Arrays (vectors and matrices) of these types can also be declared.

Type declarations can be given in two different ways:

The latter construction can be useful if you want to separate already existing Mathematica code with the type information needed to be able to generate C++ code using MathCode.

3.3. Generating a Stand-Alone Program

So far we have only seen examples in which the installed C++ code can be run within Mathematica. However, we can also produce a stand-alone executable. This offers a degree of portability that can be useful in practice.

To illustrate, we take the same example function sumint that we discussed in the previous sections. The sequence of commands is very much as in the previous section, except for the option for the MathCode function MakeBinary, and an appropriate option MainFileAndFunction for the function SetCompilationOptions immediately after BeginPackage. Figure 2 illustrates the process of building the two kinds of executable, namely fooml.exe and foo.exe (on some systems foomain.exe) from a package called foo.

Figure 2. Building two executables from the package foo, possibly including numerical libraries.

The option MainFileAndFunction is used to specify the main file. The functions defined in Mathematica must have the prefix Global_T (packagename_T in general) to be recognized in the main file.

Now we are ready to generate and compile the C++ code for the package foo. We can do this in two ways: we can either employ the MathCode function BuildCode, as in the previous examples, or first execute CompilePackage (which generates the C++ source and header files) and then the function MakeBinary (which creates the executable).

The last command generates the stand-alone executable foo.exe that can be executed from a command line, or, alternatively, by using the Mathematica function Run.

If you desire, you can, in addition to the stand-alone executable foo.exe, also generate fooml.exe that can be run from within Mathematica, just like before.

Now the following command runs the C++ program foo.cc.

3.3.1. Generating a DLL

Here we briefly mention the possibility of generating a DLL, without giving a full example. To generate a DLL from a package, you have to write a file containing one simple wrapper function in order to make a generated function visible outside the DLL. You write a wrapper function for each generated function. The flags used are as follows:

Here “ext.cpp” is a C++ file with wrapper functions, and “” is a flag for the Visual C++ linker. For other C++ compilers this procedure is not automatic and requires several operating system commands, but the wrapper functions are not needed.

3.4. The Compilable Subset

MathCode generates C++ code for a subset of Mathematica functions, called the compilable subset. The following items give a sample of the compilable subset. For a complete list of Mathematica functions in the compilable subset, see [1].

• Statically typed functions, where the types of function arguments and return values are given by the types discussed in Section 3.2
• Scoping constructs: , , , ,
• Procedural constructs: , , , ,
• Lists and tables: , , , ,
• Size functions: ,
• Arithmetic and logical expressions, for example: +, -, *, /, ==, !=, >, !, &&, ||, and so forth
• Elementary functions and some others, for example: , , , , , , , ,
• Constants: True, False, E, Pi
• Assignments: :=, =
• Functional commands: ,
• Some special commands: ,

Functions not in the compilable subset can be used in external code by callbacks to Mathematica (see Section 3.5.2 for an example).

Examples of functions that are not a part of the compilable subset include: , , , , , .

These functions can be used if Mathematica can evaluate them at compile time to expressions that belong to the compilable subset. In general, Mathematica functions that perform symbolic operations are not in the compilable subset. Also, many functions in the subset are implemented with limitations, that is, more difficult cases are not always supported. However, MathCode currently provides several ways to extend the compilable subset, as we discuss in the next section.

3.5 Ways to Extend the Compilable Subset

3.5.1. Symbolic Expansion of Function Bodies

Functions not entirely written using Mathematica code in the compilable subset, but whose definitions can be evaluated symbolically to expressions that belong to the compilable subset, can be handled by MathCode.

Generate C++ code and compile it to an executable file.

The option EvaluateFunctions tells MathCode to let Mathematica expand the function body as much as possible. Everything works fine because the result belongs to the compilable subset.

The generated executable is connected to Mathematica:

3.5.2. Callbacks to Mathematica

Consider the following function whose definition includes the Zeta function, which does not belong to the compilable subset.

Let us plot the function:

We now make the declarations:

These declare statements do not change the way Mathematica computes the function.

Let us now generate C++ code and compile it to an executable file. The option CallBackFunctions tells MathCode which functions have to be evaluated by Mathematica. As a result, although the function Zeta is not in the compilable subset, an executable is still generated and communicates with the kernel to evaluate Zeta.

The generated executable is connected to Mathematica:

Now it is the external code that is used to compute the function:

In this case the external code calls Mathematica when the Zeta function has to be evaluated. After the evaluation the computation proceeds in the external code.

Note that it is the installed code for the function f that is executed above, and not the original Mathematica function. In the installed code, the argument of f must be real, according to our declaration. As a result, , in which we pass a rational number as an argument, is left unevaluated.

We again plot the function, but this time using the external code to evaluate it:

3.5.3. External Functions

We can have references to external objects in C++ code generated by MathCode. Let us consider three very simple external functions that compute , , and to illustrate the idea. These must be defined as follows in an external source file that must be in the working directory.

Observe here that each function definition, which is in C language syntax, is followed by a “wrapper” that enables MathCode to recognize the object as external. We can then create an object file corresponding to these functions and link the object as follows.

We define a function to create a list of numbers using the external functions.

We now compile the package. Since this is a very small example, we do not bother to create a special package for the code.

Let us now create the MathLink binary; to do this when there are external functions, we must specify the option NeedsExternalObjectModule as follows.

Here, as we noted above, external1 and external2 represent the external object modules external1.o and external2.o. Install the MathCode-compiled code so it is called using MathLink.

When we make the following plot, it is the external code for extsqr, extexp, and extoscillation that is used.

3.6. Common Subexpression Elimination

Consider the following function whose definition contains a number of common subexpressions (e.g., and ).

There are very efficient algorithms to evaluate functions containing common subexpressions. The basic idea is to evaluate common subexpressions only once and put the results in temporary variables.

Now we generate C++ code using MathCode and run it.

MathCode does common subexpression elimination (CSE) when the option EvaluateFunctions is given to or . This basic strategy could be further improved for special cases in future versions of MathCode. Moreover, since mathematical expressions are intrinsically free of side effects and do not have a specific evaluation order, the CSE optimization may change the order of computing subexpressions if this improves performance. Changing the order can sometimes have a small influence on the result when floating-point arithmetic is used.

We take a look at the generated C++ file.

Note how the computation of the function has been divided into small subexpressions that are evaluated only once and then stored in temporary variables for future use. This gives very efficient code for large functions. The speed enhancement of roughly 150% brought about by CSE in this example is not appreciable because the example itself is rather small.

3.7. Extended Matrix Operations

When dealing with matrices, it is very convenient to have a short notation for part extraction. MathCode extends the functionality of or to achieve this.

Consider the following matrix:

We can extract rows 2 to 4 as follows, with the shorthand available in MathCode.

We can extract the elements in all rows that belong to column 3 and higher:

We can assign values to a submatrix of A.

All these operations belong to the compilable subset and can result in compact code. Note: denotes the same Mathematica computation as , and is equivalent to .

3.8. Array Declaration and Dimension

In this subsection, we give a few examples of array declarations. There are two main cases to consider.

• Arrays that are passed as function parameters or returned as function values, where the actual array size has been previously allocated
• Declaration of array variables, usually specifying both the type and the allocation of the declared array

There are five allowed ways to specify array dimension sizes in array types for function arguments and results.

• Integer constant dimension sizes, for example:
• Symbolic-constant dimension sizes, for example:
• Unknown dimension sizes with unnamed placeholders, for example:
• Unknown dimension sizes with named placeholders, for example:
• Unknown dimension sizes with variables as dimension sizes, for example:

The dimension sizes can be constant, in which case the size information is part of the type. Alternatively, the sizes are unknown and thus fixed later at runtime when the array is allocated. Such unknown dimension sizes are specified through named (e.g., n_) or unnamed (_) placeholders.

All arrays that are passed as arguments to functions have already been allocated at runtime. Thus, their sizes are already determined. These sizes might, however, be different for different calls. Therefore it is not allowed to specify conflicting dimension sizes through integer variables (e.g., ) in array types of function parameters or results, as can be done for ordinary declared variables. Only constants and named, or unnamed, placeholders are allowed.

We now give examples of the five different ways of specifying array dimension information in variable declarations. The examples show a global variable declaration using Declare, but the same kinds of declarations can also be used for local declarations in functions.

The fifth case is where sizes are specified through integer variables. This is needed to handle declaration and allocation of arrays for which the sizes are not determined until runtime.

Integer variables, such as n and m, are assumed to be assigned once; that is, their values are not changed after the initial assignment, so that the declared sizes of allocated arrays are kept consistent with the values of those variables. This single-assignment property is not checked by the current version of the system, however. Thus, the user is responsible for maintaining such consistency.

4. Application Examples

4.1. Summary of Examples

In the following we present a few complete application examples using MathCode. The first example application is a small Mathematica program called SinSurface (Section 4.2), which has been designed to illustrate two basic modes of the code generator: compiling without symbolic evaluation (the default mode, in which the function body is translated into C++ as it is), and compilation preceded by symbolic expansion, which is indicated by setting the option (the function body is expanded using symbolic operations, simplified, and then translated).

The second example, presented in Section 4.3, is an implementation of the Gaussian elimination procedure to solve a linear algebraic system of equations (see any standard text on numerical techniques for a discussion of the procedure, e.g., [2]). Here we compile generated C++ code with various options and do a detailed performance analysis.

In Section 4.4, we discuss the example of SuperLU, an external library [3] that performs efficient sparse matrix operations. We give an example of a program useful in solving partial differential equations that calls the SuperLU library and some of its object modules to solve a matrix equation of the type , where is a very sparse square matrix.

4.2. The SinSurface Application Example

Here we describe the SinSurface program example. The actual computation is performed by the functions calcPlot, sinFun2, and their helper functions. The two functions calcPlot and sinFun2 in the SinSurface package will be translated into C++ and are declared together with a global array xyMatrix.

The array xyMatrix represents a grid on which the numerical function sinFun2 will be computed. The function calcPlot accepts five arguments: four of these are coordinates describing a square in the - plane and one is a counter (iter) to make the function repeat the computation as many times as necessary in order to measure execution time. For each point on a grid, the numeric function sinFun2 is called to compute a value that is stored as an element in the matrix representing the grid.

4.2.1. Introduction

The SinSurface example application computes a function (here sinFun2) over a two-dimensional grid. The function values are stored in the matrix xyMatrix. The execution of compiled C++ code for the function sinFun2 is over 500 times faster than evaluating the same function interpretively within Mathematica.

The function sinFun2 computes essentially the same values as , but in a more complicated way, using a rather large expression obtained through converting the arguments into polar coordinates (through ArcTan) and then using a series expansion of both Sin and Cos, up to 10 terms. The resulting large symbolic expression (more than a page) becomes the body of sinFun2, and is then used as input to CompileEvaluateFunction to generate efficient C++ code. The symbolic expression and the call to CompileEvaluateFunction is initiated by using the EvaluateFunctions option.

4.2.2. Initialization

We first set the directory in which MathCode will store the auxiliary files, the C++ code, and executable, and then load MathCode.

The SinSurface package starts in the usual way with a BeginPackage declaration that references other packages. MathCodeContexts is needed in order to call the code generation related functions.

Next we define possibly exported symbols. Even though it is not necessary here, we enclose these names within … as a kind of context bracket, since this can be put into a cell, which can be conveniently re-evaluated by itself if new names are added to the list.

Now we set compilation options as follows. This defines how the functions and variables in the package should be compiled to C++. By default, all typed variables and functions are compiled. However, the compilation process can be controlled in a more detailed way by giving compilation options to CompilePackage or via SetCompilationOptions. For example, in this package the function sinFun2 should be symbolically evaluated before being translated to code, because it contains symbolic operations; the functions sin, cos, and arcTan should not be compiled at all, because they are expanded within the body of sinFun2. The remaining typed function, calcPlot, will be compiled in the normal way.

4.2.3. The Body of the SinSurface Package

We begin the implementation section of the SinSurface package, where functions are defined. This is usually private, to avoid accidental name shadowing due to identical local variables in several packages.

Declare public global variables and private package-global variables:

Taylor-expanded sin and cos functions called by sinFun2 are now defined, just for the sake of the example, even though such a series gives lower relative accuracy close to zero. A substitution of the symbol z for the actual parameter x is necessary to force the series expansion before replacing with the actual parameter.

Define arcTan, which converts a grid point to an angle, called by sinFun2:

sinFun2 is the function to be computed and plotted, called by calcPlot. It provides a computationally heavy (series expansion) and complicated way of calculating an approximation to . This gives an example of a combination of symbolic and numeric operations as well as a rather standard mix of arithmetic operations. The expanded symbolic expression, which comprises the body of sinFun2, is about two pages long when printed.

Note that the types of local variables to sinFun2 need not be declared, since setting the EvaluateFunctions option will make the whole function body be symbolically expanded before translation.

Note also that a function should be without side effects in order to be symbolically expanded before final code generation. For example, there should be no assignments to global variables or input/output, since the relative order of these actions when executing the code often changes when the symbolic expression is created and later rearranged and optimized by the code generator.

The function calcPlot calculates data for a plot of sinFun2 over a grid, which is returned as a array.

4.2.4. Execution

We first execute the application interpretively within Mathematica, and then use Compile on the key function and execute the application again. Then we compile the application to C++, build an executable, and call the same functions from Mathematica via MathLink.

Let us first do the Mathematica evaluation and plot.