5. Discussion of the Results Using Mathematica 5.1

Darren Glosemeyer

Kernel Developer Kernel Technology Group Wolfram Research, Inc. darreng@wolfram.com

The following results were obtained in Mathematica 5.1 on an Pentium 4 Windows XP computer. Version 5.1 does a little better on some examples than 5.0.

The following comments are for the examples for which problems were noted in 5.0. Results for the other examples in the NIST benchmarks still give correct results in 5.1. For most examples, just the parameter tables are shown; however, the values were also checked against the benchmark.

For three examples, differences between Mathematica's results and the benchmark result are attributable to numerical error in machine-precision arithmetic, numerical error in the NIST data, or a combination of those two sources. In the other examples, NonlinearRegress either gets a correct result with default settings or obtains a correct result with an additional option in 5.1.

Wampler1

The numerical error in this example is a result of numerical error in machine-precision computation and can be expected.

Significance arithmetic (variable-precision arithmetic) can be used to get results of a desired precision or accuracy. Regress can be forced to use significance arithmetic by applying SetPrecision to the data. Here is an example with precision 20 data with the result displayed at machine-precision. Because there is no random error in the original data, division by 0 will result in error messages when attempting to compute the t statistics and p-values.

Wampler2

In this example, there are two sources of numerical error: rounding in the original data and machine-precision arithmetic. While the NIST data was generated from a polynomial expression, the data is only given to five digits of accuracy, resulting in some roundoff or truncation error in the data.

As in the case of Wampler1, more precise results can be obtained by using higher precision input for the regression.

However, increasing the precision of the input cannot make up for the fact that there is some numerical error in the original data. As the precision is increased, the significance of the roundoff or truncation error in the data from the NIST will show up as small deviations from the benchmark results.

MGH17

Each value in the first set of starting points is about two orders of magnitude from the optimal value. Overflows are quickly encountered using the default Levenberg-Marquardt method. Despite the tough starting values, good results can be obtained using Newton's method instead of the default Levenberg-Marquardt method.

As in 5.0, this example converges to the correct result from the second set of starting values with default options.

Lanczos1

With both sets of starting points, this example converges to the expected parameter estimates with default options in 5.1. The SEs and values are on the same order as the benchmark values, which is reasonable given the magnitudes of the SEs and and the fact that the benchmark data is given to 13 digits. For the first starting points, the SEs agree with the benchmark in at least the first digit. For the second starting points, the SEs agree with the benchmark in at least the first two digits.

With an increased AccuracyGoal, values matching the benchmark to three digits can be obtained.

Lanczos2

With both sets of starting points, this example converges to the expected result with default options in 5.1.

MGH09

This example converges a little faster in 5.1 than in 5.0. Convergence is obtained for the first starting points with MaxIterations->400, as opposed to MaxIterations->600 in 5.0.

Correct results are obtained from the second set of starting points with the default option settings, as is the case in 5.0.

MGH10

Convergence from each set of starting values was obtained with MaxIterations->300 in 5.1. This is an improvement for the first set of points, for which MaxIterations->500 was used to obtain the benchmark result in 5.0.

Bennett5

For this example, convergence in 5.1 can be obtained with half the iterations used in 5.0. From the first set of starting values, correct results are obtained with MaxIterations->750. From the second set of starting values, setting MaxIterations->200 is sufficient to obtain the benchmark results.