4. Discussion of the Results Using Mathematica 5.0

(a) Linear Regression

There is really not much to say about the performance of Mathematica in doing linear regression. Mathematica is remarkably accurate with only one negligible but puzzling anomaly: In the cases of Wampler1 and Wampler2, both fifth degree polynomials with artificially constructed data, the true SEs are 0, as is the correct estimate of . However, Mathematica delivers very small numbers of the order of to . While this is not a practical problem of any significance, some explanation is needed in view of Mathematica's otherwise stellar performance. I would attribute it to Mathematica's use of variable-precision floating-point arithmetic [47], but the matter clearly requires further looking into.

(b) Nonlinear Regression

Nonlinear regressions present a greater challenge. In all but five out of 27 difficult nonlinear regression problems, Mathematica converged to the benchmark values within the default number of iterations, MaxIter=100 for both of the two start values given by the NIST. But there were serious problems with five of the problems, including one case in which no convergence was obtained even starting from the final benchmark result. However, in all five cases, informative error messages were given by Mathematica and no spurious results were presented. Here is a summary.

MGH17

As noted earlier, this is an exceptionally difficult problem given the observations on and . Start 1 produced an overflow in the computation. There were no problems, however, encountered in obtaining the benchmark results from start 2.

Lanczos2

Note that this is the same functional form as MGH17 without a constant term. In both examples the data has been generated by the NIST. No convergence was obtained for the default number of iterations from start 1 with an error message to that effect. Convergence was, however, obtained for MaxIter=200 from start 1 and from start 2 for MaxIter=100 to the correct benchmark results.

MGH09

No convergence was obtained from start 1 even by increasing the number of iterations to 500. The following error message suggests that the function is extremely flat, at least in one direction.

Convergence to the benchmark results for MaxIter=100 from start 2 was obtained without any problems.

MGH10

No convergence was obtained from start 1 or from start 2 for 10,000 iterations. Starting from the benchmark results produced an abnormal program termination with a message that the fitting algorithm failed.

Bennett5

For start 1, convergence was obtained only for 10,000 iterations. Although the benchmark coefficients and standard errors were obtained, the benchmark estimate of residual variance was not. For start 2, convergence to the benchmark results within 100 iterations was obtained.

Note that Mathematica's NonlinearRegress is extremely fast. The time to run a regression once the data is set up and the command given is imperceptible for the default number of 100 iterations. Even for 10,000 iterations, the running time is barely perceptible on a Dell Workstation, PWS 340, with an Pentium chip.

Mathematica's NonlinearRegress is an extremely fast and accurate algorithm when it works. When it does not, it delivers error messages rather than spurious results. Presently, there is no provision in the NonlinearFit package for obtaining reasonably good start values.