*************************************************************************** A Brief Discussion of current and future Nonlinear Modeling Capability The goal is to implement many of the techniques discussed in "The Elements of Statistical Learning," By Robert Hastie, Robert Tibshirani and Jerome Friedman second edition 2009. Jer 1 April 2010 B34S 8.11E has added extensive new nonlinear capability including Random Forest Modeling, Projection Pursuit, Exploritory Projection Pursuit, Recursive Discriminant analysis and LOESS models. 20 May 2006 MARS Modeling Capability in B34S using Fortran Routines developed independently of the J. Friedman (version 3.5) Fortran that was released in 1991 (now trademarked as MARS(tm)), was initially released as of 16 December 2005. This new capability has proved to be quite stable. A number of enhancements such as MARS_VAR have now been released. The new MARS capability (MARSPLINE, MARS_VAR) is based, on extensions of the excellent GPL library of MARS modeling routines available under R(r) as a contributed package. The MARS modeling capability in B34S based on the Friedman MARS(TM) 1991 (version 3.5) Fortran code, or any later version of this Fortran code, is not being distributed commercially at this time. The new MARS modeling capability can be compared and contrasted with ACE, GAM and Pispline approaches to nonlinear modeling. A wide range of nonlinear diagnostic tests and sample time series problems are available. The new MARS modeling capability appears to be especially useful in the modeling of higher order systems. Using the B34S nonlinear modeling capability, combination forecasting can be dynamically built in the field using user subroutines programmed in the B34S matrix language. For example MARS / ARIMA models can be estimated. The developer of B34S thanks T. J. Hastie and R. J. Tibshirani and the other developers responsible for porting their excellent code as an open source GPL contribution to the R(r) programming language. Modifications and extensions to their library will be made available for stand-alone use together with a stand alone library of B34S utility routines that have been placed under the GPL 2 license. With the addition of LINPACK routines, users can make further improvements in their stand-alone applications of MARS models using variants of this most excellent GPL 2 code. Examples illustrating these improvements will be discussed in the third edition of "Diagnostically Specifying and Testing Econometric Models" by Houstion H. Stokes which is under development. The 1997 edition of this book contains some preliminary discussion of some of the nonlinear modeling issues and at that time focused on results obtained with the now dated MARS(tm)-1991 Friedman Fortran code.. A major rewrite of B34S nonlinear modeling documentation is underway to reflect the new developments. The developer of B34S, which can dynamically link with R(r) to exploit further applications not currently available in B34S, wishes to thank all developers and researchers in the area of nonlinear modeling who have made their Fortran available under the GPL license. It is the opinion of the developer of B34S that the MARS capability currently in the 2.2.1 or later versions of R(r) is of the highest quality. It has been found that this implementation, on average, will produce substantially superior models than were possible using the older (1991) Friedman MARS(TM) code. This is being documented in "white papers." A major advantage of the R(r) code is that it is GPL so the user can see what is being calculated, but most important, so the user can make improvements such as has been done in the B34S implementation. The most excellent book "Extending the Linear Model with R" by Julian Faraway (2006) provides many examples of using this GPL code The way the MARSPLINE command and MARS_VAR commands have been implemented in B34S allows the user to extract the MARS knot vectors for use in other econometric applications such as GARCH etc. This is possible because of the way that the GAM, ACE and MARS methods actually work. In stage one the transformed right hand side vectors are built. In the final estimation stage, the QR approach is used to run OLS to obtain coefficients and significance tests on these coefficients. A logical next step is to extract the transformed vectors for input into other models, plots etc. The way B34S is designed, this can even be done inside a nonlinear estimation since the B34S nonlinear estimation commands require users branch to a subroutine to resolve the model and calculate the objective function. A MARS step can be nested inside this step, if desired, to allow true "joint" estimation. The MARS_VAR command allows one or more left hand side variables. The procedure generates right hand side vectors that are significant for one or more left hand side series. If lags are passed to ACE, GAM, MARSPLINE or MARS_VAR, then possibly different thresholds are estimated. Thus in a stock market application, for example, a 10% fall yesterday might have an effect on the market today while a 10% fall 6 periods ago might have little or no effect. More applications of these powerful new estimation tools are actively under study. A number of "white papers" showing applications of these techniques on known problems are under development as are "front end" GUI third party B34S enhancements. ***************************************************************