This book is about tools for finding parameters of functions that describe phenomena or systems where these parameters are implicated in ways that do not allow their determination by solving sets of linear equations. Furthermore, it is about doing so with R.

R is a computer language and suite of libraries intended primarily to be used for statistical and mathematical computations. It has a strong but quite small base system that is rich in functions for scientific computations as well as a huge collection of add-in packages for particular extra problems and situations.

Among both the base and add-in tools are facilities for finding the “best” parameters of functions that describe systems or phenomena. Such tools are used to fit equations to data or improve the performance of devices or find the most efficient path from A to B, and so on. Some such problems have a structure where we only need solve one or two sets of linear equations. Others—the subject of this book—require iterative methods to approximate the solution we desire. Generally we refer to these as nonlinear parameters. A formal definition will be given later, but the tools we shall discuss can be applied anyway.

Sometimes the problems involve constraints as well as objective functions. Dealing with constraints is also a subject for this book. When there are many constraints, the problem is usually called mathematical programming, a field with many subspecialties depending on the nature of the objective and constraints. While some review of the R tools for mathematical programming is included, such problems have not been prominent in my work, so there is less depth of treatment here. This does not imply that the subject should be ignored, even though for me, and also to some extent for R, mathematical programming has been a lower priority than nonlinear function minimization with perhaps a few constraints. This reflects the origins of R in the primarily statistical community.

The topics in this book are most likely to be of interest to researchers and practitioners who have to solve parameter determination problems. In some cases, they will be developing specialized tools, such as packages for maximum likelihood estimation, so the tools discussed here are like the engines used by car designers to prepare their particular offering. In other cases, workers will want to use the tools directly. In either case, my aim is to provide advice and suggestions of what is likely to work well and what should generally be avoided.

The book should also interest the serious users of composite packages that use optimization internally. I have noticed that some such packages call the first or most obvious tool in sight. I believe the user really should be offered a choice or at least informed of what is used. Those who stand most to benefit from the book in this way are workers who have large computationally intensive problems or else awkward problems where standard methods may “fail”—or more likely simply terminate before finding a satisfactory solution.

Thus my goal in preparing this book is to provide an appreciation of the tools and where they are most useful. Most users of R are not specialists in computation, and the workings of the specialized tools are a black box. This can lead to mis application. A lumberjack's chain saw is a poor choice for a neurosurgeon's handiwork. But a system like R needs to have a range of choices for different occasions. Users also need to help in making appropriate choices.

I also hope to provide enough simplified examples to both help users and stimulate developers. While we have a lot of tools for nonlinear parameter determination in R, we do not have enough well-organized guidance on their use or ways to effectively retire methods that are not good enough.

As a developer of such tools, I have found myself confronted on a number of occasions by users telling me that a colleague said my program was very good, but “it didn't work.” This has almost always been because the mode of use or choice of tool was inappropriate. Ideally, I aim to have the software identify such misapplication but that is a very difficult challenge that I have spent a good deal of my career addressing.

That said, I must note that many many items that appear in this book were developed or adjusted during its writing. The legacy of a very large and widely used open source system leaves much detritus that is not useful and many inconvenient loose ends. Some have been addressed as the writing progressed.

A word about the structure of this book. Readers who read cover to cover will find a certain amount of duplication. I have tried to make each chapter somewhat self-contained. This does not mean that I will not refer the reader to other chapters, but it does mean that sometimes there is repetition of ideas that are also treated elsewhere. This is done to make it easier to read chapters independently.

Any book, of course, owes a lot to people other than the author. I cannot hope to do justice to every person who has helped with ideas or support, but the following, in no particular order, are the names of some people who I feel have contributed to the writing of this book: Mary Nash, Ravi Varadhan, Stephen Nash, Hans Werner Borchers, Claudia Beleites, Nathan Lemoine, Paul Gilbert, Dirk Eddelbeuttel, John Fox, Ben Bolker, Doug Bates, Kate Mullen, Richard Alexander and Gabor Grothendieck.

John C. Nash