Cover image for Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition

Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition

Author Heinz Neudecker , Jan R. Magnus
Release Date: 2019/03/01
ISBN: 9781119541202
Topic:
0%
35
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

A brand new, fully updated edition of a popular classic on matrix differential calculus with applications in statistics and econometrics

This exhaustive, self-contained book on matrix theory and matrix differential calculus provides a treatment of matrix calculus based on differentials and shows how easy it is to use this theory once you have mastered the technique. Jan Magnus, who, along with the late Heinz Neudecker, pioneered the theory, develops it further in this new edition and provides many examples along the way to support it.

Matrix calculus has become an essential tool for quantitative methods in a large number of applications, ranging from social and behavioral sciences to econometrics. It is still relevant and used today in a wide range of subjects such as the biosciences and psychology. Matrix Differential Calculus with Applications in Statistics and Econometrics, Third Edition contains all of the essentials of multivariable calculus with an emphasis on the use of differentials. It starts by presenting a concise, yet thorough overview of matrix algebra, then goes on to develop the theory of differentials. The rest of the text combines the theory and application of matrix differential calculus, providing the practitioner and researcher with both a quick review and a detailed reference.

  • Fulfills the need for an updated and unified treatment of matrix differential calculus
  • Contains many new examples and exercises based on questions asked of the author over the years
  • Covers new developments in field and features new applications
  • Written by a leading expert and pioneer of the theory
  • Part of the Wiley Series in Probability and Statistics 

Matrix Differential Calculus With Applications in Statistics and Econometrics Third Edition is an ideal text for graduate students and academics studying the subject, as well as for postgraduates and specialists working in biosciences and psychology.

Table of Contents

  1. Cover
  2. Preface
  3. Part One: Matrices
    1. Chapter 1: Basic properties of vectors and matrices
      1. 1 INTRODUCTION
      2. 2 SETS
      3. 3 MATRICES: ADDITION AND MULTIPLICATION
      4. 4 THE TRANSPOSE OF A MATRIX
      5. 5 SQUARE MATRICES
      6. 6 LINEAR FORMS AND QUADRATIC FORMS
      7. 7 THE RANK OF A MATRIX
      8. 8 THE INVERSE
      9. 9 THE DETERMINANT
      10. 10 THE TRACE
      11. 11 PARTITIONED MATRICES
      12. 12 COMPLEX MATRICES
      13. 13 EIGENVALUES AND EIGENVECTORS
      14. 14 SCHUR'S DECOMPOSITION THEOREM
      15. 15 THE JORDAN DECOMPOSITION
      16. 16 THE SINGULAR‐VALUE DECOMPOSITION
      17. 17 FURTHER RESULTS CONCERNING EIGENVALUES
      18. 18 POSITIVE (SEMI)DEFINITE MATRICES
      19. 19 THREE FURTHER RESULTS FOR POSITIVE DEFINITE MATRICES
      20. 20 A USEFUL RESULT
      21. 21 SYMMETRIC MATRIX FUNCTIONS
    2. Chapter 2: Kronecker products, vec operator, and Moore‐Penrose inverse
      1. 1 INTRODUCTION
      2. 2 THE KRONECKER PRODUCT
      3. 3 EIGENVALUES OF A KRONECKER PRODUCT
      4. 4 THE VEC OPERATOR
      5. 6 EXISTENCE AND UNIQUENESS OF THE MP INVERSE
      6. 7 SOME PROPERTIES OF THE MP INVERSE
      7. 8 FURTHER PROPERTIES
      8. 9 THE SOLUTION OF LINEAR EQUATION SYSTEMS
      9. BIBLIOGRAPHICAL NOTES
    3. Chapter 3: Miscellaneous matrix results
      1. 1 INTRODUCTION
      2. 2 THE ADJOINT MATRIX
      3. 3 PROOF OF THEOREM 3.1
      4. 4 BORDERED DETERMINANTS
      5. 5 THE MATRIX EQUATION AX = 0
      6. 6 THE HADAMARD PRODUCT
      7. 7 THE COMMUTATION MATRIX Kmn
      8. 8 THE DUPLICATION MATRIX Dn
      9. 9 RELATIONSHIP BETWEEN Dn +1 AND Dn , I
      10. 10 RELATIONSHIP BETWEEN Dn +1 AND Dn , II
      11. 11 CONDITIONS FOR A QUADRATIC FORM TO BE POSITIVE (NEGATIVE) SUBJECT TO LINEAR CONSTRAINTS
      12. 12 NECESSARY AND SUFFICIENT CONDITIONS FOR r(A : B) = r(A) + r(B)
      13. 13 THE BORDERED GRAMIAN MATRIX
      14. 14 THE EQUATIONS X 1 A + X 2 B′ = ′ = G 1, X 1 B = G 2
      15. BIBLIOGRAPHICAL NOTES
  4. Part Two: Differentials: the theory
    1. Chapter 4: Mathematical preliminaries
      1. 1 INTRODUCTION
      2. 2 INTERIOR POINTS AND ACCUMULATION POINTS
      3. 3 OPEN AND CLOSED SETS
      4. 4 THE BOLZANO‐WEIERSTRASS THEOREM
      5. 5 FUNCTIONS
      6. 6 THE LIMIT OF A FUNCTION
      7. 7 CONTINUOUS FUNCTIONS AND COMPACTNESS
      8. 8 CONVEX SETS
      9. 9 CONVEX AND CONCAVE FUNCTIONS
    2. Chapter 5: Differentials and differentiability
      1. 1 INTRODUCTION
      2. 2 CONTINUITY
      3. 3 DIFFERENTIABILITY AND LINEAR APPROXIMATION
      4. 4 THE DIFFERENTIAL OF A VECTOR FUNCTION
      5. 5 UNIQUENESS OF THE DIFFERENTIAL
      6. 6 CONTINUITY OF DIFFERENTIABLE FUNCTIONS
      7. 7 PARTIAL DERIVATIVES
      8. 8 THE FIRST IDENTIFICATION THEOREM
      9. 9 EXISTENCE OF THE DIFFERENTIAL, I
      10. 10 EXISTENCE OF THE DIFFERENTIAL, II
      11. 11 CONTINUOUS DIFFERENTIABILITY
      12. 12 THE CHAIN RULE
      13. 13 CAUCHY INVARIANCE
      14. 14 THE MEAN‐VALUE THEOREM FOR REAL‐VALUED FUNCTIONS
      15. 15 DIFFERENTIABLE MATRIX FUNCTIONS
      16. 16 SOME REMARKS ON NOTATION
      17. 17 COMPLEX DIFFERENTIATION
      18. BIBLIOGRAPHICAL NOTES
    3. Chapter 6: The second differential
      1. 1 INTRODUCTION
      2. 2 SECOND‐ORDER PARTIAL DERIVATIVES
      3. 3 THE HESSIAN MATRIX
      4. 4 TWICE DIFFERENTIABILITY AND SECOND‐ORDER APPROXIMATION, I
      5. 5 DEFINITION OF TWICE DIFFERENTIABILITY
      6. 6 THE SECOND DIFFERENTIAL
      7. 7 SYMMETRY OF THE HESSIAN MATRIX
      8. 8 THE SECOND IDENTIFICATION THEOREM
      9. 9 TWICE DIFFERENTIABILITY AND SECOND‐ORDER APPROXIMATION, II
      10. 10 CHAIN RULE FOR HESSIAN MATRICES
      11. 11 THE ANALOG FOR SECOND DIFFERENTIALS
      12. 12 TAYLOR'S THEOREM FOR REAL‐VALUED FUNCTIONS
      13. 13 HIGHER‐ORDER DIFFERENTIALS
      14. 14 REAL ANALYTIC FUNCTIONS
      15. 15 TWICE DIFFERENTIABLE MATRIX FUNCTIONS
      16. BIBLIOGRAPHICAL NOTES
    4. Chapter 7: Static optimization
      1. 1 INTRODUCTION
      2. 2 UNCONSTRAINED OPTIMIZATION
      3. 3 THE EXISTENCE OF ABSOLUTE EXTREMA
      4. 4 NECESSARY CONDITIONS FOR A LOCAL MINIMUM
      5. 5 SUFFICIENT CONDITIONS FOR A LOCAL MINIMUM: FIRST‐DERIVATIVE TEST
      6. 6 SUFFICIENT CONDITIONS FOR A LOCAL MINIMUM: SECOND‐DERIVATIVE TEST
      7. 7 CHARACTERIZATION OF DIFFERENTIABLE CONVEX FUNCTIONS
      8. 8 CHARACTERIZATION OF TWICE DIFFERENTIABLE CONVEX FUNCTIONS
      9. 9 SUFFICIENT CONDITIONS FOR AN ABSOLUTE MINIMUM
      10. 10 MONOTONIC TRANSFORMATIONS
      11. 11 OPTIMIZATION SUBJECT TO CONSTRAINTS
      12. 12 NECESSARY CONDITIONS FOR A LOCAL MINIMUM UNDER CONSTRAINTS
      13. 13 SUFFICIENT CONDITIONS FOR A LOCAL MINIMUM UNDER CONSTRAINTS
      14. 14 SUFFICIENT CONDITIONS FOR AN ABSOLUTE MINIMUM UNDER CONSTRAINTS
      15. 15 A NOTE ON CONSTRAINTS IN MATRIX FORM
      16. 16 ECONOMIC INTERPRETATION OF LAGRANGE MULTIPLIERS
      17. APPENDIX: THE IMPLICIT FUNCTION THEOREM
  5. Part Three: Differentials: the practice
    1. Chapter 8: Some important differentials
      1. 1 INTRODUCTION
      2. 2 FUNDAMENTAL RULES OF DIFFERENTIAL CALCULUS
      3. 3 THE DIFFERENTIAL OF A DETERMINANT
      4. 4 THE DIFFERENTIAL OF AN INVERSE
      5. 5 DIFFERENTIAL OF THE MOORE‐PENROSE INVERSE
      6. 6 THE DIFFERENTIAL OF THE ADJOINT MATRIX
      7. 7 ON DIFFERENTIATING EIGENVALUES AND EIGENVECTORS
      8. 8 THE CONTINUITY OF EIGENPROJECTIONS
      9. 9 THE DIFFERENTIAL OF EIGENVALUES AND EIGENVECTORS: SYMMETRIC CASE
      10. 10 TWO ALTERNATIVE EXPRESSIONS FOR dλ
      11. 11 SECOND DIFFERENTIAL OF THE EIGENVALUE FUNCTION
    2. Chapter 9: First‐order differentials and Jacobian matrices
      1. 1 INTRODUCTION
      2. 2 CLASSIFICATION
      3. 3 DERISATIVES
      4. 4 DERIVATIVES
      5. 5 IDENTIFICATION OF JACOBIAN MATRICES
      6. 6 THE FIRST IDENTIFICATION TABLE
      7. 7 PARTITIONING OF THE DERIVATIVE
      8. 8 SCALAR FUNCTIONS OF A SCALAR
      9. 9 SCALAR FUNCTIONS OF A VECTOR
      10. 10 SCALAR FUNCTIONS OF A MATRIX, I: TRACE
      11. 11 SCALAR FUNCTIONS OF A MATRIX, II: DETERMINANT
      12. 12 SCALAR FUNCTIONS OF A MATRIX, III: EIGENVALUE
      13. 13 TWO EXAMPLES OF VECTOR FUNCTIONS
      14. 14 MATRIX FUNCTIONS
      15. 15 KRONECKER PRODUCTS
      16. 16 SOME OTHER PROBLEMS
      17. 17 JACOBIANS OF TRANSFORMATIONS
    3. Chapter 10: Second‐order differentials and Hessian matrices
      1. 1 INTRODUCTION
      2. 2 THE SECOND IDENTIFICATION TABLE
      3. 3 LINEAR AND QUADRATIC FORMS
      4. 4 A USEFUL THEOREM
      5. 5 THE DETERMINANT FUNCTION
      6. 6 THE EIGENVALUE FUNCTION
      7. 7 OTHER EXAMPLES
      8. 8 COMPOSITE FUNCTIONS
      9. 9 THE EIGENVECTOR FUNCTION
      10. 10 HESSIAN OF MATRIX FUNCTIONS, I
      11. 11 HESSIAN OF MATRIX FUNCTIONS, II
  6. Part Four: Inequalities
    1. Chapter 11: Inequalities
      1. 1 INTRODUCTION
      2. 2 THE CAUCHY‐SCHWARZ INEQUALITY
      3. 3 MATRIX ANALOGS OF THE CAUCHY‐SCHWARZ INEQUALITY
      4. 4 THE THEOREM OF THE ARITHMETIC AND GEOMETRIC MEANS
      5. 5 THE RAYLEIGH QUOTIENT
      6. 6 CONCAVITY OF λ 1 AND CONVEXITY OF λ n
      7. 7 VARIATIONAL DESCRIPTION OF EIGENVALUES
      8. 8 FISCHER'S MIN‐MAX THEOREM
      9. 9 MONOTONICITY OF THE EIGENVALUES
      10. 10 THE POINCARÉ SEPARATION THEOREM
      11. 11 TWO COROLLARIES OF POINCARÉ'S THEOREM
      12. 12 FURTHER CONSEQUENCES OF THE POINCARÉ THEOREM
      13. 13 MULTIPLICATIVE VERSION
      14. 14 THE MAXIMUM OF A BILINEAR FORM
      15. 15 HADAMARD'S INEQUALITY
      16. 16 AN INTERLUDE: KARAMATA'S INEQUALITY
      17. 17 KARAMATA'S INEQUALITY AND EIGENVALUES
      18. 18 AN INEQUALITY CONCERNING POSITIVE SEMIDEFINITE MATRICES
      19. 19 A REPRESENTATION THEOREM FOR
      20. 20 A REPRESENTATION THEOREM FOR (tr A)
      21. 21 HÖLDER'S INEQUALITY
      22. 22 CONCAVITY OF log|A|
      23. 23 MINKOWSKI'S INEQUALITY
      24. 24 QUASILINEAR REPRESENTATION OF |A|
      25. 25 MINKOWSKI'S DETERMINANT THEOREM
      26. 26 WEIGHTED MEANS OF ORDER p
      27. 27 SCHLÖMILCH'S INEQUALITY
      28. 28 CURVATURE PROPERTIES OF Mp (x, a)
      29. 29 LEAST SQUARES
      30. 30 GENERALIZED LEAST SQUARES
      31. 31 RESTRICTED LEAST SQUARES
      32. 32 RESTRICTED LEAST SQUARES: MATRIX VERSION
  7. Part Five: The linear model
    1. Chapter 12: Statistical preliminaries
      1. 1 INTRODUCTION
      2. 2 THE CUMULATIVE DISTRIBUTION FUNCTION
      3. 3 THE JOINT DENSITY FUNCTION
      4. 4 EXPECTATIONS
      5. 5 VARIANCE AND COVARIANCE
      6. 6 INDEPENDENCE OF TWO RANDOM VARIABLES
      7. 7 INDEPENDENCE OF n RANDOM VARIABLES
      8. 8 SAMPLING
      9. 9 THE ONE‐DIMENSIONAL NORMAL DISTRIBUTION
      10. 10 THE MULTIVARIATE NORMAL DISTRIBUTION
      11. 11 ESTIMATION
    2. Chapter 13: The linear regression model
      1. 1 INTRODUCTION
      2. 2 AFFINE MINIMUM‐TRACE UNBIASED ESTIMATION
      3. 3 THE GAUSS‐MARKOV THEOREM
      4. 4 THE METHOD OF LEAST SQUARES
      5. 5 AITKEN'S THEOREM
      6. 6 MULTICOLLINEARITY
      7. 7 ESTIMABLE FUNCTIONS
      8. 8 LINEAR CONSTRAINTS: THE CASEℳ(R) ⊂ ℳ(X)ℳ(R) ⊂ ℳ(X)
      9. 9 LINEAR CONSTRAINTS: THE GENERAL CASE
      10. 10 LINEAR CONSTRAINTS: THE CASEℳ(R) ∩ ℳ(X) = {0}ℳ(R) ∩ ℳ(X) = {0}
      11. 11 A SINGULAR VARIANCE MATRIX: THE CASEℳ(X) ⊂ ℳ(V)ℳ(X) ⊂ ℳ(V)
      12. 12 A SINGULAR VARIANCE MATRIX: THE CASE r(X′V X) = r(X)
      13. 13 A SINGULAR VARIANCE MATRIX: THE GENERAL CASE, I
      14. 14 EXPLICIT AND IMPLICIT LINEAR CONSTRAINTS
      15. 15 THE GENERAL LINEAR MODEL, I
      16. 16 A SINGULAR VARIANCE MATRIX: THE GENERAL CASE, II
      17. 17 THE GENERAL LINEAR MODEL, II
      18. 18 GENERALIZED LEAST SQUARES
      19. 19 RESTRICTED LEAST SQUARES
    3. Chapter 14: Further topics in the linear model
      1. 1 INTRODUCTION
      2. 2 BEST QUADRATIC UNBIASED ESTIMATION OF σ
      3. 3 THE BEST QUADRATIC AND POSITIVE UNBIASED ESTIMATOR OF σ
      4. 4 THE BEST QUADRATIC UNBIASED ESTIMATOR OF σ
      5. 5 BEST QUADRATIC INVARIANT ESTIMATION OF σ
      6. 6 THE BEST QUADRATIC AND POSITIVE INVARIANT ESTIMATOR OF σ
      7. 7 THE BEST QUADRATIC INVARIANT ESTIMATOR OF σ
      8. 8 BEST QUADRATIC UNBIASED ESTIMATION: MULTIVARIATE NORMAL CASE
      9. 9 BOUNDS FOR THE BIAS OF THE LEAST‐SQUARES ESTIMATOR OF σ, I
      10. 10 BOUNDS FOR THE BIAS OF THE LEAST‐SQUARES ESTIMATOR OF σ, II
      11. 11 THE PREDICTION OF DISTURBANCES
      12. 12 BEST LINEAR UNBIASED PREDICTORS WITH SCALAR VARIANCE MATRIX
      13. 13 BEST LINEAR UNBIASED PREDICTORS WITH FIXED VARIANCE MATRIX, I
      14. 14 BEST LINEAR UNBIASED PREDICTORS WITH FIXED VARIANCE MATRIX, II
      15. 15 LOCAL SENSITIVITY OF THE POSTERIOR MEAN
      16. 16 LOCAL SENSITIVITY OF THE POSTERIOR PRECISION
  8. Part Six: Applications to maximum likelihood estimation
    1. Chapter 15: Maximum likelihood estimation
      1. 1 INTRODUCTION
      2. 2 THE METHOD OF MAXIMUM LIKELIHOOD (ML)
      3. 3 ML ESTIMATION OF THE MULTIVARIATE NORMAL DISTRIBUTION
      4. 4 SYMMETRY: IMPLICIT VERSUS EXPLICIT TREATMENT
      5. 5 THE TREATMENT OF POSITIVE DEFINITENESS
      6. 6 THE INFORMATION MATRIX
      7. 7 ML ESTIMATION OF THE MULTIVARIATE NORMAL DISTRIBUTION: DISTINCT MEANS
      8. 8 THE MULTIVARIATE LINEAR REGRESSION MODEL
      9. 9 THE ERRORS‐IN‐VARIABLES MODEL
      10. 10 THE NONLINEAR REGRESSION MODEL WITH NORMAL ERRORS
      11. 11 SPECIAL CASE: FUNCTIONAL INDEPENDENCE OF MEAN AND VARIANCE PARAMETERS
      12. 12 GENERALIZATION OF THEOREM 15.6
    2. Chapter 16: Simultaneous equations
      1. 1 INTRODUCTION
      2. 2 THE SIMULTANEOUS EQUATIONS MODEL
      3. 3 THE IDENTIFICATION PROBLEM
      4. 4 IDENTIFICATION WITH LINEAR CONSTRAINTS ON B AND Γ ONLY AND Γ ONLY
      5. 5 IDENTIFICATION WITH LINEAR CONSTRAINTS ON B, Γ, AND Σ, Γ, AND Σ
      6. 6 NONLINEAR CONSTRAINTS
      7. 7 FIML: THE INFORMATION MATRIX (GENERAL CASE)
      8. 8 FIML: ASYMPTOTIC VARIANCE MATRIX (SPECIAL CASE)
      9. 9 LIML: FIRST‐ORDER CONDITIONS
      10. 10 LIML: INFORMATION MATRIX
      11. 11 LIML: ASYMPTOTIC VARIANCE MATRIX
      12. BIBLIOGRAPHICAL NOTES
    3. Chapter 17: Topics in psychometrics
      1. 1 INTRODUCTION
      2. 2 POPULATION PRINCIPAL COMPONENTS
      3. 3 OPTIMALITY OF PRINCIPAL COMPONENTS
      4. 4 A RELATED RESULT
      5. 5 SAMPLE PRINCIPAL COMPONENTS
      6. 6 OPTIMALITY OF SAMPLE PRINCIPAL COMPONENTS
      7. 7 ONE‐MODE COMPONENT ANALYSIS
      8. 8 ONE‐MODE COMPONENT ANALYSIS AND SAMPLE PRINCIPAL COMPONENTS
      9. 9 TWO‐MODE COMPONENT ANALYSIS
      10. 10 MULTIMODE COMPONENT ANALYSIS
      11. 11 FACTOR ANALYSIS
      12. 12 A ZIGZAG ROUTINE
      13. 13 NEWTON‐RAPHSON ROUTINE
      14. 14 KAISER'S VARIMAX METHOD
      15. 15 CANONICAL CORRELATIONS AND VARIATES IN THE POPULATION
      16. 16 CORRESPONDENCE ANALYSIS
      17. 17 LINEAR DISCRIMINANT ANALYSIS
      18. BIBLIOGRAPHICAL NOTES
  9. Part Seven: Summary
    1. Chapter 18: Matrix calculus: the essentials
      1. 1 INTRODUCTION
      2. 2 DIFFERENTIALS
      3. 3 VECTOR CALCULUS
      4. 4 OPTIMIZATION
      5. 5 LEAST SQUARES
      6. 6 MATRIX CALCULUS
      7. 7 INTERLUDE ON LINEAR AND QUADRATIC FORMS
      8. 8 THE SECOND DIFFERENTIAL
      9. 9 CHAIN RULE FOR SECOND DIFFERENTIALS
      10. 10 FOUR EXAMPLES
      11. 11 THE KRONECKER PRODUCT AND VEC OPERATOR
      12. 12 IDENTIFICATION
      13. 13 THE COMMUTATION MATRIX
      14. 14 FROM SECOND DIFFERENTIAL TO HESSIAN
      15. 15 SYMMETRY AND THE DUPLICATION MATRIX
      16. 16 MAXIMUM LIKELIHOOD
      17. FURTHER READING
  10. Bibliography
  11. Index of symbols
  12. Subject index
  13. End User License Agreement
3.141.24.134