Contents
2 Signal Processing Transforms
Serhan Yarkan and Khalid A. Qaraqe
2.3 Fourier Series and Transform
2.5 Cosine and Sine Transforms
2.6.1 Properties of Laplace Transform
2.7.1 Properties of Hartley Transform
2.8.1 Properties of Hilbert Transform
2.9 Discrete-Time Fourier Transform
2.10.1 Properties of Z-Transform
2.11 Conclusion and Further Reading
Fatemeh Hamidi Sepehr and Erchin Serpedin
3.1.1 Subspaces and Direct Sums
3.1.2 Spanning and Linear Independency
3.1.10 Gram-Schmidt Orthogonalization
3.2.1 Range and Nullspace of a Linear Transformation
3.2.2 Composition and Invertibility
3.2.3 Matrix Representation of Linear Transformations
3.2.5 Linear Functionals and Dual Spaces
3.2.6 Adjoint of a Linear Transformation
3.2.7 Four Fundamental Subspaces
3.3 Operator Norms and Matrix Norms
3.4 Systems of Linear Equations
3.5 Determinant, Adjoint, and Inverse of a Matrix
3.7 Unitary and Orthogonal Operators and Matrices
3.9 LDL and Cholesky Decomposition
3.11 Householder and Givens Transformations
3.12 Best Approximations and Orthogonal Projections
3.13 Least Squares Approximations
3.14.1 Principal Angles Between Subspaces
3.15 Eigenvalues and Eigenvectors
3.16 Schur Factorization and Spectral Theorem
3.17 Singular Value Decomposition (SVD)
3.19 Application of SVD and Rayleigh Quotient: Principal Component Analysis
3.20.8 Positive and Negative Definite Matrices
3.20.9 Matrix Condition Number
3.20.10 Sherman-Morrison-Woodbury Identity
3.21.5 Differentiation of Matrix and Vector Functions
3.22 References and Further Studies
Tolga Duman
4.3 Polynomials with Coefficients in GF(2)
4.4.2 Factorization of the Polynomial Xn + 1
4.5 Some Notes on Applications of Finite Fields
Vivek Sarin
5.2 Sensitivity and Conditioning
5.4.1 Polynomial Interpolation
5.4.2 Piecewise Polynomial Interpolation
5.5.5 Linear Fractional Interpolation
5.6 Eigenvalues and Singular Values
5.6.3 Computing Singular Values
Walter D. Wallis
6.1 Two Principles of Enumeration
6.2 Permutations and Combinations
6.3 The Principle of Inclusion and Exclusion
6.7 Paths and Cycles in Graphs
6.11 Balanced Incomplete Block Designs
7 Probability, Random Variables, and Stochastic Processes
Dinesh Rajan
7.1 Introduction to Probability
7.2.1 Discrete Random Variables
7.2.2 Continuous Random Variables
7.3.1 Expected Values, Characteristic Functions
7.3.3 Functions of Multiple Random Variables
7.3.4 Convergence of Random Variables
7.3.5 Law of Large Numbers (LLN) and Central Limit Theorem (CLT)
7.4.2 Random Process as the Input to a Linear System
7.5.2 Continuous Time Markov Chains
7.6 Summary and Further Reading
Romain Couillet and Merouane Debbah
8.2 Spectral Distribution of Random Matrices
8.2.2 Limiting Spectral Distribution
8.3.1 Exact Eigenvalue Separation
8.5.1 Binary Hypothesis Testing
Hongbin Li
9.2 Concentration Inequalities
10 Fundamentals of Estimation Theory
Yik-Chung Wu
10.2 Bound on Minimum Variance – Cramér-Rao Lower Bound
10.2.2 Finding MVUE Attaining the CRLB
10.3.2 Finding MVUE from Sufficient Statistics
10.4 Maximum Likelihood Estimation
10.4.1 ML Estimation Principle
10.4.2 Properties of the ML Estimator
10.5.1 Geometrical Interpretation
10.5.2 Recursive LS Estimation
10.5.3 Weighted LS and Iterative Reweighted LS
10.5.4 Constrained LS Estimation
10.6 Regularized LS Estimation
10.6.2 LS Estimation with Quadratic Constraint
10.7.1 Minimum Mean Square Error Estimation
10.7.2 General Bayesian Estimator
10.7.3 Handling Nuisance Parameters
10.8 References and Further Reading
11 Fundamentals of Detection Theory
Venugopal V. Veeravalli
11.1.1 Statistical Decision Theory Framework
11.1.2 Probabilistic Structure for Observation Space
11.1.3 Conditional Density and Conditional Risk
11.1.6 Randomized Decision Rules
11.1.7 General Method for Finding Bayes Rules
11.2 Bayesian Binary Detection
11.3.1 Bayes Risk Line and Minimum Risk Curve
11.4 Binary Neyman-Pearson Detection
11.4.1 Solution to the N-P Optimization Problem
11.4.2 N-P Rule and Receiver Operating Characteristic
11.5 Bayesian Composite Detection
11.6 Neyman-Pearson Composite Detection
11.6.1 UMP Detection with One Composite Hypothesis
11.6.2 UMP Detection with Both Composite Hypotheses
11.6.3 Generalized Likelihood Ratio (GLR) Detection
11.6.4 Locally Most Powerful (LMP) Detection
11.7 Binary Detection with Vector Observations
11.7.1 Conditionally Independent Observations
11.7.2 Deterministic Signals in Correlated Gaussian Noise
11.7.3 Gaussian Signals in Gaussian Noise
11.8 Summary and Further Reading
12 Monte Carlo Methods for Statistical Signal Processing
Xiaodong Wang
12.1.1 Model-Based Signal Processing
12.3 Markov Chain Monte Carlo (MCMC) Methods
12.3.1 General MCMC Algorithms
12.3.2 Applications of MCMC in Digital Communications
12.4 Sequential Monte Carlo (SMC) Methods
12.4.3 Applications of SMC in Bioinformatics
12.5 Conclusions and Further Readings
13 Factor Graphs and Message Passing Algorithms
Aitzaz Ahmad, Erchin Serpedin, and Khalid A. Qaraqe
13.1.3 Organization of the Chapter
13.3 Modeling Systems Using Factor Graphs
13.4 Relationship with Other Probabilistic Graphical Models
13.5 Message Passing in Factor Graphs
13.6 Factor Graphs with Cycles
13.6.1 Message Passing Schedules
13.6.2 Iterative Message Passing
13.7 Some General Remarks on Factor Graphs
13.7.2 Hybrid Equality Constraint
13.8 Some Important Message Passing Algorithms
13.8.1 Forward/Backward Algorithm
13.8.4 Expectation Maximization (EM) Algorithm
13.9 Applications of Message Passing in Factor Graphs
13.9.1 Detection of OFDM Signals in the Presence of Carrier Frequency Offset and Phase Noise
14 Unconstrained and Constrained Optimization Problems
Shuguang Cui, Anthony Man-Cho So, and Rui Zhang
14.1 Basics of Convex Analysis
14.2 Unconstrained vs. Constrained Optimization
14.2.1 Optimality Conditions for Unconstrained Optimization
14.2.2 Optimality Conditions for Constrained Optimization
15 Linear Programming and Mixed Integer Programming
Bogdan Dumitrescu
15.1.2 Transformations of the Standard Problem
15.1.3 Optimality Characterization
15.2 Modeling Problems via Linear Programming
15.2.1 Optimization with 1-norm and ∞-norm
15.2.2 Chebyshev Center of a Polytope
15.2.3 Classification with Separating Hyperplanes
15.2.4 Linear Fractional Programming
15.2.5 Continuous Constraints and Discretization
15.3 Mixed Integer Programming
15.3.1 Problem Statement and LP Relaxation
15.3.3 Examples of Mixed Integer Problems
15.4 Historical Notes and Further Reading
16 Majorization Theory and Applications
Jiaheng Wang and Daniel Palomar
16.1.2 Schur-Convex/Concave Functions
16.1.3 Relation to Matrix Theory
16.1.4 Multiplicative Majorization
16.1.5 Stochastic Majorization
16.2 Applications of Majorization Theory
16.2.2 Linear MIMO Transceiver Design
16.2.3 Nonlinear MIMO Transceiver Design
16.3 Conclusions and Further Readings
Thomas Chen
17.2.1 Discrete-Time Markov Chains
17.2.2 Continuous-Time Markov Chains
17.4.1 Steady-State Probabilities for Number in System
17.4.3 Probability Distribution of Delay Through System
17.5.1 Example: Queueing Model of a Packet Switch
17.8.3 Distribution of Number in System
17.8.4 Mean Delay Through System
17.8.5 Distribution of Delay Through System
17.8.7 Example: Data Frame Retransmissions
18 Network Optimization Techniques
Michał Pióro
18.2 Basic Multicommodity Flow Networks Optimization Models
18.2.2 Link-Path vs. Node-Link Formulation Applied to Allocation Problems
18.3 Optimization Methods for Multicommodity Flow Networks
18.4 Optimization Models for Multistate Networks
18.4.3 Multihour and Multiperiod Design
Erik G. Larsson and Eduard Jorswieck
19.4 Noncooperative Games and the Nash Equilibrium
19.5.1 Bargaining without Transferable Utilities
19.5.2 Bargaining with Transferable Utilities
19.6 Games with Incomplete Information
19.7.1 Nash Equilibrium in Extensive Form Games
19.7.2 Subgame-Perfect Equilibrium
19.7.3 Incomplete Information in Extensive Form Games
19.8 Repeated Games and Evolutionary Stability
19.8.2 Evolutionary Game Theory
19.9 Coalitional Form/Characteristic Function Form
19.10 Mechanism Design and Implementation Theory
19.10.3 Direct Revelation Principle
19.10.4 Clarke-Groves and AGV-Arrow Mechanism
19.11 Applications to Signal Processing and Communications
20 A Short Course on Frame Theory
Veniamin I. Morgenshtern and Helmut Bölcskei
20.1 Examples of Signal Expansions
20.2 Signal Expansions in Finite-Dimensional Spaces
20.2.3 Redundant Signal Expansions
20.3 Frames for General Hilbert Spaces
20.3.2 The Canonical Dual Frame
20.3.5 Exact Frames and Biorthonormality
20.4.1 Sampling Theorem as a Frame Expansion
20.4.2 Design Freedom in Oversampled A/D Conversion
20.4.3 Noise Reduction in Oversampled A/D Conversion