Home Page Icon
Home Page
Table of Contents for
Half Title
Close
Half Title
by Dinesh Rajan, Thomas Chen, Erchin Serpedin
Mathematical Foundations for Signal Processing, Communications, and Networking
Cover
Half Title
Title Page
Copyright Page
Table of Contents
List of Figures
List of Tables
Preface
Editors
List of Contributors
List of Acronyms
Notations and Symbols
1 Introduction
2 Signal Processing Transforms
2.1 Introduction
2.2 Basic Transformations
2.3 Fourier Series and Transform
2.3.1 Basic Definitions
2.3.2 Fourier Series
2.3.3 Fourier Transform
2.4 Sampling
2.4.1 Impulse-Train Sampling
2.4.2 Aliasing
2.5 Cosine and Sine Transforms
2.5.1 Cosine Transform
2.5.2 Sine Transform
2.6 Laplace Transform
2.6.1 Properties of Laplace Transform
2.7 Hartley Transform
2.7.1 Properties of Hartley Transform
2.8 Hilbert Transform
2.8.1 Properties of Hilbert Transform
2.9 Discrete-Time Fourier Transform
2.10 The Z-Transform
2.10.1 Properties of Z-Transform
2.11 Conclusion and Further Reading
2.12 Exercises
References
3 Linear Algebra
3.1 Vector Spaces
3.1.1 Subspaces and Direct Sums
3.1.2 Spanning and Linear Independency
3.1.3 Bases
3.1.4 Dimension
3.1.5 Ordered Basis
3.1.6 Norms
3.1.7 Inner-Products
3.1.8 Induced Norms
3.1.9 Orthogonality
3.1.10 Gram-Schmidt Orthogonalization
3.2 Linear Transformations
3.2.1 Range and Nullspace of a Linear Transformation
3.2.2 Composition and Invertibility
3.2.3 Matrix Representation of Linear Transformations
3.2.4 Projection Operators
3.2.5 Linear Functionals and Dual Spaces
3.2.6 Adjoint of a Linear Transformation
3.2.7 Four Fundamental Subspaces
3.3 Operator Norms and Matrix Norms
3.4 Systems of Linear Equations
3.5 Determinant, Adjoint, and Inverse of a Matrix
3.6 Cramer’s Rule
3.7 Unitary and Orthogonal Operators and Matrices
3.8 LU Decomposition
3.9 LDL and Cholesky Decomposition
3.10 QR Decomposition
3.11 Householder and Givens Transformations
3.11.1 Orthogonal Reduction
3.12 Best Approximations and Orthogonal Projections
3.13 Least Squares Approximations
3.14 Angles Between Subspaces
3.14.1 Principal Angles Between Subspaces
3.15 Eigenvalues and Eigenvectors
3.15.1 Diagonalization
3.16 Schur Factorization and Spectral Theorem
3.17 Singular Value Decomposition (SVD)
3.18 Rayleigh Quotient
3.19 Application of SVD and Rayleigh Quotient: Principal Component Analysis
3.20 Special Matrices
3.20.1 Block Matrices
3.20.2 Circulant Matrices
3.20.3 Toeplitz Matrices
3.20.4 Hankel Matrices
3.20.5 Vandermonde Matrices
3.20.6 Normal Matrices
3.20.7 Stochastic Matrices
3.20.8 Positive and Negative Definite Matrices
3.20.9 Matrix Condition Number
3.20.10 Sherman-Morrison-Woodbury Identity
3.20.11 Schur Complement
3.20.12 Generalized Inverses
3.21 Matrix Operations
3.21.1 Kronecker Product
3.21.2 Hadamard Product
3.21.3 Dot Product
3.21.4 Direct Sum
3.21.5 Differentiation of Matrix and Vector Functions
3.22 References and Further Studies
3.23 Exercises
References
4 Elements of Galois Fields
4.1 Groups, Rings, and Fields
4.1.1 Groups
4.1.2 Rings
4.1.3 Fields
4.2 Galois Fields
4.3 Polynomials with Coefficients in GF(2)
4.4 Construction of GF(2m)
4.4.1 Conjugate Elements
4.4.2 Factorization of the Polynomial Xn + 1
4.5 Some Notes on Applications of Finite Fields
4.6 Exercises
References
5 Numerical Analysis
5.1 Numerical Approximation
5.2 Sensitivity and Conditioning
5.3 Computer Arithmetic
5.4 Interpolation
5.4.1 Polynomial Interpolation
5.4.2 Piecewise Polynomial Interpolation
5.5 Nonlinear Equations
5.5.1 Interval Bisection
5.5.2 Newton’s Method
5.5.3 Secant Method
5.5.4 Muller’s Method
5.5.5 Linear Fractional Interpolation
5.5.6 Zeros of Polynomials
5.6 Eigenvalues and Singular Values
5.6.1 Power Iterations
5.6.2 QR Algorithm
5.6.3 Computing Singular Values
5.7 Further Reading
5.8 Exercises
References
6 Combinatorics
6.1 Two Principles of Enumeration
6.2 Permutations and Combinations
6.3 The Principle of Inclusion and Exclusion
6.4 Generating Functions
6.5 Recurrence Relations
6.6 Graphs
6.7 Paths and Cycles in Graphs
6.8 Trees
6.9 Encoding and Decoding
6.10 Latin Squares
6.11 Balanced Incomplete Block Designs
6.12 Conclusion
6.13 Exercises
References
7 Probability, Random Variables, and Stochastic Processes
7.1 Introduction to Probability
7.2 Random Variables
7.2.1 Discrete Random Variables
7.2.2 Continuous Random Variables
7.3 Joint Random Variables
7.3.1 Expected Values, Characteristic Functions
7.3.2 Inequalities
7.3.3 Functions of Multiple Random Variables
7.3.4 Convergence of Random Variables
7.3.5 Law of Large Numbers (LLN) and Central Limit Theorem (CLT)
7.4 Random Processes
7.4.1 Stationary Process
7.4.2 Random Process as the Input to a Linear System
7.5 Markov Process
7.5.1 Markov Chains
7.5.2 Continuous Time Markov Chains
7.5.3 Hidden Markov Model
7.6 Summary and Further Reading
7.7 Exercises
References
8 Random Matrix Theory
8.1 Probability Notations
8.2 Spectral Distribution of Random Matrices
8.2.1 Wishart Matrices
8.2.2 Limiting Spectral Distribution
8.3 Spectral Analysis
8.3.1 Exact Eigenvalue Separation
8.3.2 Support of l.s.d.
8.4 Statistical Inference
8.5 Applications
8.5.1 Binary Hypothesis Testing
8.5.2 Parameter Estimation
8.6 Conclusion
8.7 Exercises
References
9 Large Deviations
9.1 Introduction
9.2 Concentration Inequalities
9.3 Rate Function
9.4 Cramér’s Theorem
9.5 Method of Types
9.6 Sanov’s Theorem
9.7 Hypothesis Testing
9.8 Further Readings
9.9 Exercises
References
10 Fundamentals of Estimation Theory
10.1 Introduction
10.2 Bound on Minimum Variance – Cramér-Rao Lower Bound
10.2.1 Computation of CRLB
10.2.2 Finding MVUE Attaining the CRLB
10.3 MVUE Using RBLS Theorem
10.3.1 Sufficient Statistics
10.3.2 Finding MVUE from Sufficient Statistics
10.4 Maximum Likelihood Estimation
10.4.1 ML Estimation Principle
10.4.2 Properties of the ML Estimator
10.5 Least Squares Estimation
10.5.1 Geometrical Interpretation
10.5.2 Recursive LS Estimation
10.5.3 Weighted LS and Iterative Reweighted LS
10.5.4 Constrained LS Estimation
10.6 Regularized LS Estimation
10.6.1 ℓ2 Regularization
10.6.2 LS Estimation with Quadratic Constraint
10.6.3 ℓ1 Regularization
10.7 Bayesian Estimation
10.7.1 Minimum Mean Square Error Estimation
10.7.2 General Bayesian Estimator
10.7.3 Handling Nuisance Parameters
10.8 References and Further Reading
10.9 Exercises
References
11 Fundamentals of Detection Theory
11.1 Introduction
11.1.1 Statistical Decision Theory Framework
11.1.2 Probabilistic Structure for Observation Space
11.1.3 Conditional Density and Conditional Risk
11.1.4 Bayesian Approach
11.1.5 Minimax Approach
11.1.6 Randomized Decision Rules
11.1.7 General Method for Finding Bayes Rules
11.2 Bayesian Binary Detection
11.2.1 Likelihood Ratio Test
11.2.2 Uniform Costs
11.2.3 Examples
11.3 Binary Minimax Detection
11.3.1 Bayes Risk Line and Minimum Risk Curve
11.3.2 Equalizer Rule
11.3.3 Examples
11.4 Binary Neyman-Pearson Detection
11.4.1 Solution to the N-P Optimization Problem
11.4.2 N-P Rule and Receiver Operating Characteristic
11.4.3 Examples
11.5 Bayesian Composite Detection
11.6 Neyman-Pearson Composite Detection
11.6.1 UMP Detection with One Composite Hypothesis
11.6.2 UMP Detection with Both Composite Hypotheses
11.6.3 Generalized Likelihood Ratio (GLR) Detection
11.6.4 Locally Most Powerful (LMP) Detection
11.7 Binary Detection with Vector Observations
11.7.1 Conditionally Independent Observations
11.7.2 Deterministic Signals in Correlated Gaussian Noise
11.7.3 Gaussian Signals in Gaussian Noise
11.8 Summary and Further Reading
11.9 Exercises
References
12 Monte Carlo Methods for Statistical Signal Processing
12.1 Introduction
12.1.1 Model-Based Signal Processing
12.1.2 Examples
12.2 Monte Carlo Methods
12.3 Markov Chain Monte Carlo (MCMC) Methods
12.3.1 General MCMC Algorithms
12.3.2 Applications of MCMC in Digital Communications
12.4 Sequential Monte Carlo (SMC) Methods
12.4.1 General SMC Algorithms
12.4.2 Resampling Procedures
12.4.3 Applications of SMC in Bioinformatics
12.5 Conclusions and Further Readings
12.6 Exercises
References
13 Factor Graphs and Message Passing Algorithms
13.1 Introduction
13.1.1 Why Factor Graphs?
13.1.2 Literature Survey
13.1.3 Organization of the Chapter
13.2 Factor Graphs
13.3 Modeling Systems Using Factor Graphs
13.3.1 Behavioral Modeling
13.3.2 Probabilistic Modeling
13.4 Relationship with Other Probabilistic Graphical Models
13.4.1 Markov Random Fields
13.4.2 Bayesian Networks
13.5 Message Passing in Factor Graphs
13.5.1 Sum-Product Algorithm
13.5.2 Max-Product Algorithm
13.6 Factor Graphs with Cycles
13.6.1 Message Passing Schedules
13.6.2 Iterative Message Passing
13.7 Some General Remarks on Factor Graphs
13.7.1 Mappers
13.7.2 Hybrid Equality Constraint
13.7.3 Quantizers
13.7.4 Continuous Variables
13.8 Some Important Message Passing Algorithms
13.8.1 Forward/Backward Algorithm
13.8.2 The Viterbi Algorithm
13.8.3 Kalman Filter
13.8.4 Expectation Maximization (EM) Algorithm
13.9 Applications of Message Passing in Factor Graphs
13.9.1 Detection of OFDM Signals in the Presence of Carrier Frequency Offset and Phase Noise
13.10 Exercises
References
14 Unconstrained and Constrained Optimization Problems
14.1 Basics of Convex Analysis
14.2 Unconstrained vs. Constrained Optimization
14.2.1 Optimality Conditions for Unconstrained Optimization
14.2.2 Optimality Conditions for Constrained Optimization
14.2.3 Lagrangian Duality
14.3 Application Examples
14.4 Exercises
References
15 Linear Programming and Mixed Integer Programming
15.1 Linear Programming
15.1.1 General Presentation
15.1.2 Transformations of the Standard Problem
15.1.3 Optimality Characterization
15.1.4 Duality Aspects
15.1.5 The Simplex Method
15.1.6 Interior-Point Methods
15.2 Modeling Problems via Linear Programming
15.2.1 Optimization with 1-norm and ∞-norm
15.2.2 Chebyshev Center of a Polytope
15.2.3 Classification with Separating Hyperplanes
15.2.4 Linear Fractional Programming
15.2.5 Continuous Constraints and Discretization
15.3 Mixed Integer Programming
15.3.1 Problem Statement and LP Relaxation
15.3.2 Branch and Bound
15.3.3 Examples of Mixed Integer Problems
15.4 Historical Notes and Further Reading
15.5 Exercises
References
16 Majorization Theory and Applications
16.1 Majorization Theory
16.1.1 Basic Concepts
16.1.2 Schur-Convex/Concave Functions
16.1.3 Relation to Matrix Theory
16.1.4 Multiplicative Majorization
16.1.5 Stochastic Majorization
16.2 Applications of Majorization Theory
16.2.1 CDMA Sequence Design
16.2.2 Linear MIMO Transceiver Design
16.2.3 Nonlinear MIMO Transceiver Design
16.2.4 Impact of Correlation
16.2.5 Robust Design
16.3 Conclusions and Further Readings
16.4 Exercises
References
17 Queueing Theory
17.1 Introduction
17.2 Markov Chains
17.2.1 Discrete-Time Markov Chains
17.2.2 Continuous-Time Markov Chains
17.3 Queueing Models
17.4 M/M/1 Queue
17.4.1 Steady-State Probabilities for Number in System
17.4.2 Little’s Formula
17.4.3 Probability Distribution of Delay Through System
17.5 M/M/1/N Queue
17.5.1 Example: Queueing Model of a Packet Switch
17.6 M/M/N/N Queue
17.7 M/M/1 Queues in Tandem
17.7.1 Product Form
17.7.2 Jackson Networks
17.8 M/G/1 Queue
17.8.1 Embedded Markov Chain
17.8.2 Mean Number in System
17.8.3 Distribution of Number in System
17.8.4 Mean Delay Through System
17.8.5 Distribution of Delay Through System
17.8.6 Example: Mixed Packets
17.8.7 Example: Data Frame Retransmissions
17.9 Conclusions
17.10 Exercises
References
18 Network Optimization Techniques
18.1 Introduction
18.2 Basic Multicommodity Flow Networks Optimization Models
18.2.1 Notions and Notations
18.2.2 Link-Path vs. Node-Link Formulation Applied to Allocation Problems
18.2.3 Dimensioning Problems
18.3 Optimization Methods for Multicommodity Flow Networks
18.3.1 Decomposition Methods
18.3.2 Solving MIP Problems
18.3.3 Heuristic Approaches
18.4 Optimization Models for Multistate Networks
18.4.1 Protection Models
18.4.2 Restoration Models
18.4.3 Multihour and Multiperiod Design
18.5 Concluding Remarks
18.6 Exercises
References
19 Game Theory
19.1 Introduction
19.2 Utility Theory
19.3 Games on the Normal Form
19.3.1 Zero-Sum Games
19.3.2 Non-Zero-Sum Games
19.4 Noncooperative Games and the Nash Equilibrium
19.5 Cooperative Games
19.5.1 Bargaining without Transferable Utilities
19.5.2 Bargaining with Transferable Utilities
19.6 Games with Incomplete Information
19.7 Extensive Form Games
19.7.1 Nash Equilibrium in Extensive Form Games
19.7.2 Subgame-Perfect Equilibrium
19.7.3 Incomplete Information in Extensive Form Games
19.8 Repeated Games and Evolutionary Stability
19.8.1 Repeated Games
19.8.2 Evolutionary Game Theory
19.9 Coalitional Form/Characteristic Function Form
19.9.1 The Core
19.9.2 The Kernel
19.9.3 The Nucleolus
19.9.4 The Shapley Value
19.10 Mechanism Design and Implementation Theory
19.10.1 Social Choice
19.10.2 Auctions
19.10.3 Direct Revelation Principle
19.10.4 Clarke-Groves and AGV-Arrow Mechanism
19.11 Applications to Signal Processing and Communications
19.12 Acknowledgments
19.13 Exercises
References
20 A Short Course on Frame Theory
20.1 Examples of Signal Expansions
20.2 Signal Expansions in Finite-Dimensional Spaces
20.2.1 Orthonormal Bases
20.2.2 General Bases
20.2.3 Redundant Signal Expansions
20.3 Frames for General Hilbert Spaces
20.3.1 The Frame Operator
20.3.2 The Canonical Dual Frame
20.3.3 Signal Expansions
20.3.4 Tight Frames
20.3.5 Exact Frames and Biorthonormality
20.4 The Sampling Theorem
20.4.1 Sampling Theorem as a Frame Expansion
20.4.2 Design Freedom in Oversampled A/D Conversion
20.4.3 Noise Reduction in Oversampled A/D Conversion
20.5 Important Classes of Frames
20.5.1 Weyl-Heisenberg Frames
20.5.2 Wavelets
20.6 Exercises
References
Index
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Prev
Previous Chapter
Cover
Next
Next Chapter
Title Page
Mathematical Foundations
for
SIGNAL PROCESSING, COMMUNICATIONS, AND NETWORKING
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset