Deep learning has achieved tremendous success in various applications of machine learning, data analytics, and computer vision. It is easy to be parallelized with a low inference complexity and could be jointly tuned in an end-to-end manner. However, generic deep architectures, often referred to as “black-box” methods, largely ignore the problem-specific formulations and domain knowledge. They rely on stacking somewhat ad-hoc modules, which makes it prohibitive to interpret their working mechanisms. Despite a few hypotheses and intuitions, it is widely recognized as difficult to understand why deep models work, and how they can be related to classical machine learning models. On the other hand, sparsity and low rankness are well exploited regularization in classical machine learning. By exploiting the latent low-dimensional subspace structure of high-dimension data, they have also led to great success in many image processing and understanding tasks.

This book provides an overview on the recent research trend on integrated deep learning models with sparse and low rank models. It will be suitable for the audiences who have basic knowledge of deep learning and sparse/low rank models, and place strong emphasis on the concepts and applications, in the hope that it can reach a broader audience. The research advances covered in this book bridge the classical sparse and low rank models that emphasize problem-specific prior and interpretability, with deep network models that allow for larger learning capacity and better utilization of Big Data. The toolkit of deep learning will be shown to be closely tied with the sparse/low rank models and algorithms. Such a viewpoint is expected to motivate a rich variety of theoretical and analytic tools, to guide the architecture design and interpretation of deep models. The theoretical and modeling progress will be complemented with many applications in computer vision, machine learning, signal processing, data mining, and more.

The Authors