The process of deriving real‐world application from scientific knowledge is usually a very, very long process. However, with the advancement in complementary metal oxide semiconductor (CMOS) image sensor, and its application in handheld device, image interpolation has rapidly migrated from complex mathematics and academic publications to everyday applications in smartphones, laptops and tablets. Image interpolation has become a red‐hot research topic in both academia and industry. One of the highly cited academic works in image interpolation is authored by Dr. Tam, which is an excerpt from her master thesis. Her work is also the origin of this book. However, this book is not intended to be a memoir of the work done by Dr. Tam and her research group; it is intended to be the course materials for senior‐ and graduate‐level courses, training materials for engineers, and also a reference text for readers who are working in the field of digital imaging.

All the image interpolation algorithms discussed in this book will include both theories, where detailed analytic analysis are derived, and implementations through MATLAB into useful tools. Numerous algorithms are reviewed in this book together with detailed discussions on their origins, performances, and limitations. We are particularly happy with the numerical simulations presented for all the algorithms described in this book to clarify the observable but difficult to explain image interpolation artifacts, as the author shares the well‐known Chinese saying that a picture is worth a thousand words. Furthermore, many of our unpublished works are included in this book, where new algorithms are developed to overcome various limitations.

This book is authored as much as it is collected. We have tried our best to cite references whenever we are aware of related works on the topics. However, we suspect that some topics may have been independently studied by many individuals, and thus we might have missed their citation. Over 30 years of research works are collected in one place, and we presented each selected topics in a self‐contained format. If you are interested in further reading on any of these topics, you should look into the cited references and the Summary sections at the end of each chapter in this book. On a subject such as this one, which has been continuously investigated for over half a century, inevitably a number of valuable research results are not included in this book. It is nonetheless expected that the contents of this book will enable the careful readers to independently explore the more advanced image interpolation/processing technique.

Although much of the materials covered by this book are new to most students, our goal is to provide a working knowledge of various image interpolation algorithms without the need for additional course work besides freshman‐level engineering mathematics and a junior‐level matrix lab programming. To perform numerical simulation using computer, we must use a language that a computer can understand. This is why we choose to use MATLAB in this book, because MATLAB is not only a computer language. MATLAB, which is built with matrix data structure, is also a language of arithmetic. Once the MATLAB implementation of the algorithms have been learned, it will be fairly straightforward to implement them in other computer languages and VHDL for hardware synthesis. While almost all the MATLAB example codes presented in this book are co‐developed from the basic and do not require any toolbox to run with, in Chapter 6, the author just cannot resist to make use of the wavelet toolbox developed by Prof. T.Q. Nguyen of UCSD who is also the PhD adviser of Dr. Kok back in the University of Wisconsin–Madison. The toolbox has made everything easy, which definitely helped the readers to understand the topics and ease their practical implementation tremendously.

The book is divided into nine chapters. Chapter 1 provides an account of basic signal processing and mathematical tools used in subsequent chapters. It also serves the purpose of getting the readers to be familiar with the mathematical notations adopted in the book. Chapter 2 introduces the important concepts of digital imaging and the operations that are useful to image interpolation algorithms. The quality and performance measures between the processed image and the original image are presented in Chapter 3. The human visual system that is first discussed in Chapter 2 will be extended here for the discussion of the structural similarity quality index. The nonparametric image interpolation algorithm developed around algebraic functions are presented in Chapter 4. This chapter ends with a discussion on the deficiency of nonadaptive interpolation methods. Chapter 5 discusses the interpolation by Fourier and other orthogonal series. We are particularly interested in interpolating image in the discrete cosine transform domain, which is motivated by current trends in international image compression and storage standards. The blocking noise resulted from transform domain zero padding interpolation with small block size is alleviated by variations of overlap and add interpolation techniques. An iterative algorithm is presented to improve the least squares solution of the conventional transform coefficients zero padding image interpolation algorithm. Note that iterative image interpolation algorithms are considered to be offline image interpolation algorithms. More about iterative interpolation algorithm that helps to maintain the original pixel values while improving the performance of the non‐iterative image interpolation algorithms will be presented in subsequent chapters. Chapter 6 extends the block‐based transform domain image interpolation to the wavelet domain. A number of the techniques presented in previous chapters are applicable to the wavelet domain image interpolation too, and various researchers have been given them different names in the literature. The performance of wavelet image interpolation can be improved by exploiting the scale‐space relationships obtained by multi‐resolution analysis through wavelet transform (a version of the human visual system). The explicit edge detection‐based image interpolation methods discussed in Chapter 7 interpolate the image according to the edge‐directed image perception property of human visual system. Various edge‐directed interpolation methods will be discussed where edges are explicitly obtained by various edge detection methods discussed in Chapter 2, and implicit edge detection methods that the nature of the pixels to be interpolated is determined in the course of the estimation. The chapter concludes with discussions on the pros and cons of edge‐directed image interpolation algorithm using explicit edge detection. Another type of edge‐detected image interpolation method will be presented in Chapter 8, which is based on the edge geometric duality where a covariance‐based implicit edge location and estimation method will interpolate the image along the edge to achieve good visual quality. Digital signal processing theory tells us that there is always room to improve the solutions of any estimation problem. Various improvements to the edge‐directed interpolation problem will be discussed in this chapter to improve the preservation of edge geometric duality between the original image and the interpolated image, to reduce the interpolation error propagation by removing inter‐processing dependence, and finally to improve the estimation solution through an iterative re‐estimation algorithm. The book changes its course from linear statistical‐based interpolation technique to fractal interpolation in Chapter 9.

It should be noticed that fractal is usually not considered to be a statistical‐based interpolation algorithm. On the other hand, the generation of fractal map is based on similarity between image features, where the similarity is computed or classified via the statistics of the image or image blocks. Finally, an iterative algorithm is presented to improve the fractal image interpolation algorithm with the constraint that the original low‐resolution image is the pivot of the interpolated image, i.e. the location and intensity invariance of the low‐resolution image in the interpolation image is guaranteed. The advantage of such algorithmic constraint not only allows the preservation of the original low‐resolution image pixel values in the interpolated image but also ensures the highest preservation of the structure property of the interpolated image. As a result, fractal image interpolation has been embedded in a number of successful image processing softwares. The book concludes with an appendix that lists all the MATLAB source codes discussed in the book.

Many people have contributed, directly or indirectly, over a long period of time, to the subjects presented in this book. Their contributions are cited appropriately in this book, and also in the Summary section at the end of each chapter. The Summary sections also aimed to detail the state‐of‐the‐art development with respect to the topics discussed in each chapter. The exercises presented in the Exercise sections are essential parts of this text and often provide a discovery‐like experience regarding the associated topics. It is our hope that the exercises will provide general guidelines to assist the readers to design new image interpolation algorithms for their own applications. The readers' effort spent on tackling the exercises will help them to develop a thorough consideration on the design of image processing algorithms for their future career in research and development in the field.

The book is definitely not meant to represent a comprehensive history about the development of image interpolation algorithms. On the other hand, it does provide a not so short review, which chronologically follows the evolution of some of the image interpolation algorithms that have direct implications on commercially available image processing softwares. In particular, we avoided with our best effort to provide a comprehensive survey of every image interpolation algorithms in literature and market. Instead, our selection of topics is on the importance of the algorithms with respect to their applications in image processing softwares in today's or near‐future market. Our hope is that the book offers the readers a range of interesting topics and the current state‐of‐the‐art image interpolation methods. In simple terms, image interpolation is an open problem that has no definite winner. Analyzing the design and performance trade‐offs and proposing a range of attractive solutions to various image interpolation problems are the basic aims of this book. The book will underline the range of design considerations in an unbiased fashion, and the readers will be able to glean information from it in order to solve their own particular image interpolation problems. Most of all, we hope that the readers will find it an enjoyable and relatively effortless reading, providing them with intellectual stimulation.