In 2015, Google made TensorFlow open source, including all of its reference implementation. All of the source code was made available on GitHub under the Apache 2.0 license. Since then, TensorFlow has been widely adopted in academia and industrial research, and the most stable version, 1.6, has recently been released with a unified API.
It is important to note that the APIs in TensorFlow 1.6 (and higher) are not all backward compatible for pre v1.5 code. This means that some programs that worked on pre v1.5 will not necessarily work on TensorFlow 1.6.
Now let us see the new and exciting features that TensorFlow v1.6 has.
From TensorFlow v1.5, prebuilt binaries are now built against CUDA 9.0 and cuDNN 7. However, from v1.6's release, TensorFlow prebuilt binaries use AVX instructions, which may break TensorFlow on older CPUs. Nevertheless, since v1.5, an added support for CUDA on NVIDIA Tegra devices has been available.
TensorFlow Lite is TensorFlow's lightweight solution for mobile and embedded devices. It enables low-latency inference of on-device machine learning models with a small binary size and fast performance supporting hardware acceleration.
TensorFlow Lite uses many techniques for achieving low latency like optimizing the kernels for specific mobile apps, pre-fused activations, quantized kernels that allow smaller and faster (fixed-point math) models, and in the future, leverage-specialized machine learning hardware to get the best possible performance for a particular model on a particular device.
Figure 1: A conceptual view on how to use trained model on Android and iOS devices using TensorFlow Lite
Machine learning is changing the computing paradigm, and we see an emerging trend of new use cases on mobile and embedded devices. Consumer expectations are also trending toward natural, human-like interactions with their devices, driven by the camera and voice interaction models.
Therefore, the user's expectations are no longer limited to the computer, and the computational power of mobile devices has also increased exponentially due to hardware acceleration, and frameworks such as the Android Neural Networks API and C++ API for iOS. As shown in the preceding figure, a pre-trained model can be converted into a lighter version to be running as an Android or iOS app.
Therefore, widely available smart appliances create new possibilities for on-device intelligence. These allow us to use our smartphones to perform real-time computer vision and Natural Language Processing (NLP).
Eager execution is an interface for TensorFlow that provides an imperative programming style. When you enable eager execution, TensorFlow operations (defined in a program) execute immediately.
It is to be noted that from TensorFlow v1.7, eager execution will be moved out of contrib. This means that using tf.enable_eager_execution() is recommended. We will see an example on this in a later section.
Pre v1.5 XLA was unstable and had a very limited number of features. However, v1.6 has more support for XLA. This includes the following:
- Complex64 support to XLA compiler has been added
- Now the Fast Fourier Transformation (FFT) support has been added for both CPU and GPU
- The bfloat support is now added to the XLA infrastructure
- The ClusterSpec propagation work with XLA devices has been enabled
- Android TF can now be built with CUDA acceleration on compatible Tegra devices
- Support for adding the deterministic executor to generate an XLA graph has been enabled
Numerous bugs reported by the open source community have been fixed and a significant amount of API-level changes have been integrated with this version.
However, since we have not explored anything with TensorFlow yet, we will see how to leverage these features for developing real-life deep learning applications later on. Before that, let's see how to prepare your programming environment.