0%

Deep Learning with Python, Second Edition introduces the field of deep learning using Python and the powerful Keras library. In this revised and expanded new edition, Keras creator François Chollet offers insights for both novice and experienced machine learning practitioners. As you move through this book, you’ll build your understanding through intuitive explanations, crisp illustrations, and clear examples. You’ll quickly pick up the skills you need to start developing deep-learning applications.

Table of Contents

  1. Deep Learning with Python
  2. Copyright
  3. dedication
  4. brief contents
  5. contents
  6. front matter
    1. preface
    2. acknowledgments
    3. about this book
    4. Who should read this book
    5. About the code
    6. liveBook discussion forum
    7. about the author
    8. about the cover illustration
  7. 1 What is deep learning?
    1. 1.1 Artificial intelligence, machine learning, and deep learning
    2. 1.1.1 Artificial intelligence
    3. 1.1.2 Machine learning
    4. 1.1.3 Learning rules and representations from data
    5. 1.1.4 The “deep” in “deep learning”
    6. 1.1.5 Understanding how deep learning works, in three figures
    7. 1.1.6 What deep learning has achieved so far
    8. 1.1.7 Don’t believe the short-term hype
    9. 1.1.8 The promise of AI
    10. 1.2 Before deep learning: A brief history of machine learning
    11. 1.2.1 Probabilistic modeling
    12. 1.2.2 Early neural networks
    13. 1.2.3 Kernel methods
    14. 1.2.4 Decision trees, random forests, and gradient boosting machines
    15. 1.2.5 Back to neural networks
    16. 1.2.6 What makes deep learning different
    17. 1.2.7 The modern machine learning landscape
    18. 1.3 Why deep learning? Why now?
    19. 1.3.1 Hardware
    20. 1.3.2 Data
    21. 1.3.3 Algorithms
    22. 1.3.4 A new wave of investment
    23. 1.3.5 The democratization of deep learning
    24. 1.3.6 Will it last?
  8. 2 The mathematical building blocks of neural networks
    1. 2.1 A first look at a neural network
    2. 2.2 Data representations for neural networks
    3. 2.2.1 Scalars (rank-0 tensors)
    4. 2.2.2 Vectors (rank-1 tensors)
    5. 2.2.3 Matrices (rank-2 tensors)
    6. 2.2.4 Rank-3 and higher-rank tensors
    7. 2.2.5 Key attributes
    8. 2.2.6 Manipulating tensors in NumPy
    9. 2.2.7 The notion of data batches
    10. 2.2.8 Real-world examples of data tensors
    11. 2.2.9 Vector data
    12. 2.2.10 Timeseries data or sequence data
    13. 2.2.11 Image data
    14. 2.2.12 Video data
    15. 2.3 The gears of neural networks: Tensor operations
    16. 2.3.1 Element-wise operations
    17. 2.3.2 Broadcasting
    18. 2.3.3 Tensor product
    19. 2.3.4 Tensor reshaping
    20. 2.3.5 Geometric interpretation of tensor operations
    21. 2.3.6 A geometric interpretation of deep learning
    22. 2.4 The engine of neural networks: Gradient-based optimization
    23. 2.4.1 What’s a derivative?
    24. 2.4.2 Derivative of a tensor operation: The gradient
    25. 2.4.3 Stochastic gradient descent
    26. 2.4.4 Chaining derivatives: The Backpropagation algorithm
    27. 2.5 Looking back at our first example
    28. 2.5.1 Reimplementing our first example from scratch in TensorFlow
    29. 2.5.2 Running one training step
    30. 2.5.3 The full training loop
    31. 2.5.4 Evaluating the model
    32. Summary
  9. 3 Introduction to Keras and TensorFlow
    1. 3.1 What’s TensorFlow?
    2. 3.2 What’s Keras?
    3. 3.3 Keras and TensorFlow: A brief history
    4. 3.4 Setting up a deep learning workspace
    5. 3.4.1 Jupyter notebooks: The preferred way to run deep learning experiments
    6. 3.4.2 Using Colaboratory
    7. 3.5 First steps with TensorFlow
    8. 3.5.1 Constant tensors and variables
    9. 3.5.2 Tensor operations: Doing math in TensorFlow
    10. 3.5.3 A second look at the GradientTape API
    11. 3.5.4 An end-to-end example: A linear classifier in pure TensorFlow
    12. 3.6 Anatomy of a neural network: Understanding core Keras APIs
    13. 3.6.1 Layers: The building blocks of deep learning
    14. 3.6.2 From layers to models
    15. 3.6.3 The “compile” step: Configuring the learning process
    16. 3.6.4 Picking a loss function
    17. 3.6.5 Understanding the fit() method
    18. 3.6.6 Monitoring loss and metrics on validation data
    19. 3.6.7 Inference: Using a model after training
    20. Summary
  10. 4 Getting started with neural networks: Classification and regression
    1. 4.1 Classifying movie reviews: A binary classification example
    2. 4.1.1 The IMDB dataset
    3. 4.1.2 Preparing the data
    4. 4.1.3 Building your model
    5. 4.1.4 Validating your approach
    6. 4.1.5 Using a trained model to generate predictions on new data
    7. 4.1.6 Further experiments
    8. 4.1.7 Wrapping up
    9. 4.2 Classifying newswires: A multiclass classification example
    10. 4.2.1 The Reuters dataset
    11. 4.2.2 Preparing the data
    12. 4.2.3 Building your model
    13. 4.2.4 Validating your approach
    14. 4.2.5 Generating predictions on new data
    15. 4.2.6 A different way to handle the labels and the loss
    16. 4.2.7 The importance of having sufficiently large intermediate layers
    17. 4.2.8 Further experiments
    18. 4.2.9 Wrapping up
    19. 4.3 Predicting house prices: A regression example
    20. 4.3.1 The Boston housing price dataset
    21. 4.3.2 Preparing the data
    22. 4.3.3 Building your model
    23. 4.3.4 Validating your approach using K-fold validation
    24. 4.3.5 Generating predictions on new data
    25. 4.3.6 Wrapping up
    26. Summary
  11. 5 Fundamentals of machine learning
    1. 5.1 Generalization: The goal of machine learning
    2. 5.1.1 Underfitting and overfitting
    3. 5.1.2 The nature of generalization in deep learning
    4. 5.2 Evaluating machine learning models
    5. 5.2.1 Training, validation, and test sets
    6. 5.2.2 Beating a common-sense baseline
    7. 5.2.3 Things to keep in mind about model evaluation
    8. 5.3 Improving model fit
    9. 5.3.1 Tuning key gradient descent parameters
    10. 5.3.2 Leveraging better architecture priors
    11. 5.3.3 Increasing model capacity
    12. 5.4 Improving generalization
    13. 5.4.1 Dataset curation
    14. 5.4.2 Feature engineering
    15. 5.4.3 Using early stopping
    16. 5.4.4 Regularizing your model
    17. Summary
  12. 6 The universal workflow of machine learning
    1. 6.1 Define the task
    2. 6.1.1 Frame the problem
    3. 6.1.2 Collect a dataset
    4. 6.1.3 Understand your data
    5. 6.1.4 Choose a measure of success
    6. 6.2 Develop a model
    7. 6.2.1 Prepare the data
    8. 6.2.2 Choose an evaluation protocol
    9. 6.2.3 Beat a baseline
    10. 6.2.4 Scale up: Develop a model that overfits
    11. 6.2.5 Regularize and tune your model
    12. 6.3 Deploy the model
    13. 6.3.1 Explain your work to stakeholders and set expectations
    14. 6.3.2 Ship an inference model
    15. 6.3.3 Monitor your model in the wild
    16. 6.3.4 Maintain your model
    17. Summary
  13. 7 Working with Keras: A deep dive
    1. 7.1 A spectrum of workflows
    2. 7.2 Different ways to build Keras models
    3. 7.2.1 The Sequential model
    4. 7.2.2 The Functional API
    5. 7.2.3 Subclassing the Model class
    6. 7.2.4 Mixing and matching different components
    7. 7.2.5 Remember: Use the right tool for the job
    8. 7.3 Using built-in training and evaluation loops
    9. 7.3.1 Writing your own metrics
    10. 7.3.2 Using callbacks
    11. 7.3.3 Writing your own callbacks
    12. 7.3.4 Monitoring and visualization with TensorBoard
    13. 7.4 Writing your own training and evaluation loops
    14. 7.4.1 Training versus inference
    15. 7.4.2 Low-level usage of metrics
    16. 7.4.3 A complete training and evaluation loop
    17. 7.4.4 Make it fast with tf.function
    18. 7.4.5 Leveraging fit() with a custom training loop
    19. Summary
  14. 8 Introduction to deep learning for computer vision
    1. 8.1 Introduction to convnets
    2. 8.1.1 The convolution operation
    3. 8.1.2 The max-pooling operation
    4. 8.2 Training a convnet from scratch on a small dataset
    5. 8.2.1 The relevance of deep learning for small-data problems
    6. 8.2.2 Downloading the data
    7. 8.2.3 Building the model
    8. 8.2.4 Data preprocessing
    9. 8.2.5 Using data augmentation
    10. 8.3 Leveraging a pretrained model
    11. 8.3.1 Feature extraction with a pretrained model
    12. 8.3.2 Fine-tuning a pretrained model
    13. Summary
  15. 9 Advanced deep learning for computer vision
    1. 9.1 Three essential computer vision tasks
    2. 9.2 An image segmentation example
    3. 9.3 Modern convnet architecture patterns
    4. 9.3.1 Modularity, hierarchy, and reuse
    5. 9.3.2 Residual connections
    6. 9.3.3 Batch normalization
    7. 9.3.4 Depthwise separable convolutions
    8. 9.3.5 Putting it together: A mini Xception-like model
    9. 9.4 Interpreting what convnets learn
    10. 9.4.1 Visualizing intermediate activations
    11. 9.4.2 Visualizing convnet filters
    12. 9.4.3 Visualizing heatmaps of class activation
    13. Summary
  16. 10 Deep learning for timeseries
    1. 10.1 Different kinds of timeseries tasks
    2. 10.2 A temperature-forecasting example
    3. 10.2.1 Preparing the data
    4. 10.2.2 A common-sense, non-machine learning baseline
    5. 10.2.3 Let’s try a basic machine learning model
    6. 10.2.4 Let’s try a 1D convolutional model
    7. 10.2.5 A first recurrent baseline
    8. 10.3 Understanding recurrent neural networks
    9. 10.3.1 A recurrent layer in Keras
    10. 10.4 Advanced use of recurrent neural networks
    11. 10.4.1 Using recurrent dropout to fight overfitting
    12. 10.4.2 Stacking recurrent layers
    13. 10.4.3 Using bidirectional RNNs
    14. 10.4.4 Going even further
    15. Summary
  17. 11 Deep learning for text
    1. 11.1 Natural language processing: The bird’s eye view
    2. 11.2 Preparing text data
    3. 11.2.1 Text standardization
    4. 11.2.2 Text splitting (tokenization)
    5. 11.2.3 Vocabulary indexing
    6. 11.2.4 Using the TextVectorization layer
    7. 11.3 Two approaches for representing groups of words: Sets and sequences
    8. 11.3.1 Preparing the IMDB movie reviews data
    9. 11.3.2 Processing words as a set: The bag-of-words approach
    10. 11.3.3 Processing words as a sequence: The sequence model approach
    11. 11.4 The Transformer architecture
    12. 11.4.1 Understanding self-attention
    13. 11.4.2 Multi-head attention
    14. 11.4.3 The Transformer encoder
    15. 11.4.4 When to use sequence models over bag-of-words models
    16. 11.5 Beyond text classification: Sequence-to-sequence learning
    17. 11.5.1 A machine translation example
    18. 11.5.2 Sequence-to-sequence learning with RNNs
    19. 11.5.3 Sequence-to-sequence learning with Transformer
    20. Summary
  18. 12 Generative deep learning
    1. 12.1 Text generation
    2. 12.1.1 A brief history of generative deep learning for sequence generation
    3. 12.1.2 How do you generate sequence data?
    4. 12.1.3 The importance of the sampling strategy
    5. 12.1.4 Implementing text generation with Keras
    6. 12.1.5 A text-generation callback with variable-temperature sampling
    7. 12.1.6 Wrapping up
    8. 12.2 DeepDream
    9. 12.2.1 Implementing DeepDream in Keras
    10. 12.2.2 Wrapping up
    11. 12.3 Neural style transfer
    12. 12.3.1 The content loss
    13. 12.3.2 The style loss
    14. 12.3.3 Neural style transfer in Keras
    15. 12.3.4 Wrapping up
    16. 12.4 Generating images with variational autoencoders
    17. 12.4.1 Sampling from latent spaces of images
    18. 12.4.2 Concept vectors for image editing
    19. 12.4.3 Variational autoencoders
    20. 12.4.4 Implementing a VAE with Keras
    21. 12.4.5 Wrapping up
    22. 12.5 Introduction to generative adversarial networks
    23. 12.5.1 A schematic GAN implementation
    24. 12.5.2 A bag of tricks
    25. 12.5.3 Getting our hands on the CelebA dataset
    26. 12.5.4 The discriminator
    27. 12.5.5 The generator
    28. 12.5.6 The adversarial network
    29. 12.5.7 Wrapping up
    30. Summary
  19. 13 Best practices for the real world
    1. 13.1 Getting the most out of your models
    2. 13.1.1 Hyperparameter optimization
    3. 13.1.2 ensembling
    4. 13.2 Scaling-up model training
    5. 13.2.1 Speeding up training on GPU with mixed precision
    6. 13.2.2 Multi-GPU training
    7. 13.2.3 TPU training
    8. Summary
  20. 14 Conclusions
    1. 14.1 Key concepts in review
    2. 14.1.1 Various approaches to AI
    3. 14.1.2 What makes deep learning special within the field of machine learning
    4. 14.1.3 How to think about deep learning
    5. 14.1.4 Key enabling technologies
    6. 14.1.5 The universal machine learning workflow
    7. 14.1.6 Key network architectures
    8. 14.1.7 The space of possibilities
    9. 14.2 The limitations of deep learning
    10. 14.2.1 The risk of anthropomorphizing machine learning models
    11. 14.2.2 Automatons vs. intelligent agents
    12. 14.2.3 Local generalization vs. extreme generalization
    13. 14.2.4 The purpose of intelligence
    14. 14.2.5 Climbing the spectrum of generalization
    15. 14.3 Setting the course toward greater generality in AI
    16. 14.3.1 On the importance of setting the right objective: The shortcut rule
    17. 14.3.2 A new target
    18. 14.4 Implementing intelligence: The missing ingredients
    19. 14.4.1 Intelligence as sensitivity to abstract analogies
    20. 14.4.2 The two poles of abstraction
    21. 14.4.3 The missing half of the picture
    22. 14.5 The future of deep learning
    23. 14.5.1 Models as programs
    24. 14.5.2 Blending together deep learning and program synthesis
    25. 14.5.3 Lifelong learning and modular subroutine reuse
    26. 14.5.4 The long-term vision
    27. 14.6 Staying up to date in a fast-moving field
    28. 14.6.1 Practice on real-world problems using Kaggle
    29. 14.6.2 Read about the latest developments on arXiv
    30. 14.6.3 Explore the Keras ecosystem
    31. Final words
  21. index
54.196.52.45