Cover image for Hands-On Docker for Microservices with Python

Hands-On Docker for Microservices with Python

Author Jaime Buelta
Release Date: 2019/11/01
ISBN: 9781838823818
Topic:
0%
318
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

A step-by-step guide to building microservices using Python and Docker, along with managing and orchestrating them with Kubernetes

Key Features

  • Learn to use Docker containers to create, operate, and deploy your microservices
  • Create workflows to manage independent deployments on coordinating services using CI and GitOps through GitHub, Travis CI, and Flux
  • Develop a REST microservice in Python using the Flask framework and Postgres database

Book Description

Microservices architecture helps create complex systems with multiple, interconnected services that can be maintained by independent teams working in parallel. This book guides you on how to develop these complex systems with the help of containers.

You'll start by learning to design an efficient strategy for migrating a legacy monolithic system to microservices. You'll build a RESTful microservice with Python and learn how to encapsulate the code for the services into a container using Docker. While developing the services, you'll understand how to use tools such as GitHub and Travis CI to ensure continuous delivery (CD) and continuous integration (CI). As the systems become complex and grow in size, you'll be introduced to Kubernetes and explore how to orchestrate a system of containers while managing multiple services. Next, you'll configure Kubernetes clusters for production-ready environments and secure them for reliable deployments. In the concluding chapters, you'll learn how to detect and debug critical problems with the help of logs and metrics. Finally, you'll discover a variety of strategies for working with multiple teams dealing with different microservices for effective collaboration.

By the end of this book, you'll be able to build production-grade microservices as well as orchestrate a complex system of services using containers.

What you will learn

  • Discover how to design, test, and operate scalable microservices
  • Coordinate and deploy different services using Kubernetes
  • Use Docker to construct scalable and manageable applications with microservices
  • Understand how to monitor a complete system to ensure early detection of problems
  • Become well versed with migrating from an existing monolithic system to a microservice one
  • Use load balancing to ensure seamless operation between the old monolith and the new service

Who this book is for

This book is for developers, engineers, or software architects who are trying to move away from traditional approaches for building complex multi-service systems by adopting microservices and containers. Although familiarity with Python programming is assumed, no prior knowledge of Docker is required.

Table of Contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On Docker for Microservices with Python
  3. Dedication
  4. About Packt
    1. Why subscribe?
  5. Contributors
    1. About the author
    2. About the reviewer
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Code in Action
      4. Conventions used
    4. Get in touch
      1. Reviews
  7. Section 1: Introduction to Microservices
  8. Making the Move – Design, Plan, and Execute
    1. Technical requirements
    2. The traditional monolith approach and its problems
    3. The characteristics of a microservices approach
      1. Docker containers
        1. Container orchestration and Kubernetes
    4. Parallel deployment and development speed
    5. Challenges and red flags
    6. Analyzing the current system
    7. Preparing and adapting by measuring usage
    8. Strategic planning to break the monolith
      1. The replacement approach
      2. The divide approach
      3. Change and structured approach
    9. Executing the move
      1. Web services' best friend – the load balancer
      2. Keeping the balance between new and old
        1. The pilot phase – setting up the first couple of microservices
        2. The consolidation phase – steady migration to microservices
        3. The final phase – the microservices shop
    10. Summary
    11. Questions
    12. Further reading
  9. Section 2: Designing and Operating a Single Service – Creating a Docker Container
  10. Creating a REST Service with Python
    1. Technical requirements
    2. Analyzing the Thoughts Backend microservice
      1. Understanding the security layer
    3. Designing the RESTful API
      1. Specifying the API endpoints
    4. Defining the database schema
      1. Working with SQLAlchemy
    5. Implementing the service
      1. Introducing Flask-RESTPlus
      2. Handling resources
      3. Parsing input parameters
      4. Serializing results
      5. Performing the action
      6. Authenticating the requests
    6. Testing the code
      1. Defining the pytest fixtures
      2. Understanding test_token_validation.py
      3. test_thoughts.py
    7. Summary
    8. Questions
    9. Further reading
  11. Build, Run, and Test Your Service Using Docker
    1. Technical requirements
    2. Building your service with a Dockerfile
      1. Executing commands
      2. Understanding the Docker cache
      3. Building a web service container
      4. Configuring uWSGI
      5. Refreshing Docker commands
    3. Operating with an immutable container
      1. Testing the container
      2. Creating a PostgreSQL database container
    4. Configuring your service
    5. Deploying the Docker service locally
    6. Pushing your Docker image to a remote registry
      1. Obtaining public images from Docker Hub
      2. Using tags
      3. Pushing into a registry
    7. Summary
    8. Questions
    9. Further reading
  12. Creating a Pipeline and Workflow
    1. Technical requirements
    2. Understanding continuous integration practices
      1. Producing automated builds
      2. Knowing the advantages of using Docker for builds
      3. Leveraging the pipeline concept
      4. Branching, merging, and ensuring a clear main build
    3. Configuring Travis CI
      1. Adding a repo to Travis CI
      2. Creating the .travis.yml file
      3. Working with Travis jobs
      4. Sending notifications
    4. Configuring GitHub
    5. Pushing Docker images from Travis CI
      1. Setting the secret variables
      2. Tagging and pushing builds
      3. Tagging and pushing every commit
    6. Summary
    7. Questions
    8. Further reading
  13. Section 3:Working with Multiple Services – Operating the System through Kubernetes
  14. Using Kubernetes to Coordinate Microservices
    1. Technical requirements
    2. Defining the Kubernetes orchestrator
      1. Comparing Kubernetes with Docker Swarm
    3. Understanding the different Kubernetes elements
      1. Nodes
      2. Kubernetes Control Plane
      3. Kubernetes Objects
    4. Performing basic operations with kubectl
      1. Defining an element
      2. Getting more information
      3. Removing an element
    5. Troubleshooting a running cluster
    6. Summary
    7. Questions
    8. Further reading
  15. Local Development with Kubernetes
    1. Technical requirements
    2. Implementing multiple services
      1. Describing the Users Backend microservice
      2. Describing the Frontend microservice
      3. Connecting the services
    3. Configuring the services
      1. Configuring the deployment
      2. Configuring the service
      3. Configuring the Ingress
    4. Deploying the full system locally
      1. Deploying the Users Backend
      2. Adding the Frontend
    5. Summary
    6. Questions
    7. Further reading
  16. Configuring and Securing the Production System
    1. Technical requirements
    2. Using Kubernetes in the wild
      1. Creating an IAM user
    3. Setting up the Docker registry
    4. Creating the cluster
      1. Creating the Kubernetes cluster
      2. Configuring the cloud Kubernetes cluster
        1. Configuring the AWS image registry
        2. Configuring the usage of an externally accessible load balancer
        3. Deploying the system
    5. Using HTTPS and TLS to secure external access
    6. Being ready for migration to microservices
      1. Running the example
    7. Deploying a new Docker image smoothly
      1. The liveness probe
      2. The readiness probe
      3. Rolling updates
    8. Autoscaling the cluster
      1. Creating a Kubernetes Horizontal Pod Autoscaler
        1. Deploying the Kubernetes metrics server
        2. Configuring the resources in deployments
        3. Creating an HPA
      2. Scaling the number of nodes in the cluster
        1. Deleting nodes
      3. Designing a winning autoscaling strategy
    9. Summary
    10. Questions
    11. Further reading
  17. Using GitOps Principles
    1. Technical requirements
    2. Understanding the description of GitOps
      1. Managing configuration
      2. Understanding DevOps
      3. Defining GitOps
    3. Setting up Flux to control the Kubernetes cluster
      1. Starting the system
      2. Configuring Flux
    4. Configuring GitHub
      1. Forking the GitHub repo
      2. Adding a deploy key
      3. Syncing Flux
    5. Making a Kubernetes cluster change through GitHub
    6. Working in production
      1. Creating structure
      2. Using GitHub features
      3. Working with tags
    7. Summary
    8. Questions
    9. Further reading
  18. Managing Workflows
    1. Understanding the life cycle of a feature
      1. Features that affect multiple microservices
      2. Implementing a feature
    2. Reviewing and approving a new feature
      1. Reviewing feature code
      2. Approving releases
    3. Setting up multiple environments
    4. Scaling the workflow and making it work
      1. Reviewing and approving is done by the whole team
      2. Understanding that not every approval is the same
      3. Defining a clear path for releases
        1. Emergency releases
      4. Releasing frequently and adding feature flags
        1. Using feature flags
      5. Dealing with database migrations
    5. Summary
    6. Questions
    7. Further reading
  19. Section 4: Production-Ready System – Making It Work in Real-Life Environments
  20. Monitoring Logs and Metrics
    1. Technical requirements
    2. Observability of a live system
      1. Understanding logs
      2. Understanding metrics
    3. Setting up logs
      1. Setting up an rsyslog container
      2. Defining the syslog pod
        1. log-volume
        2. syslog container
        3. The front rail container
      3. Allowing external access
      4. Sending logs
      5. Generating application logs
        1. Dictionary configuration
        2. Logging a request ID
        3. Logging each request
        4. Searching through all the logs
    4. Detecting problems through logs
      1. Detecting expected errors
      2. Capturing unexpected errors
      3. Logging strategy
      4. Adding logs while developing
    5. Setting up metrics
      1. Defining metrics for the Thoughts Backend
        1. Adding custom metrics
      2. Collecting the metrics
      3. Plotting graphs and dashboards
        1. Grafana UI
        2. Querying Prometheus
        3. Updating dashboards
    6. Being proactive
      1. Alerting
      2. Being prepared
    7. Summary
    8. Questions
    9. Further reading
  21. Handling Change, Dependencies, and Secrets in the System
    1. Technical requirements
    2. Understanding shared configurations across microservices
      1. Adding the ConfigMap file
        1. Using kubectl commands
      2. Adding ConfigMap to the deployment
        1. Thoughts Backend ConfigMap configuration
        2. Users Backend ConfigMap configuration
        3. Frontend ConfigMap configuration
    3. Handling Kubernetes secrets
      1. Storing secrets in Kubernetes
      2. Creating the secrets
        1. Storing the secrets in the cluster
        2. Secret deployment configuration
        3. Retrieving the secrets by the applications
    4. Defining a new feature affecting multiple services
      1. Deploying one change at a time
      2. Rolling back the microservices
    5. Dealing with service dependencies
      1. Versioning the services
        1. Semantic versioning
      2. Adding a version endpoint
        1. Obtaining the version
        2. Storing the version in the image
        3. Implementing the version endpoint
      3. Checking the version
        1. Required version
        2. The main function
        3. Checking the version
    6. Summary
    7. Questions
    8. Further reading
  22. Collaborating and Communicating across Teams
    1. Keeping a consistent architectural vision
    2. Dividing the workload and Conway's Law
      1. Describing Conway's Law
      2. Dividing the software into different kinds of software units
      3. Designing working structures
        1. Structuring teams around technologies
        2. Structuring teams around domains
        3. Structuring teams around customers
        4. Structuring teams around a mix
    3. Balancing new features and maintenance
      1. Regular maintenance
      2. Understanding technical debt
      3. Continuously addressing technical debt
      4. Avoiding technical debt
    4. Designing a broader release process
      1. Planning in the weekly release meeting
      2. Reflecting on release problems
      3. Running post-mortem meetings
    5. Summary
    6. Questions
    7. Further reading
  23. Assessments
    1. Chapter 1
    2. Chapter 2
    3. Chapter 3
    4. Chapter 4
    5. Chapter 5
    6. Chapter 6
    7. Chapter 7
    8. Chapter 8
    9. Chapter 9
    10. Chapter 10
    11. Chapter 11
    12. Chapter 12
  24. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think
18.118.137.243