Preface

As data warehouse management becomes increasingly integral to successful organizations, choosing and running the right solution is more important than ever. Microsoft Azure Synapse is an enterprise-grade, cloud-based data warehousing platform, and this book holds the key to using Synapse to its full potential. If you want the skills and confidence to create a robust enterprise analytical platform, this cookbook is a great place to start.

You'll learn about and execute enterprise-level deployment on medium-to-large data platforms. Using the step-by-step recipes and accompanying theory covered in the book, you'll understand how to integrate various services with Synapse to make it a robust solution for all your data needs. Whether you're new to Azure Synapse or just getting started, you'll find the instructions you need to solve any problem you may face, including using Azure services for data visualization as well as Artificial Intelligence (AI) and Machine Learning (ML) solutions.

By the end of this Azure book, you'll have the skills you need to implement an enterprise-grade analytical platform, enabling your organization to explore and manage heterogeneous data workloads and employ various data integration services to solve real-time industry problems.

Who this book is for

This book is for data architects, data engineers, and developers who want to learn and understand the main concepts of Azure Synapse Analytics and implement them in real-world scenarios.

What this book covers

Chapter 1, Choosing the Optimal Method for Loading Data to Synapse, will help you learn how to choose between different options when loading data into Synapse and the optimal way to perform different data loadings.

Chapter 2, Creating Robust Data Pipelines and Data Transformation, will help you understand Synapse notebooks and its interfaces to create a file that will contain the real code – the logic. You will also learn how to visualize data within a notebook and other big data scenarios.

Chapter 3, Processing Data Optimally across Multiple Nodes, explores the Synapse SQL architecture components and how to leverage the scale-out capabilities to distribute the computational processing of data across multiple nodes.

Chapter 4, Engineering Real-time Analytics with Azure Synapse Link Using Cosmos DB, describes how you can architect and perform real-time analytics with Synapse, integrate Synapse Link for Cosmos DB, and enable the Internet of Things (IoT).

Chapter 5, Data Transformation and Processing with Synapse Notebooks, teaches you how to use Python to read data from Azure Data Lake Storage Gen2 into a Spark DataFrame using Azure Synapse Analytics.

Chapter 6, Enriching Data Using the Azure ML AutoML Regression Model, helps you uncover the power of Azure Machine Learning along with Spark MLlib and Synapse Studio.

Chapter 7, Visualizing and Reporting Petabytes of Data, teaches you how to present data using visualizations with Power BI, integrate Power BI with Synapse, and use the power of the serverless SQL pool for data exploration.

Chapter 8, Data Cataloging and Governance, teaches you how to provide comprehensive data governance for an analytical workload and embed data discovery and classification with Synapse using Azure Purview integration.

Chapter 9, MPP Platform Migration to Synapse, teaches you how to get started with the migration of a legacy data warehouse using Azure Synapse Pathway.

To get the most out of this book

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book's GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Azure-Synapse-Analytics-cookbook. If there's an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781803231501_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: This indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Define and load the entire DataFrame to pandas using the toPandas() function and define the chart type that you want to plot.

A block of code is set as follows:

mydataframeplot = mydataframe1.toPandas()

ax = mydataframeplot['passenger_count'].plot(kind='hist', bins= 20, facecolor='orange')

ax.set_title('Total Passenger distribution')

ax.set_xlabel('No. of Passengers')

ax.set_ylabel('Counts')

chartplt.suptitle('Trend')

chartplt.show()

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[default]

exten => s,1,Dial(Zap/1|30)

exten => s,2,Voicemail(u100)

exten => s,102,Voicemail(b100)

exten => i,1,Voicemail(s0)

Any command-line input or output is written as follows:

SELECT name, is_auto_create_stats_on

FROM sys.databases

Bold: This indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "Go to the existing Synapse Analytics workspace and navigate to Synapse Studio."

Tips or Important Notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you've read Azure Synapse Analytics Cookbook, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.