Table of Contents
Preface
Chapter 1: Choosing the Optimal Method for Loading Data to Synapse
Choosing a data loading option
Getting ready
How to do it…
How it works…
There's more…
Achieving parallelism in data loading using PolyBase
Moving and transforming using a data flow
Getting ready
How to do it…
How it works…
Adding a trigger to a data flow pipeline
Getting ready
How to do it…
How it works…
Unsupported data loading scenarios
How to do it…
There's more…
Data loading best practices
How to do it…
Chapter 2: Creating Robust Data Pipelines and Data Transformation
Reading and writing data from ADLS Gen2 using PySpark
Getting ready
How to do it…
How it works…
Visualizing data in a Synapse notebook
Getting ready
How to do it…
How it works…
Chapter 3: Processing Data Optimally across Multiple Nodes
Working with the resource consumption model of Synapse SQL
Architecture components of Synapse SQL
Resource consumption
Optimizing analytics with dedicated SQL pool and working on data distribution
Understanding columnstore storage details
Knowing when to use round-robin, hash-distributed, and replicated distributions
Knowing when to partition a table
Checking for skewed data and space usage
Best practices
Workload management for dedicated SQL pool
Working with serverless SQL pool
Getting ready
How to do it…
There's more…
Processing and querying very large datasets
Getting ready
How to do it…
Script for statistics in Synapse SQL
How to do it…
There's more…
Chapter 4: Engineering Real-Time Analytics with Azure Synapse Link Using Cosmos DB
Integrating an Azure Synapse ETL pipeline with Cosmos DB
Introducing Cosmos DB
Azure Synapse Link integration
Supported features of Azure Synapse Link
Azure Synapse runtime support
Structured streaming support
Network and data security support for Azure Synapse Link with Cosmos DB
Setting up Azure Cosmos DB analytical store
Getting ready
How to do it…
Enabling Azure Synapse Link and connecting Azure Cosmos DB to Azure Synapse
Getting ready
How to do it…
IoT end-to-end solutions and getting real-time insights
Getting ready
How to do it…
Use cases using Synapse Link
Chapter 5: Data Transformation and Processing with Synapse Notebooks
Landing data in ADLS Gen2
Getting ready
How to do it…
Exploring data with ADLS Gen2 to pandas DataFrame in Synapse notebook
Getting ready
How to do it…
There's more…
Processing data from a PySpark notebook within Synapse
How to do it…
Performing read-write operations to a Parquet file using Spark in Synapse
Getting ready
How to do it…
Analytics with Spark
Getting ready
How it works…
Chapter 6: Enriching Data Using the Azure ML AutoML Regression Model
Training a model using AutoML in Synapse
Getting ready
How to do it…
How it works…
Building a regression model from Azure Machine Learning in Synapse Studio
Getting ready
How to do it…
How it works…
Modeling and scoring using SQL pools
Getting ready
How to do it…
How it works…
An overview of Spark MLlib and Azure Synapse
Integrating AI and Cognitive Services
Getting ready
How to do it…
How it works…
Chapter 7: Visualizing and Reporting Petabytes of Data
Combining Power BI and aserverless SQL pool
Getting ready
How to do it…
How it works…
Working on a composite model
Getting ready
How to do it…
How it works…
Using materialized views to improve performance
Getting ready
How to do it…
How it works…
Chapter 8: Data Cataloging and Governance
Configuring your Azure Purview account for Synapse SQL pool
Getting ready
How to do it…
How it works…
Scanning data using the Purview data catalog
Getting ready
How to do it…
How it works…
Enumerating resources within Synapse Studio
Getting ready
How to do it…
How it works…
Chapter 9: MPP Platform Migration to Synapse
Understanding data migration challenges
Tables and databases
Data modeling
Data Manipulation Language statements
Functions, stored procedures, sequences, and triggers
Configuring Azure Synapse Pathway
Getting ready
How to do it…
How it works…
Evaluating a data source to be migrated
Getting ready
How to do it…
Generating a data migration assessment
Getting ready
How to do it…
Supported data sources for migration
IBM Netezza and Azure Synapse platform differences
Oracle Exadata and Azure Synapse platform differences
Snowflake and Azure Synapse platform differences
Microsoft SQL Server and Azure Synapse platform differences
Other Books You May Enjoy
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.