Course Description: DP-3011: Implementing a Data Analytics Solution with Azure Databricks

Learn how to leverage the full benefits of Apache Spark and powerful clusters running on the Azure Databricks platform to run large data engineering workloads in the cloud.

• Level: Intermediate
• Product: Azure Databricks
• Role: Data Engineer
  

Goals

• Set up a development environment in Azure Machine Learning
• Preparing data for model training
• Create and configure a model training script as a command job
• Managing artifacts using MLflow
• Implement a model for real-time consumption

Training Route

• Explore Azure Databricks - Azure Databricks is a cloud service that provides a scalable platform for data analytics using Apache Spark.
• Perform data analysis with Azure Databricks: Learn how to perform data analysis with Azure Databricks. Explore various data ingestion methods and how to integrate data from sources such as Azure Data Lake and Azure SQL Database. This module guides you through using collaborative notebooks to perform exploratory data analysis (EDA), so you can visualize, manipulate, and examine data to discover patterns, anomalies, and correlations.
• Using Apache Spark on Azure Databricks: Azure Databricks is built on Apache Spark and enables data engineers and analysts to run Spark jobs to transform, analyze, and visualize data at scale.
• Using Delta Lake in Azure Databricks: Delta Lake is an open-source relational storage area for Spark that you can use to implement a data lake architecture in Azure Databricks.
• Building data pipelines with Delta Live Tables: Building data pipelines with Delta Live Tables enables real-time, scalable, and reliable data processing using the advanced features of Delta Lake in Azure Databricks
• Deploying workloads with Azure Databricks Workflows: Deploying workloads with Azure Databricks Workflows involves orchestrating and automating complex data processing pipelines, machine learning workflows, and analytics tasks. In this module, you will learn how to deploy workloads with Databricks Workflows.
• Using SQL stores in Azure Databricks: Azure Databricks provides SQL stores that enable analysts to work with data through familiar relational SQL queries.
• Running Azure Databricks notebooks with Azure Data Factory: Using pipelines in Azure Data Factory to run notebooks in Azure Databricks enables you to automate data engineering processes at cloud scale.

Prerequisites

No prerequisites required

Language

• Course: English / Spanish
• Labs: English / Spanish