Use the Serverless Cloud for ETL Scenarios
A real world scenario that leverages Azure Functions
The essence of serverless is less server. The details of the server are abstracted away and the developer is able to focus on code without getting bogged down in infrastructure concerns. Extract, Transform, and Load (ETL) is a common scenario in enterprise applications that serverless applications are uniquely positioned to address. Developers are able to handle file uploads, set timers and schedules, and run transformations using serverless without touching hardware, installing third-party services or writing custom polling software.
There is a better way! As much as I love to speak and write about serverless, I still believe the best way to learn what it is, and understand its value, is to go hands-on. To give you a full end-to-end scenario, I wrote a walk through and published it to GitHub.
This lab exposes you to the full spectrum of capabilities and workflows associated with serverless application development. You will:
- Create a serverless Azure function locally and debug it on your machine (no Internet required) using the cross-platform functions host
- Set up a trigger to automatically detect when a file is uploaded to file storage — no polling required (and you’ll also test this locally using the Azure Storage Emulator)
- Create a local SQL database that represents a common scenario: the legacy on-premises database
- Learn how to use Entity Framework in Azure Functions by parsing a CSV file and inserting new records into the database
- Create a cloud-hosted Azure SQL Database and migrate your local database to the cloud
- Deploy your local code to Azure
- Run the full process entirely from the cloud without configuring a single piece of hardware or consulting with a DBA
If the previous list includes scenarios you are interested in and would like to learn more about, carve out an hour on your schedule and head on over to the hands-on walk through!
- Deep Data Dive with Kusto for Azure Data Explorer and Log Analytics (Azure)
- Introducing Durable Entities for Serverless State (Azure Functions)
- Moving From Lambda ƛ to Azure Functions <⚡> (Azure Functions)
- Serverless HTTP With Durable Functions (Azure)
- The Year of Angular on .NET Core, WebAssembly, and Blazor: 2019 in Review (Azure)