Azure Durable Functions (2019-04-27)
2 likes788 views
This document provides an overview of Durable Functions, which allow for implementing stateful workflows in Azure Functions. Durable Functions enable writing stateful workflows as code by using orchestrator functions to coordinate asynchronous activity functions. Key components include orchestrator functions that call and coordinate stateless activity functions, and an orchestration client to start and manage orchestrations. Patterns demonstrated include function chaining, fan-out/fan-in, and using an orchestration client to start and monitor orchestrations.
![Function Chaining Pattern
public static async Task<object> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallActivityAsync<object>("F1");
var y = await ctx.CallActivityAsync<object>("F2", x);
return await ctx.CallActivityAsync<object>("F3", y);
}
catch (Exception ex)
{
// error handling / fall back / compensation
}
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-24-320.jpg)
![Function Chaining Pattern
public static async Task<object> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallActivityAsync<object>("F1");
var y = await ctx.CallActivityAsync<object>("F2", x);
return await ctx.CallActivityAsync<object>("F3", y);
}
catch (Exception ex)
{
// error handling / fall back / compensation
}
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-25-320.jpg)
![Function Chaining Pattern
public static async Task<object> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallActivityAsync<object>("F1");
var y = await ctx.CallActivityAsync<object>("F2", x);
return await ctx.CallActivityAsync<object>("F3", y);
}
catch (Exception ex)
{
// error handling / fall back / compensation
}
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-26-320.jpg)
![Fan-out & Fan-In Pattern
public static async Task<int> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
object[] workBatch = await ctx.CallActivityAsync<object[]>("F1");
var tasks = new Task<long>[workBatch.Length];
for (int i = 0; i < workBatch.Length; i++)
{
tasks[i] = ctx.CallActivityAsync<int>("F2", workBatch[i]);
}
await Task.WhenAll(tasks);
long sum = tasks.Sum(t => t.Result);
return sum;
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-27-320.jpg)
![Fan-out & Fan-In Pattern
public static async Task<int> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
object[] workBatch = await ctx.CallActivityAsync<object[]>("F1");
var tasks = new Task<long>[workBatch.Length];
for (int i = 0; i < workBatch.Length; i++)
{
tasks[i] = ctx.CallActivityAsync<int>("F2", workBatch[i]);
}
await Task.WhenAll(tasks);
long sum = tasks.Sum(t => t.Result);
return sum;
}
CallActivityWithRetryAsync](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-28-320.jpg)
![Orchestration Client
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync("myOrchestrator", eventData);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var res = starter.CreateCheckStatusResponse(req, instanceId);
return res;
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-29-320.jpg)
![Orchestration Client
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync("myOrchestrator", eventData);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var res = starter.CreateCheckStatusResponse(req, instanceId);
return res;
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-30-320.jpg)
![Orchestration Client
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync("myOrchestrator", eventData);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var res = starter.CreateCheckStatusResponse(req, instanceId);
return res;
}](https://image.slidesharecdn.com/durablefunctionsvpublic-190426120801/85/Azure-Durable-Functions-2019-04-27-31-320.jpg)
Azure Durable Functions (2019-04-27)
- 1. Paco de la Cruz Durable Functions Serverless and Stateful Orchestrations on Azure @pacodelacruz Global Azure Bootcamp 2019, Melbourne
- 7. “Serverless” and its benefits Event-driven scaling not resource-driven Pay only for what you use Server abstraction Focus on value
- 8. Azure Functions in a nutshell Event Triggers Code Outputs React and get inputs from a growing list of services (triggers and input bindings) C#, F#, Node.js, Java, Python (Preview) Send results to a growing list of services (output bindings)
- 9. Some limitations/challenges of Azure Functions Manageable Sequencing + Error Handling / Compensation
- 10. Manageable Sequencing + Error Handling / Compensation Some limitations/challenges of Azure Functions Fanning-out & Fanning-in
- 11. Some limitations/challenges of Azure Functions Manageable Sequencing + Error Handling / Compensation Fanning-out & Fanning-in Async Long-Running Http APIs (Status polling) Start Get Status
- 12. Some limitations/challenges of Azure Functions Manageable Sequencing + Error Handling / Compensation Fanning-out & Fanning-in Async Long-Running Http APIs (Status polling) Start Get Status Long-Running Stateful Repeating Process
- 13. Some limitations/challenges of Azure Functions Manageable Sequencing + Error Handling / Compensation Fanning-out & Fanning-in Long-Running Stateful Repeating Process Async Long-Running Http APIs (Status polling) Start Get Status Human Interaction
- 14. Some limitations/challenges of Azure Functions Manageable Sequencing + Error Handling / Compensation Fanning-out & Fanning-in Human InteractionLong-Running Stateful Repeating Process Async Long-Running Http APIs (Status polling) Start Get Status External Events Correlation
- 15. Some limitations/challenges of Azure Functions Manageable Sequencing + Error Handling / Compensation Fanning-out & Fanning-in Human InteractionLong-Running Stateful Repeating Process Async Long-Running Http APIs (Status polling) Start Get Status External Events Correlation
- 16. Durable Function Patterns Function Chaining Fanning-out & Fanning-in Async HTTP APIs Human InteractionMonitoring External Events Correlation Start Get Status
- 17. Durable Functions in a nutshell Based on Durable Task Framework Persistence on Azure Storage (Fully Managed and Abstracted) To Implement stateful workflows-as-code (C#, F#, Java and Node.js) Azure Functions Extension
- 18. Durable Functions Components Activity Function Activity Function Activity Function Orchestrator Function Orchestration Client
- 19. Durable Functions Components Activity Function Activity Function Activity Function Orchestrator Function Orchestration Client Stateful Process Manager Call Activity Functions Advanced Retries Error Handling Fall Back / Compensation Checkpointing Dehydrates during activities Rehydrates at responses / events
- 20. Durable Functions Components Activity Function Activity Function Activity Function Orchestrator Function Orchestration Client Stateless Single Step Inputs and Outputs Stateful Process Manager Call Activity Functions Advanced Retries Error Handling Fall Back / Compensation Checkpointing Dehydrates during activities Rehydrates at responses / events
- 21. Durable Functions Components Activity Function Activity Function Activity Function Orchestrator Function Orchestration Client Start Get Status Send Event Wait for Completion Terminate Stateless Single Step Inputs and Outputs Stateful Process Manager Call Activity Functions Advanced Retries Error Handling Fall Back / Compensation Checkpointing Dehydrates during activities Rehydrates at responses / events
- 22. Orchestration Function Limitations Orchestration code must • Be deterministic (e.g. no NewGuid(), Random, DateTime.Now(), Http calls, etc.) • Be non-blocking: (no I/O, Thread.Sleep(), etc.) • Never initiate any async operations: without using its context • Avoid infinite loops There are workarounds
- 23. Orchestration Considerations Activity function calls and responses are sent via queues Async calls persist the orchestration state into a storage table Activity function outputs are persisted (Event Sourcing) Activity function responses, timer or external events will recover the persisted orchestration state into memory The orchestration is replayed up-to the last persisted point Activity Functions are not replayed
- 24. Function Chaining Pattern public static async Task<object> Run( [OrchestrationTrigger] DurableOrchestrationContext ctx) { try { var x = await ctx.CallActivityAsync<object>("F1"); var y = await ctx.CallActivityAsync<object>("F2", x); return await ctx.CallActivityAsync<object>("F3", y); } catch (Exception ex) { // error handling / fall back / compensation } }
- 25. Function Chaining Pattern public static async Task<object> Run( [OrchestrationTrigger] DurableOrchestrationContext ctx) { try { var x = await ctx.CallActivityAsync<object>("F1"); var y = await ctx.CallActivityAsync<object>("F2", x); return await ctx.CallActivityAsync<object>("F3", y); } catch (Exception ex) { // error handling / fall back / compensation } }
- 26. Function Chaining Pattern public static async Task<object> Run( [OrchestrationTrigger] DurableOrchestrationContext ctx) { try { var x = await ctx.CallActivityAsync<object>("F1"); var y = await ctx.CallActivityAsync<object>("F2", x); return await ctx.CallActivityAsync<object>("F3", y); } catch (Exception ex) { // error handling / fall back / compensation } }
- 27. Fan-out & Fan-In Pattern public static async Task<int> Run( [OrchestrationTrigger] DurableOrchestrationContext ctx) { object[] workBatch = await ctx.CallActivityAsync<object[]>("F1"); var tasks = new Task<long>[workBatch.Length]; for (int i = 0; i < workBatch.Length; i++) { tasks[i] = ctx.CallActivityAsync<int>("F2", workBatch[i]); } await Task.WhenAll(tasks); long sum = tasks.Sum(t => t.Result); return sum; }
- 28. Fan-out & Fan-In Pattern public static async Task<int> Run( [OrchestrationTrigger] DurableOrchestrationContext ctx) { object[] workBatch = await ctx.CallActivityAsync<object[]>("F1"); var tasks = new Task<long>[workBatch.Length]; for (int i = 0; i < workBatch.Length; i++) { tasks[i] = ctx.CallActivityAsync<int>("F2", workBatch[i]); } await Task.WhenAll(tasks); long sum = tasks.Sum(t => t.Result); return sum; } CallActivityWithRetryAsync
- 29. Orchestration Client public static async Task<HttpResponseMessage> Run( [HttpTrigger(AuthorizationLevel.Function, methods: "post", Route = "orchestrators/{functionName}")] HttpRequestMessage req, [OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log) { // Function input comes from the request content. dynamic eventData = await req.Content.ReadAsAsync<object>(); string instanceId = await starter.StartNewAsync("myOrchestrator", eventData); log.LogInformation($"Started orchestration with ID = '{instanceId}'."); var res = starter.CreateCheckStatusResponse(req, instanceId); return res; }
- 30. Orchestration Client public static async Task<HttpResponseMessage> Run( [HttpTrigger(AuthorizationLevel.Function, methods: "post", Route = "orchestrators/{functionName}")] HttpRequestMessage req, [OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log) { // Function input comes from the request content. dynamic eventData = await req.Content.ReadAsAsync<object>(); string instanceId = await starter.StartNewAsync("myOrchestrator", eventData); log.LogInformation($"Started orchestration with ID = '{instanceId}'."); var res = starter.CreateCheckStatusResponse(req, instanceId); return res; }
- 31. Orchestration Client public static async Task<HttpResponseMessage> Run( [HttpTrigger(AuthorizationLevel.Function, methods: "post", Route = "orchestrators/{functionName}")] HttpRequestMessage req, [OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log) { // Function input comes from the request content. dynamic eventData = await req.Content.ReadAsAsync<object>(); string instanceId = await starter.StartNewAsync("myOrchestrator", eventData); log.LogInformation($"Started orchestration with ID = '{instanceId}'."); var res = starter.CreateCheckStatusResponse(req, instanceId); return res; }
- 32. Durable Functions vs Logic Apps? vs Durable Functions Logic Apps Both allow implementing advanced workflow patterns C#, F#, Java, JavaScript Visual designer and WDL Bindings (~ 20 supported) 250+ connectors Portable Runtime Runs only on Azure Monitoring based on App Insights & APIs Rich monitoring & management tools Serverless + dedicated & isolated Serverless + dedicated & isolated (ISE) platform.deloitte.com.au/articles/azure-durable-functions-vs-logic-apps
- 33. Additional Resources Twitter @azurefunctions Documentation aka.ms/durablefunctions Live Web Cast aka.ms/azurefunctionslive Repos github.com/Azure/azure-functions-durable-extension github.com/Azure/azure-functions-durable-js Samples github.com/Azure/azure-functions-durable- extension/tree/master/samples Pluralsight course app.pluralsight.com/library/courses/ azure-durable-functions-fundamentals
- 34. Any questions so far?
- 36. Demo 1 Development and Debugging Furry Models Cat Application Approval via email
- 37. Orchestration Function Start Send Approval Request via Email Create Timer External Event Timer Expires WhenAny Move Blob to Corresponding Container End (Requests blob container) (Approved or Rejected blob container)
- 38. Orchestration Function Start Send Approval Request via Email Create Timer External Event Timer Expires WhenAny Move Blob to Corresponding Container End BlobTrigger Function HttpTrigger Process Approval (Requests blob container) (Approved or Rejected blob container) Orchestration Client Activity Functions ActivityTrigger Send Approval Request via Email ActivityTrigger Move Blob to Corresponding Container HttpTrigger Check Status
- 39. Demo 2 Execution & Monitoring on Azure Furry Models Cat Application Approval via Slack
- 40. Orchestration Function Start Send Approval Request via Slack Create Timer External Event Timer Expires WhenAny Move Blob to Corresponding Container End BlobTrigger Function HttpTrigger Process Approval (Requests blob container) (Approved or Rejected blob container) Orchestration Client Activity Functions ActivityTrigger Send Approval Request via Slack ActivityTrigger Move Blob to Corresponding Container HttpTrigger Check Status
- 41. Let your Cat apply! Great opportunity to get famous! Send to: …@...com Subject: I want to be a Furry Model! Attach your cat best picture
- 42. Details about the demos: platform.deloitte.com.au/articles/azure-durable-functions-approval-workflow-with-sendgrid platform.deloitte.com.au/articles/azure-durable-functions-approval-workflow-with-slack github.com/pacodelacruz/durable-functions-furry-models
- 43. Q & A
- 44. Thanks!