AWS Step Functions

• Used to orchestrate and design workflows

• Easy visualizations

• Advanced Error Handling and Retry mechanism outside the code

  • if you have an ETL for example, it's very complex and needs to orchestrate between 20 different services, you can note how every service reacted, what happened and get some audits around how things worked and if they didn't work; some debugging.

• Audit of the history of workflows

• Ability to "Wait" for an arbitrary amount of time

• Max execution time of a State Machine is 1 year

Here we use a step function to train a ML model:

This whole orchestration is done by a step function. Each actual step will invoke a different service, for example, lambda, SageMaker, S3 and so on but here, the step function is here to define the workflow about how things should work at a high level.

Examples Tune a Machine Learning Model:

Examples Manage a Batch Job

Going into the ML exam remember any time you need to orchestrate some things or make sure that one thing happens and then another thing happens and then another thing happens, then step functions are the perfect candidate for this.