Scheduling-optimisation pipeline

An ML pipeline for intelligent scheduling, on MLflow and Spark.

As Data Team Lead, an end-to-end ML pipeline for intelligent scheduling optimisation, built on MLflow and Spark — tracked, reproducible and retrainable.

Key features

• ML pipeline for intelligent scheduling
• Tracked and reproducible with MLflow

Built with — how & why

• 
  Spark

  Distributed processing over the scheduling data.
• 
  Databricks

  Managed Spark + MLflow for the training loop.
• 
  Python

  Pipeline and modelling code.