Databricks and AWS SageMaker: a pipeline for secure LLM fine-tuning

AWS and Databricks demonstrated how to build an LLM fine-tuning pipeline that simultaneously addresses two challenges: maintaining control over data and models through a centralized catalog while preserving functionality and development speed.

Architecture workflow

The solution integrates three components. Databricks Unity Catalog handles governance and access — a single table defines who can access which data. Amazon EMR Serverless prepares data for training, while SageMaker AI executes the model fine-tuning itself. After training, artifacts (model weights, metrics) are registered back in Unity Catalog. This approach allows teams of data engineers, ML engineers, and data scientists to work in a unified space without needing to copy data between services or configure separate access layers for each tool.

Key stages

• Governance at the input: Unity Catalog defines access policies for source data — which tables are visible, which fields are masked
• Preprocessing: EMR Serverless transforms raw data into a format suitable for LLM training
• Fine-tuning: SageMaker AI fine-tunes Ministral-3-3B-Instruct (Mistral's model) using the prepared data
• Lineage tracking: The entire chain from source tables to the final model remains traceable — for auditing and compliance
• Artifact registration: The trained model and metrics are returned to Unity Catalog as managed assets

Why this is needed now

Many organizations face one of two scenarios. Either data and models are scattered across different services with no visibility — who uses what, where data comes from, who changed it. Or companies try to impose order through custom monitoring and access systems, but this requires months of development and maintenance.

"Instead of building governance from scratch, we give you a ready-made

integration where all components already speak the same language"

The AWS and Databricks solution eliminates this choice. Governance and lineage are built into the architecture from the start, not added on top.

What this means

For large organizations and financial institutions, it means LLM fine-tuning can now be deployed without risking loss of control over data. For engineering teams — there's no need to write custom tracking systems. The integration closes the gap between security requirements and ML development speed.