AWS shows how to cut text-to-SQL costs with Amazon Nova Micro and Bedrock

AWS has shown how to get text-to-SQL for internal databases without paying for expensive, always-on model hosting. The company fine-tuned Amazon Nova Micro for non-standard SQL dialects and deployed it through Amazon Bedrock, paying only for actual requests.

Why it matters

For enterprise use, an off-the-shelf model is often not enough: it writes standard SQL reasonably well but starts making mistakes when a company has its own conventions, rare functions, unusual table schemas and domain rules. That is why a user's text query has to be adapted to a specific dialect and database structure, which means fine-tuning the model on your own examples. This is especially noticeable in BI systems and internal analytics chats, where a syntax error immediately breaks the whole workflow.

The problem is that fine-tuning usually brings ongoing infrastructure costs with it. If a custom model runs on dedicated servers, the company pays even when there are no requests. In its write-up, AWS proposes another option: fine-tune Amazon Nova Micro with LoRA and run it in Amazon Bedrock in on-demand inference mode, where billing is per token rather than for permanently reserved capacity.

AWS's two approaches

AWS describes two workflows for the same task.

The first is managed fine-tuning inside Amazon Bedrock. It suits teams that value simplicity, a fast start and minimal fuss with ML infrastructure. Data is uploaded to S3, the fine-tuning job is launched from the console or the API, and AWS then manages the training and the subsequent deployment of the custom Nova Micro version. This option is aimed more at an application team than at a dedicated ML platform team.

The second path is training through Amazon SageMaker AI. It is more complex but gives more control over the training recipe: you can change the batch size, dropout, optimizer parameters, context window, LoRA settings and the learning-rate warmup strategy. In the example, AWS used the sql-create-context dataset, built from WikiSQL and Spider, and converted the question-SQL pairs into the bedrock-conversation-2024 format for training and validation. The price is greater configuration complexity and more explicit work with infrastructure.

Here is how the options differ in practice:

• Bedrock: less operational load and a faster path to a working prototype
• SageMaker AI: more control over hyperparameters and integration into MLOps
• Both approaches use the same data-preparation pipeline and are then deployed in Bedrock
• Final inference runs on a serverless model billed per token, with no permanent hosting

Price and latency

AWS gives concrete figures. For managed fine-tuning in Bedrock, training cost is calculated at $0.001 per 1,000 tokens per epoch: in the example with 2,000 samples, five epochs and roughly 800 tokens per record, that came to about $8. The SageMaker variant used an ml.g5.48xlarge instance at $16.288 per hour; training on a 20,000-row dataset took about four hours and cost roughly $65.15.

The article's key point is not the one-off training price but the cost of running the model. AWS estimated a sample production load of 22,000 requests a month, or 100 users making 10 requests a day over 22 working days. With an average request of 800 input and 60 output tokens, monthly inference for such a custom text-to-SQL model came to $0.80. This is possible because a custom Nova Micro in Bedrock is billed the same as the base model, with no separate markup for serverless deployment.

There is a speed trade-off, but a moderate one. On a cold start, the average time to first token rose to 639 ms, 34% higher than the base model. In normal operation, the average TTFT was 380 ms over 50 calls, only 7% worse than the base. Total generation latency was about 477 ms, and output speed held at 183 tokens per second. AWS checked quality not only through latency but also with LLM-as-a-Judge, comparing generated SQL against reference answers.

What it means

For teams that want to build text-to-SQL into analytics products, internal BI tools or chat interfaces to a database, the AWS case looks practical: you can get a custom SQL generator without the ongoing cost of dedicated infrastructure. If launch speed matters more, Bedrock is the logical choice; if you need full control over training, the SageMaker AI combination looks stronger.