Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableAWS Machine Learning Blog→ original

Fine-Tuned Qwen3-14B from Exowatt Beats Claude Opus 4.7: 97.5% vs. 62.5%, and 49 Times Cheaper

Exowatt, which builds solar stations for data centers, fine-tuned the small Qwen3-14B model on its own data instead of calling an expensive API. On classifying energy-market events (such as a price spike), it scores 97.5% macro-F1, which averages quality across all event types, while Claude Opus 4.7 scores 62.5%. A thousand responses cost $0.60 against $29.26, which is 49 times cheaper. For a narrow, repetitive task, the small model beats the flagship on value.

AI-processed from AWS Machine Learning Blog; edited by Hamidun News
Fine-Tuned Qwen3-14B from Exowatt Beats Claude Opus 4.7: 97.5% vs. 62.5%, and 49 Times Cheaper
Source: AWS Machine Learning Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceAWS Machine Learning Blog ↗
Editor-in-Chief
Key metric #1The Qwen3-1.7B version (1.71 GiB) runs on a regular CPU without internet access
Key metric #2Training the 1.7B took 32 minutes and $20–25, while the 14B took about 8.5 hours
Key metric #3684 verified solutions were turned into 2,522 examples through paraphrasing

Exowatt builds modular solar stations with thermal storage and powers data centers directly, bypassing years-long queues for grid connection. Alarms go off around the clock at each site, and operators must also decide how much stored energy to keep in reserve and when to release it. You cannot staff an operator at every facility, and sending every decision to the API of a flagship such as Claude Opus 4.

7 is too expensive. So the company fine-tuned, meaning it further trained ready-made open models on its own examples, two small Qwen3 models. The 14-billion-parameter version identifies energy-market events with a macro-F1 score of 97.

5%, against 62.5% for Opus 4.7.

This metric averages quality across all event types, so rare cases carry the same weight as common ones. A thousand responses cost $0.60 instead of $29.

26, roughly 49 times cheaper. Details were published on the AWS Machine Learning blog.

Two Tasks, Two Models

The first task is triaging alarms at the site. The model receives an alert along with telemetry and returns one of a small set of diagnoses (system offline, unit running at reduced power, sensor mismatch), citing the supporting data. If the evidence is thin, it honestly declines to answer. This has to run on site, often without a GPU and without internet access, so the team chose Qwen3-1.7B. After quantization to 8 bits (weight compression), the model takes up about 1.71 GiB and runs through llama.cpp in a Kubernetes pod on a regular CPU server at the facility.

The second task is energy-market analysis. The Qwen3-14B model reads a six-hour data package (about 90 rows of real-time prices, day-ahead forecasts, and grid notifications) and, in four seconds, identifies one of five events, such as a price spike. Every claim is tied to a specific row of data. The model is hosted on Amazon Bedrock via Custom Model Import: there is no idle time, and $6.86 per hour is charged only while the model is active.

Numbers: Accuracy and Cost

On alarm triage, the fine-tuned Qwen3-1.7B reached 91.1% macro-F1. Claude Sonnet 4.6 scored 69.8%, Opus 4.7 scored 68%, and Haiku 4.5 scored 51.9%. The original 1.7B did not pass the check for valid answers at all. External models cost $4.70–22 per 1,000 decisions, while on Exowatt's own CPUs there is no per-request charge.

On the energy market, Qwen3-14B scored 97.5% against 62.5% for Opus 4.7 and 53.8% for Sonnet 4.6. The base Qwen3-14B without fine-tuning scored 41.5%. It produced correct JSON but did not know the market. A thousand responses from the fine-tuned model cost $0.60, versus $29.26 for Opus. On a second, later test set, the model held at 96.8%.

What Worked in Practice

The most instructive point is the data. For alarm triage, the team collected just 684 verified decisions on generated scenarios, and then a larger model paraphrased the text without changing the decision, the labels, or the numbers. Of 2,218 variants, 1,838 passed verification, yielding 2,522 examples. The paraphrases cost about $3.72. The result: macro-F1 rose from 58.4% on the original 684 rows to 91.1%. A control run on the same 684 rows over 11 epochs reached only 87.9%, so variety of phrasing helped more than repeating the same examples.

For the market, the team used five years of public ERCOT (the Texas grid operator) data, about 163 million rows. Rules labeled 25,000 of them, and the training portion was expanded to 86,281 rows with paraphrases from Claude Sonnet 4.6. Three paraphrases per response raised macro-F1 from 95.0% to 97.5%. Training ran for one LoRA epoch (rank 16, alpha 32) and took about 8.5 hours. Over six days, the team launched 23 jobs (about 155 hours of training), running up to four at once. SageMaker's serverless mode allocates capacity on its own, so there was no need to choose a GPU type or wait for one to free up.

The second finding concerns protection against fabrication. The models return strict JSON, and any claim whose source reference, field, or value does not match the input data is discarded before it reaches the operator. When data is insufficient, the edge model abstains. The system has been running in shadow mode against a cloud copy of the site.

Limitations and Conclusion

This is an AWS blog about an AWS service, so it should be read with that in mind. The comparison runs on Exowatt's own test sets, some of which were built from generated scenarios. The flagship models were evaluated with prompts, without task-specific fine-tuning. Bedrock prices come from the published price list, and CPU work is treated as free, without accounting for hardware or support. A narrow model winning does not mean it is stronger overall: outside its task, it will lose.

For practitioners, the takeaway is this. If the task recurs constantly, the output format is fixed, and you have at least a few hundred verified examples, check a fine-tuned small model first. The most valuable part of the story is the pipeline: raw data, a verified set, training, evaluation, and packaging of the model. Exowatt now goes through this path in days rather than months, and plans to repeat it for new device types.

Story

Key Questions Answered

What are the key details on Two Tasks, Two Models?

The first task is triaging alarms at the site. The model receives an alert along with telemetry and returns one of a small set of diagnoses (system offline, unit running at reduced power, sensor mismatch), citing the supporting data. If the evidence is thin, it honestly declines to answer. This has to run on site, often without a GPU and without internet access, so the team chose Qwen3-1.7B.

What are the key details on Numbers: Accuracy and Cost?

On alarm triage, the fine-tuned Qwen3-1.7B reached 91.1% macro-F1. Claude Sonnet 4.6 scored 69.8%, Opus 4.7 scored 68%, and Haiku 4.5 scored 51.9%. The original 1.7B did not pass the check for valid answers at all. External models cost $4.70–22 per 1,000 decisions, while on Exowatt's own CPUs there is no per-request charge. On the energy market, Qwen3-14B scored 97.5% against 62.5% for Opus 4.7 and 53.8% for Sonnet 4.6. The base Qwen3-14B without fine-tuning scored 41.5%.

What Worked in Practice?

The most instructive point is the data. For alarm triage, the team collected just 684 verified decisions on generated scenarios, and then a larger model paraphrased the text without changing the decision, the labels, or the numbers. Of 2,218 variants, 1,838 passed verification, yielding 2,522 examples. The paraphrases cost about $3.72. The result: macro-F1 rose from 58.4% on the original 684 rows to 91.1%.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…