Unsloth and Llama 3: How to Teach a Model to Call APIs Reliably via JSON

Machine Learning Mastery has released a guide to fine-tuning Llama 3 for custom tool calling with the Unsloth library in Python. The article states the core problem in its very first line: Llama 3 handles a broad range of tasks well, but that breadth is not enough for an agent that has to return structured JSON reliably for a specific API. One missing argument or one extra phrase before the opening curly brace, and the function call fails. Only the problem statement is available from the accessible excerpt, so below we walk through the standard approach to it rather than retell the article line by line.

Why a general-purpose model breaks function calls

When an agent works with an API, a formatting error is expensive. The schema validator rejects the entire response, and a retry costs money and time. An untuned model usually fails in the same ways: text before and after the JSON, invented fields, a string where a number belongs, a function name that does not exist. A prompt with examples reduces the share of such errors but does not eliminate them, especially when the schema has dozens of functions with nested parameters.

Fine-tuning attacks the problem differently. The model sees hundreds or thousands of "user request → correct call" pairs in your exact schema and stops guessing at the format. The format becomes its default way of answering.

What Unsloth offers

Unsloth is an open-source library for faster LLM fine-tuning. Its authors claim up to 2x faster training and up to 70% lower VRAM use compared with the standard stack. These are the project's own figures, not independent measurements, and they may differ on your hardware. The practical point lies elsewhere: paired with QLoRA, the library lets you keep the base weights in 4-bit format and train only small LoRA adapters. A model with 8 billion parameters fits on a single 16 GB graphics card in this mode, for example the T4 in free Colab.

A typical setup for this task:

• Base: Llama 3 8B Instruct
• Method: LoRA or QLoRA with 4-bit quantization of the base weights
• Data: "request → function call" pairs in JSON that strictly follows your API schema
• Stack: Python, Unsloth, Hugging Face Datasets, TRL (SFTTrainer)
• Hardware: a single GPU with 16 GB of memory or more
• Output: an adapter that weighs a small fraction of the full model; if needed, you can merge it into the base and export to GGUF for llama.cpp or Ollama

Where the pitfalls are

The main risk is the data. If the training examples are sloppy, the model will learn that sloppiness with the same confidence. The chat template used in training must match the one used at inference, including Llama 3's special tokens. A mismatch here is the most common reason for "it worked in the notebook but not in prod."

The second trap is the lack of negative examples. An agent must be able to not call a function when the question is unrelated to it, or to ask a follow-up when an argument is missing. Without such cases, the fine-tuned model will start calling tools for any reason.

The third problem is evaluation. Training loss says little about the quality of an agent. You need a held-out set and three metrics: the share of syntactically valid JSON, accuracy of function selection, and accuracy of argument values.

In practice

Start small. Collect 300–500 high-quality examples from real logs, label them according to the schema, and be sure to hold out 10–20% for validation. Run QLoRA on the 8B model and compare the result with base Llama 3 given the same function schemas in its prompt. If the gain in JSON validity and argument accuracy is noticeable, scale up the dataset. If not, check the chat template and the quality of the labeling.

Fine-tuning does not replace constrained decoding. Generation guided by a grammar or JSON schema guarantees syntax, while fine-tuning handles meaning: which function to choose and what to put in the arguments. Together, the two approaches give the most predictable results. Also, by 2026 Llama 3 is no longer new, but the recipe itself carries over to other models that Unsloth supports.