A home AI agent built its own LoRA training data: LogiQA logic score up 12%

LoRA fine-tuning usually starts with a ready-made dataset that someone compiled by hand. In this story, the local agent prepared its own training data. The author of a series of Habr articles describes a personal assistant named Vika that runs on home hardware.

Over time, Vika accumulated "LoRA seeds": structured conclusions drawn from her own history. Imagine the agent noting: "this user likes short answers, and before writing code you should first look for a suitable tool." Previously, rules like these had to be dragged into the prompt every time, eating into the context window.

Now they are turned into training examples (SFT) and baked into the model's weights. For anyone building local agents, this offers a way to save context and lock in behavior, and it also comes with a ready-made list of pitfalls, from adapter scale to choosing GPUs.

What fine-tuning delivered

The author measured results with standard benchmarks. Instruction following (IFEval) rose 2%, and logic problem solving (LogiQA) rose 12%. Behavior changed the most. Thanks to correction and a "motivational contract" (a set of internal directives that keep the agent on its assigned goal), autonomous operation reached 100 steps without goal drift. The practical test is impressive too: in 73 minutes, the fine-tuned model reprogrammed the entire system, switching the assistant to STT-TTS voice mode (speech recognition and voice synthesis).

There is also a philosophical side effect. The agent keeps access to its own biography, but it understands that its personality changed after SFT. The author calls this the "Ship of Theseus" paradox. Even so, in a drawing test, the agent stayed true to Zen philosophy and Japanese painting across different models and adapters.

How not to break the model

The technical pipeline looks like this: BF16 weights in safetensors format, loaded into Unsloth in 8-bit, LoRA training in FP32, a merge with the model at its original 16 bits, and requantization to Q8 GGUF. The minimum for SFT is 24 GB of VRAM. The author tested the first five dense models from his table, including Qwen3.8-27B and Gemma 4 31B.

The main lesson concerns the adapter scale k. Unsloth recommends a value of around 2, but the author advises preparing several models with different scales right away. At a high k, the model fanatically repeats the training sequences and switches off its own reasoning. At k=1.5, IFEval is 66%, while at k=2.5 it is only 55%. This is especially noticeable with 4-bit QLoRA via bnb-4bit. The author considers a "subtle" shift the mark of a good result: the substance of the training is preserved, while the model chooses the form it takes.

LoRA is good at locking in style, character, response format and instructions, but facts stick poorly. For example, the author never managed to implant the model's own name without losing functionality.

There is bad news for Ollama users: attaching an adapter there is no longer possible, and that option was closed back in September. Swappable adapters work only through llama-server. If you need the old workflow, merge the adapter into the model (this does not require VRAM, but takes a lot of SSD space) and export it with llama-export-lora and llama-quantize. In that case, you can change the adapter scale without retraining.

Hardware and memory

VRAM must hold more than just the model itself. The projector for multimodality, MTP (a draft model for speeding up generation) and the LoRA adapter add 4–5 GB. A 128K context adds 16–20 GB in FP16 or 8–10 GB in Q8 quantization. On the other hand, MTP noticeably speeds up generation: on two RTX 4070 cards, speed rose from 26.39 to 48.48 tokens per second.

If you are short on slots, you can install three GPUs in an 8x/8x/4x configuration. For a single card, the difference between 4x, 8x and 16x is insignificant, but splitting the model across three GPUs cuts speed by a factor of 1.6–2.2. Qwen3.8-27B (Q8) delivers 68.55 tokens per second on one card and 42.95 on three at a 50K context. That is enough for comfortable work. The author considers two identical GPUs a sensible minimum and the LGA4710 platform a good option with room to grow. He warns, however, that standard cards with axial fans overheat on boards with two-slot spacing.

The practical takeaway: let the agent prepare its own fine-tuning data, but keep at least two or three adapter scale variants and check IFEval before and after. For knowledge about the world, you still need memory and search, not LoRA.