Habr walkthrough shows how to train a mini-LLM in C# using ILGPU and integrated AMD graphics

A detailed hands-on walkthrough on Habr shows a scenario that is rare in the local AI community: a small language model can be built and trained from scratch in C#, without Python and without requiring an NVIDIA card, then saved as GGUF and run in LM Studio. The author used the ILGPU library and OpenCL, relying on AMD integrated graphics, and ended up with a working but very limited LLM prototype of about 422 KB. The project is meant as an educational demonstration of the full cycle, not an attempt to compete with large open-source models.

The article covers the basic transformer architecture: tokenization, the embedding layer, self-attention, feed-forward, normalization and the output projection. For training, the author prepared two small datasets: a short pretrain corpus of general facts about the world and an instruction-tuning set of questions and answers. The goal was not to get a strong model but to go through every stage by hand, from vocabulary to inference.

The final configuration is extremely compact: a vocabulary of 512 tokens, context length 64, embedding size 64, hidden layer 128, one transformer layer and two attention heads. In total that is 103,744 parameters. Before export, the model can be saved as a binary file of roughly 6.87 MB, but the final GGUF file takes up about 422 KB and is recognized as GGUF V3. After training, pretrain loss fell to about 0.212, and after fine-tuning for the dialogue format, finetune loss fell to 0.3926. At this scale, that is more a sign that the pipeline works technically than proof of answer quality.

The training logic matters too. First the model goes through a pretraining phase on a short corpus of a few dozen sentences, then a separate instruction-tuning stage on dialogue examples. The program's interface has menus for retraining, testing the tokenizer, saving weights and launching an interactive chat. For answers, the author added a primitive confidence filter: if the minimum per-token confidence drops below a threshold, the system prints the phrase "I find it difficult to answer" instead of a reply. It is a crude but understandable way to hide the obvious garbage that a model this size almost inevitably produces. That is what makes the demo usable, at least for a basic pipeline check.

The hardest part turned out to be neither the forward pass nor the training itself, but compatibility with the llama.cpp ecosystem. The author first tried to build the model in a more LLaMA-like configuration but ran into loading errors and a mismatch in the number of tensors. In the end, the architecture had to be simplified, the tokenizer rebuilt along the lines of Microsoft.ML.Tokenizers, and the GGUF metadata described carefully: architecture, context length, attention parameters, special tokens and chat template. One more technical wrinkle: the model effectively cannot hold a long dialogue history, so the chat uses a template with only the user's latest message; otherwise the tiny context quickly breaks inference.

The practical result is modest but instructive. The model answers simple queries and opens in LM Studio, where it can work through a compatible stack, but llama.cpp warns of degraded generation quality, and the author states plainly that the training is still not good enough. In addition, inference breaks on some queries containing question and exclamation marks, so the project remains an experiment and a teaching testbed.

But the work matters for another reason: it shows that the barrier to entry for low-level LLM experiments can be lowered considerably. For .NET developers, it is a clear example of how to build a minimal model, test an OpenCL scenario outside the CUDA monopoly and understand what actually goes into running a language model locally.