Open-Source MiniMind Lets You Train Your Own LLM from Scratch for $0.42 on an RTX 3090
MiniMind is an open educational project (67.9K GitHub stars) that trains a large language model (LLM) from scratch instead of calling a ready-made one through an API. According to the author of the post on X, a run costs $0.42 on a single consumer RTX 3090 graphics card. The code is written in plain PyTorch, so every stage can be opened and studied. For a team, it is a cheap testing ground for understanding how a model becomes an assistant.
AI-processed from X: @Shoumya23; edited by Hamidun News
Most developers work with large language models (LLMs) as a black box: send a request to an API, get a response. Few have seen in practice how a model is trained, where its "character" comes from, and why it suddenly answers instructions instead of just continuing the text. The open-source MiniMind project fills this gap: you can go through the entire process of building an LLM yourself.
The author of the post on X, Shumya Vishwakarma, presents it this way: you can train your own model from scratch for $0.42 on a single RTX 3090 graphics card. This is an ordinary gaming card, not a multimillion-dollar cluster.
The repository has collected about 67.9K stars on GitHub, which signals strong interest in this format.
For an educational repository, the star count is very high. It points to demand for clear, step-by-step examples instead of bulky frameworks.
What's inside: point by point
The list from the post shows that MiniMind covers nearly the entire modern training cycle.
Pretraining. The model is built from scratch, reads a large body of text, and learns to predict the next word. In big labs, this is the most expensive stage; here it is scaled down to the size of a single graphics card.
SFT (supervised fine-tuning). Fine-tuning on pairs of "question — good answer." This is the step where a "word predictor" turns into an assistant that answers questions.
LoRA. Lightweight fine-tuning: instead of changing all the model's weights, small added matrices are trained. This lets you adapt a model to your task on modest hardware.
DPO. A way to align answers with human preferences without a separate, complex judge model: the algorithm is shown pairs of "best answer — worst answer."
GRPO. A reinforcement learning method popular in models that reason. The model generates several variants, gets them scored, and reinforces those that worked better.
Tool use. Training the model to call external tools: search, a calculator, APIs. This is the foundation of any agent.
Distillation. Transferring knowledge from a large model to a small one: the "student" reproduces the "teacher's" answers and gets part of its quality at a much smaller size.
Why everything fits on one card
Two practical points on the list are just as important. First, the implementation is written in plain PyTorch. There are no abstraction layers like ready-made "trainers" hiding the logic. You can open a file and see for yourself how the loss (the number showing how wrong the model is) is computed, how the training loop is structured, and where LoRA gets attached. For testing ideas, this is more valuable than a convenient but opaque interface.
Second, everything is designed for a single RTX 3090. It is consumer hardware that many enthusiasts already own and that can be rented for modest money. That is why the $0.42 price looks plausible: it is the order of magnitude of a short GPU rental, not a lab budget. The figure comes from the post. You will need to recalculate it for your own conditions, since rental prices and training speed vary.
Ollama support closes the loop. A trained model can be run locally with the same tool many people already use for Llama or Qwen, and checked in action right away.
What to expect, and what not to
The scale needs to be understood soberly. A model trained for a dollar on a single card will be tiny compared with commercial LLMs. It will not replace GPT-class systems and is not suitable for serious production use. Its value lies elsewhere: it is a lab where you can run experiments quickly. For example, you can see how behavior changes after DPO, or compare SFT and LoRA on your own data.
What to do in practice:
- If you build agents, go through the tool use stage on a small model: it is easier to understand where function-calling errors come from.
- If you are a product manager or founder, use the project to estimate the real cost of customizing a model: the starting price is lower than it seems.
- If you are teaching a team, give them the repository as a study guide: each training stage is isolated, and it can be understood in an evening.
MiniMind does not promise a revolution in model quality. The project lowers the entry barrier to LLM training from "you need a cluster" to "a gaming graphics card is enough." For an industry where understanding model internals is becoming a competitive advantage, that is reason enough to try it.
- Nov 21, 2026TensorRT-LLM Beats vLLM on H100: 2780 vs 2400 Tokens/s on Llama 3.3 70B
- Nov 21, 2026Open-Weight Inkling 2 Mini Matches Qwen3.8-27B on Independent Benchmark, Runs on 16 GB of Memory
- Nov 21, 2026Qwen3-0.6B Fine-Tuned on a Mac mini: Errors Dropped to 3 out of 30 from 9
- Nov 21, 2026Thinking Machines Releases Open-Weight Inkling 2 Mini with 26.4B Parameters for 32 GB Macs
Key Questions Answered
What's inside: point by point?
The list from the post shows that MiniMind covers nearly the entire modern training cycle. Pretraining. The model is built from scratch, reads a large body of text, and learns to predict the next word. In big labs, this is the most expensive stage; here it is scaled down to the size of a single graphics card. SFT (supervised fine-tuning). Fine-tuning on pairs of "question — good answer." This is the step where a "word predictor" turns into an assistant that answers questions. LoRA.
Why everything fits on one card?
Two practical points on the list are just as important. First, the implementation is written in plain PyTorch. There are no abstraction layers like ready-made "trainers" hiding the logic. You can open a file and see for yourself how the loss (the number showing how wrong the model is) is computed, how the training loop is structured, and where LoRA gets attached. For testing ideas, this is more valuable than a convenient but opaque interface. Second, everything is designed for a single RTX 3090.
What to expect, and what not to?
The scale needs to be understood soberly. A model trained for a dollar on a single card will be tiny compared with commercial LLMs. It will not replace GPT-class systems and is not suitable for serious production use. Its value lies elsewhere: it is a lab where you can run experiments quickly. For example, you can see how behavior changes after DPO, or compare SFT and LoRA on your own data.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.