Gemma 4 and Qwen Coder vs. the cloud: local LLMs on real-world work

Local models such as Gemma 4 and Qwen Coder are in an odd position. People don't take them seriously, yet few have actually tested what they can do in real work rather than on synthetic benchmarks.

The problem with YouTube tests

YouTube is full of local LLM tests. But they all look alike: someone takes a large model, runs it with whatever settings, and asks it to write a bubble sort. Of course it manages. Nobody is impressed by that.

The real question is different: can a local model write working code, refactor buggy files, and pull data out of HTML, the way real projects demand?

Most tests ignore parameters, and parameters often decide everything. Get the temperature, context window, or quantization scheme wrong, and the results fall off a cliff. Getting a bad result from a local model is easy. Getting a good one takes time.

Gemma 4 and Qwen: which models, which conditions

Vyacheslav tested several models, choosing ones that actually fit in the 16 GB of VRAM on an ordinary graphics card:

• Gemma 4 (Google) — a general-purpose model with good balance
• Qwen 3.6 (Alibaba) — balanced performance and speed
• Qwen Coder — specialized for code generation and analysis
• Run through llama.cpp with optimized parameters
• GPU optimization and the right choice of quantization for the available memory

The first part of the problem is simply getting the llama.cpp API up and running. The second is choosing the right parameters. Which quantization level? What temperature? How many tokens should the context grow to? These need to be tuned for the specific task, not guessed.

Results in an agentic environment

The author tested the models not on isolated examples but in a real agentic environment, with chains of actions where a mistake in one step breaks everything that follows.

• Writing working code on the first attempt
• Refactoring a codebase with its own logic and existing bugs
• Extracting structured data from HTML
• Following complex instructions within the task context
• Adapting when requirements change mid-session

The results showed that with correctly tuned parameters, local models perform on par with cloud solutions on typical tasks, without network latency.

Why local LLMs matter

This might look like an academic question. But there are scenarios where a cloud API isn't an option: sensitive data, air-gapped environments, regulatory requirements, and API costs at scale.

Local models give you control. You know where the computation runs. No surprises with data logging. That matters when you work with confidential information or in an environment where cloud APIs are banned.

What this means

Local LLMs have moved past the experimental stage. They are ready for real work, provided you're willing to spend time tuning parameters.

For businesses, this means that investing in a graphics card can replace cloud APIs for a whole class of problems, from coding to processing sensitive information.