SkillMem stores an AI agent skill in 9.6 KB and speeds up a team 4.2x

Teams of several AI agents solve problems better than a single model: one plans, another critiques, a third refines. To keep agents from repeating the same mistakes, developers give them "skills", short recipes such as "check your assumptions" or "verify the intermediate result". Researchers at Rensselaer Polytechnic Institute and IBM have shown how to store such skills at 9.

6 kilobytes each without losing speed. They call the method SkillMem. On Qwen3-4B, it raised accuracy on GSM8K (grade-school math word problems) from 72.

40% to 89.90%, and on GPQA-Diamond (expert-level questions in physics, chemistry and biology) from 17.97% to 35.

63%. These figures are compared against the baseline system LatentMAS, in which agents communicate through hidden vectors and do not write text to each other.

Why a text skill eats the gains of latent communication

Latent multi-agent systems are faster than conventional ones because agents do not generate intermediate messages in words. They pass the model's internal states to each other, and only the last agent produces a text answer.

But a skill is usually inserted into the prompt as plain text. Every time it is used, the model runs those tokens through itself again. Over a long series of requests, this "prefill" eats into the very speed the whole approach was meant to deliver.

The alternatives are awkward too:

• A key-value (KV) cache for a skill avoids the repeated pass, but it stores the state of every token at every layer. Its size grows with the skill's length and the model's depth. • A LoRA adapter for each skill is a set of weight corrections. It is smaller than the whole model, but a hundred skills already add up noticeably.

How a skill becomes a set of a few vectors

SkillMem stores each skill as a small set of trained vectors outside the model. The model stays frozen. On request, the system finds the right skill: skill descriptions are encoded by the same model, and selection is based on cosine similarity to the query. The skill's memory is then available to all agents in the chain.

Giving every role the same vectors is not enough, because a planner and a critic need a skill for different purposes. So several layers of the model include a cross-attention "reader". The agent's hidden state forms a query to the memory, and the memory returns the relevant pieces. The role vector comes from the agent's system instruction. A sigmoid gate decides how strongly to mix in what was read. The reader's parameters are shared across all skills and roles, so each skill carries only its own memory.

The memory is trained in an unusual way. It is not forced to reconstruct the skill text. Instead, the researchers take a model response generated with the text skill and compare each token's probability with and without the skill. That difference is the skill's effect. The memory and reader are trained to reproduce the same shift. Tokens where the skill genuinely helps get more weight. Where the skill adds nothing, the model is kept close to its original behavior.

The authors obtained the text skills by comparing successful and failed solutions of the same task by an agent team. But a ready-made, hand-written skill can also be used as input.

What the experiments on Qwen3 showed

The evaluation covered six benchmarks for reasoning, question answering and code generation, on Qwen3 models from 4 to 14 billion parameters. The memory configuration is the same for all sizes. The authors write that smaller models benefit the most, and that the speed of latent inference holds at any size.

Efficiency figures:

• 9.6 KB per skill, including the retrieval vector. • End-to-end inference is 4.23 times faster than a team of agents that communicate in text (TextMAS). • It is 1.63 times faster than LatentMAS with skills supplied as text. • With a library of 100 skills, the total size, including the shared reader, is 38.3 times smaller than a KV-cache and 86.6 times smaller than LoRA adapters.

The authors compared the LoRA approach with the S2L method, which stores skills in model parameters.

What this means for engineers

The idea is practical for anyone building agent systems on small open models. Skills can be kept as a library of tiny files and loaded on request, without bloating the context or accumulating gigabytes of adapters. The larger the library, the bigger the savings.

There are limitations. Each skill has to be compiled once: you generate sample responses and train the memory. The method relies on a latent multi-agent architecture and has been tested on the Qwen3 family. A strong gain on a 4B model does not mean the effect will be of the same magnitude on larger models. Still, the direction is clear: agents' procedural knowledge can be stored as compact memory rather than as long text in the prompt.