Johns Hopkins created an agentic AI system to coordinate robot teams

Johns Hopkins Applied Physics Laboratory (APL) has presented an agentic AI architecture specifically designed to coordinate teams of robots. The system is built on large language models and enables heterogeneous — diverse in types and capabilities — robots to work together as a unified organism, autonomously coordinating and adapting to unexpected situations in real time.

The Problem of Heterogeneous Robot Teams

The challenge is that robots in real-world teams are rarely identical. One is a manipulator with a gripper, another is a mobile platform on legs with an array of sensors, a third is a specialized module with a specific tool. How can you make them work in harmony when each "speaks its own language"? Johns Hopkins has developed a scalable architecture that allows LLM agents to manage all these systems as a single whole. This is not merely a collection of separate controllers, but a genuine cognitive layer that understands the state of each robot and makes decisions for the team.

Three Pillars of the System

The architecture addresses three key tasks simultaneously. Autonomy. Each robot receives enough "intelligence" to make decisions independently, without waiting for commands from the center. This is critical for systems where a network delay of even 100 milliseconds can mean the difference between success and failure. Coordination. Agents exchange information about state, goals, and obstacles in real time. When two robots need the same resource, the system mediates the conflict and selects the best path for the team as a whole. Adaptivity. When equipment fails or conditions change, the system reassigns tasks on the fly, restructures the plan, and continues operating. This is not a rigid script, but a living response to a changing world.

What an AI Agent Can Do for a Team

Researchers have demonstrated that an LLM agent in this architecture is capable of:

• Planning for each robot — breaking down a common goal into specific tasks, accounting for each robot's capabilities
• Conflict resolution — when two robots compete for the same resource, the agent selects optimal allocation
• Interpreting sensor data — understanding exactly what happened and why the plan requires adjustment
• Task reassignment on failures — if a manipulator breaks down, the agent redirects work to an alternative method
• Learning from errors — the system remembers what works and what doesn't, and adjusts tactics accordingly

This is far more than simply automating a sequence of commands. This is coordination at the level of high-level decision making. The robot doesn't simply execute a fixed script, but reasons about context, weighs options, and selects the optimal path to achieve the goal.

From Theory to Hardware

Johns Hopkins did not stop at simulations and theoretical models in the style of many academic laboratories. The team actually deployed its architecture on physical robots and demonstrated how the system functions in "combat conditions." This is a critically important step, because in AI laboratories, things often "work perfectly," and when theory meets reality — unexpected problems emerge. The researchers shared practical lessons from this experience in their presentation. They concern both technical problems — network latency, asynchronicity between components, state interpretation errors — and more strategic questions about how to properly decompose a complex task for an agent.

What This Means

This is a significant step toward semi-autonomous multi-robot systems that can operate in real-world conditions with minimal human oversight. Of course, we're not talking about futuristic R2-D2s yet, but engineering teams of robots that can work on construction sites, in factories, during rescue operations, in dangerous or inaccessible places. If this architecture proves to be truly scalable (and Johns Hopkins is sending confident signals about its belief in the approach), then over the next two to three years, we will likely see the first industrial prototypes and pilots based on it.