How to Test an LLM: Simon Willison's Pelican Test
Simon Willison tests LLMs with the prompt "generate an SVG of a pelican riding a bicycle." It looks like a joke, but the results say something real about what neural networks can do. This article re-runs the experiment with Russian-style variants such as "make an SVG of a cat that codes."
AI-processed from Habr AI; edited by Hamidun News
Simon Willison came up with an original way to test LLMs: ask a model to draw an SVG pelican riding a bicycle.
Why does the pelican test reveal what an LLM can really do?
Because SVG requires structured code, and a pelican on a bicycle is a complex, composite image. The test shows whether a model can do more than predict tokens: it has to build an idea and turn it into code.
How do you use Willison's test to evaluate a model?
Ask the LLM to draw an SVG pelican riding a bicycle. The output shows how well the model understands complex composite images, which makes this test more informative than many serious benchmarks.
Key Questions Answered
Why does the pelican test reveal what an LLM can really do?
Because SVG requires structured code, and a pelican on a bicycle is a complex, composite image. The test shows whether a model can do more than predict tokens: it has to build an idea and turn it into code.
How do you use Willison's test to evaluate a model?
Ask the LLM to draw an SVG pelican riding a bicycle. The output shows how well the model understands complex composite images, which makes this test more informative than many serious benchmarks.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.