Are AI Labs Training Models for the "Pelican on a Bicycle" Test? A Look at Pelicanmaxxing

On July 22, 2026, developer Dylan Castillo published a study testing the "pelicanmaxxing" hypothesis: that AI labs deliberately train models to draw a good pelican on a bicycle for Simon Willison's unscientific benchmark. Castillo ran 48 prompts three times each through 7 models and found no signs of such tuning.

What is the "pelican on a bicycle" benchmark

The "pelican on a bicycle" benchmark is Simon Willison's personal, informal test, in which he asks each new language model to generate an SVG drawing of a pelican riding a bicycle. Willison himself calls it "deeply unscientific," but over a couple of years the test has become so recognizable in the community that people began to suspect labs might be optimizing models for it specifically, to look good. The phenomenon got the tongue-in-cheek name pelicanmaxxing (pelimaxxing in Castillo's write-up).

How the experiment works

Castillo built a grid of 8 animals and 6 vehicle types, giving 48 prompt combinations, and ran each one three times through 7 different models. The matrix lets him compare whether models draw the "pelican + bicycle" pair anomalously better than any other combination, such as "flamingo on a scooter" or "heron in a boat."

• Publication date: July 22, 2026
• 8 animals × 6 vehicle types = 48 prompts, each run 3 times
• 7 models were tested: GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.5 Flash, Grok 4.5, Qwen3.7-Max, GLM-5.2 and DeepSeek V4 Pro
• Two separate models graded the results: GPT-5.6 Luna and Gemini 3.1 Flash-Lite
• The author built an interactive grid with filters for browsing all the generations

Did the study find traces of tuning?

Castillo found no evidence of pelicanmaxxing in any of his five lines of testing. Pelicans on bicycles do not look better than other scenes, pelicans are not drawn more neatly than other animals, bicycles are not drawn more neatly than other vehicles, and the combination itself does not come out better than the models' individual skills predict, even after adjusting for difficulty. The fifth check showed that the scenes do not look memorized.

GLM-5.2 came closest to an anomaly: it showed the largest quality gain specifically on the "pelican + bicycle" cell, and its very first sample caught the author's attention. But according to Castillo, the effect remains small and statistically insignificant.

"GLM-5.2 came closest: it has the biggest boost specifically on the pelican-on-a-bicycle cell.

But the effect is small and not statistically significant, so I wouldn't put much weight on it," said Dylan Castillo, the study's author.

What this means

Simon Willison, whose benchmark sparked the whole discussion, called Castillo's work "excellent" and noted that he himself had previously checked models only selectively, without such rigor. The study's conclusion should reassure skeptics: there is currently no evidence that labs "cheat" on one specific viral test. Models draw a pelican on a bicycle exactly as well as their general generation skills allow.

Frequently asked questions

What is pelicanmaxxing?

It is the suspicion that AI labs deliberately train models to do well specifically on Simon Willison's "pelican on a bicycle" test. Dylan Castillo's July 22, 2026 study found no evidence for it.

Which models performed best?

Across the 48-prompt matrix, GLM-5.2 came closest to a "suspicious" result, with the largest gain on the "pelican + bicycle" pair. However, the effect is not statistically significant, so the author does not treat it as proof of tuning.