How 15 AI models approach finding the optimal XML parser for iOS: benchmark results

A developer conducted an unusual experiment: he loaded the same task into 15 popular AI models and compared their results with his manual benchmark of XML parsers for iOS.

About the Task

Six months ago, the author published the results of his own research — which XML parser for iOS, tvOS, and macOS performs fastest. It was painstaking work: several hours analyzing GitHub repositories manually, checking popularity (minimum 500 stars), support for Objective-C and Swift languages, integration via CocoaPods or SwiftPM. After three hours of hard work (and several cups of coffee), a comprehensive rating of optimal parsers was born.

Can AI Do It Faster

Then came a natural thought: why spend 3 hours if the internet promises that AI can handle it in 5 minutes? Moreover, there's a real chance that the manual benchmark contained an error somewhere — incorrect code interpretation, a missed detail in specifications. And if that's the case, AI systems, possessing vast amounts of knowledge, might find a more correct result. The decision was natural: load the same task into 15 different AI models (OpenAI, Anthropic, Google, Meta, Xai, Perplexity, and others), collect their results, and honestly compare them with each other and with the original benchmark. A fair experiment.

Results Disappointed

The results fell far short of expectations. Contrary to everything, GPT 5.5 Pro not only failed to perform better than the others — it placed last.

This shocked immediately: OpenAI's flagship lost in all categories of analysis, identification of popular repositories, and evaluation of parser performance. Claude Opus 4.7, renowned for deep analysis and ability to maintain context, also failed to take first place, though results were above average.

Instead, unexpectedly ahead were more specialized and compact models that better navigated the practical details of the task. The author honestly admits: perhaps there was indeed an error in his manual benchmark, and thus he chose a not entirely optimal parser. But even if that's the case, the result shows an interesting pattern: the size and self-proclaimed quality of an AI model does not always guarantee success in a specific practical task.

What Does This Mean

The experiment reminds developers that AI is a tool with its own strengths and weaknesses. For specific technical tasks, it's worth checking not only the popularity of the model, but also its real performance in your particular case. And yes, sometimes what promises to deliver in 5 minutes might require your careful attention and validation.