OpenAI tested everyday AI agents on 169 tasks and found their weak spot

Most AI benchmarks are convenient for developers and look little like real life. The model answers a question, the system compares the answer to a reference, and the table gets one more percentage. But users usually don't need a correct paragraph. They need to find a daycare after moving, prepare a family budget, fill out an application or arrange care for a relative.

OpenAI introduced LifeVal, a benchmark of 169 personalized tasks in which the agent must not only reason but act: read conversation history, take files into account, use a browser, change data in apps and create finished materials. It is an attempt to measure the gap between "the model explains things well" and "the job is actually done."

What the evaluation looks like in practice

Imagine a task: a family has moved and asks for help choosing a daycare. An ordinary chatbot will list general selection criteria. In LifeVal, the agent receives the address, the family's preferences, past conversations and access to a working environment. It must find suitable options, check admission conditions, build a short list and flag decisions that must not be made without a human.

The persuasiveness of the text is not what is scored. The grading model sees the original request, the task criteria, the files created and the changes to system state. If the agent wrote that it filled out a form but the form stayed empty, there are no points for that action. An attempt to access the internal interface of the test environment instead of the ordinary website counts as hacking and zeroes the result.

The task set was built from time-use research and a user survey. The study involved 3,837 people, and the resulting map covered 148 categories of everyday tasks: personal finance, relocation, job search, shopping, education, health, caring for loved ones and obtaining government services. The tasks themselves were created by people with practical experience, and each then went through at least three stages of manual review.

Where agents still break

The most telling problem is incorrect information or recommendations. An agent can confidently carry out a long chain of actions yet miss one important detail from the user's earlier conversation. For an everyday task that is enough to make the whole result useless: picking housing outside the budget, ordering a product with an allergen or choosing an unsuitable service.

There is also a subtle conflict between autonomy and safety. More cautious models more often stop at the edge of a decision: they don't complete an action without confirmation or report that they couldn't verify the data. This looks like a failure in the table, although in a real product that is sometimes exactly how a reliable agent should behave.

LifeVal should not be considered a final measure of quality either. The survey was conducted in the US and in English, so the set doesn't reflect all countries and habits. The value of tasks is based on stated willingness to pay, not on years of observing real behavior. Some web data changes over time, so repeated runs may produce different results. Finally, the two automated graders are built on OpenAI's own models, which calls for caution when interpreting the numbers.

What a product team should take away

The main idea of LifeVal is more useful than any particular model ranking. Acceptance tests for an agent should be built around the completed result: did the file appear, did the required field change, were the user's constraints respected, did the agent ask for confirmation before a risky action. Memory, error recovery and the ability to stop honestly should be tested separately.

For developers, this means moving from a set of polished demos to reproducible scenarios with an initial state, a measurable outcome and an action log. For product managers, it means deciding in advance where the agent may act on its own and where it must hand control back to a human. It is at these boundaries that it is decided whether the system becomes an assistant or just a very talkative interface.