Dokumenterra team tasks Claude with writing automated tests in five hours

After a feature ships, its behavior has to be described again for a tester, and the scenarios then converted into automated tests. Dokumenterra tried using product documentation for this: Claude reads it, develops test cases and writes tests. A password reset example shows what the process produces and where a person needs to check the model's assumptions.

According to the company's CTO, preparing prompts for one feature takes an hour, after which Claude works for another five hours. This estimate comes from the team's account. The article provides no comparison with manual test development, run costs or error statistics, so the gains cannot yet be calculated.

From instructions to two tests

In the example, the agent receives password reset instructions: open the login page, follow the reset link, enter a username or email address and request an email. It reads the documentation through MCP, a connection that provides access to tools for searching and reading pages.

Claude turns these instructions into two scenarios. The first checks that the link on the login page opens the password reset form. The second checks that an empty username triggers a validation error and leaves the user on the same page. It then produces automated tests that check the page URL, field visibility and the error message.

The second scenario has an important detail: the original instructions say nothing about an empty field. The model proposed the expected behavior itself. This is a useful starting point for a tester, but it needs to be checked against the product requirements. A plausible assumption does not become a rule simply because an agent has turned it into a test.

The code shown also relies on existing fixtures and the ForgotPasswordPage page object. The example therefore does not demonstrate building the entire testing infrastructure from scratch. Moreover, these two checks do not cover the complete password reset journey: receiving the email, following its link and setting a new password.

Two reviews remain in human hands

The team describes two mandatory manual stages. First, a person reviews the documentation, then the test cases created by the model. After that, they oversee the creation of automated tests.

The first review guards against outdated requirements. If the instructions describe behavior that has already changed in the product, the agent may faithfully reproduce the error in its scenarios. A passing test run would then demonstrate compliance with an incorrect expectation.

The second review checks coverage. A specialist familiar with the users may spot missing conditions, roles or exceptions. In the password reset example, it is also worth asking which scenarios are needed for an expired link and a repeat email request. These are possible additions to the test plan; the article does not show that the team implemented them.

The author cautions that the MCP server has only recently been introduced and the results are preliminary. Its handling of information in screenshots has not been verified either. The server returns Markdown with image links, but the team has not yet established whether the agent extracts the necessary information from them.

What your team can adopt

The practical value of this case lies in the workflow. The agent receives a feature description, a task, a PR and existing test cases, finds the documentation itself and then proposes a test plan. This gives the tester concrete material to review.

Precise constraints are particularly useful in this process. A statement that file uploads are supported does little to define the expected results. Specifying a maximum size and a particular error makes it possible to develop boundary tests. If a rule allows several interpretations, the behavior needs to be agreed on before it is encoded in a test.

In this case, MCP provides access to current pages, including restricted material when the necessary permissions are available. However, the account does not establish that the approach is impossible without it: for a small experiment, a selected documentation excerpt can be supplied to the agent. The connection itself also does not guarantee that the text is up to date and correct.

A team can start with one feature: review its description, obtain scenarios, agree on the expected results and only then generate code. To assess the benefits, measure preparation and review time, the number of corrections and how reliably the tests hold up through subsequent changes. Dokumenterra's case offers a reproducible framework for such an experiment, but does not yet demonstrate savings for every team.