Taxbit launches AI assistant for tax professionals: 55% of users return with another question

If you have built a customer-support chatbot, you know how this usually goes. The bot answers confidently, cites something unclear, and then sends the user to a page they cannot access. In tax, that kind of mistake is costly: people make decisions about filing returns based on the information they receive. The team at Taxbit, a tax-compliance platform for crypto exchanges and brokers, explained how it built Assist, an assistant that can be trusted in a regulated environment.

The requirements that shaped the architecture

The assistant has four strict constraints. Every answer must cite a source the user can open, and if no answer is available, the bot must say so. The bot accounts for the user's role and organization: a support employee must not be sent into an administrative workflow. It does not provide tax advice: it may explain how Taxbit calculates cost basis, but it cannot recommend which method to choose. Finally, data must remain within its designated region.

The agent runs on the Strands Agents SDK inside AgentCore Runtime. It is a small Python application consisting of a system prompt, a model, and eight narrowly scoped tools. These include knowledge-base search, two access-control checks, and four tools for working with support tickets. The agent has no access whatsoever to customers' tax data.

The key security decision is that the model never handles identity. The user's email address, organization ID, and role come from a trusted service and are bound into the tools through a closure. The model may decide whether to check permissions, but it cannot choose whose permissions to check. User input is sanitized, so a prompt injection cannot forge trusted context.

Where the biggest quality gains actually came from

The team spent most of its effort on the knowledge base: Amazon Bedrock Knowledge Bases on top of S3 Vectors, using Cohere Embed English v3 embeddings. It draws from three sources: the public documentation site and help center, documentation generated from the product's source code, and manually written articles.

The first unexpected gain came from cleaning the corpus. The sitemap contained about 400 URLs, but only roughly 70 were suitable for answering questions. More than 300 pages under /blogs/ were marketing content, and they competed with the help center as sources for answers to compliance questions. Excluding the blog took minutes and required no work on the model.

The second and most powerful improvement came from contextual chunking. A fragment from the middle of a document loses information about what it refers to. A paragraph about rules for non-US reporting regimes is dangerous to return in response to a question about IRS Form 1099. A Lambda function at the POST_CHUNKING stage therefore prepends each chunk with the document's H1 heading and a lead paragraph of about 128 tokens. The chunk body was capped at 384 tokens so everything would fit within the embedder's 512-token window. A production audit found that about 28% of chunks fell within the 385–512-token range, meaning their tails would otherwise have been silently truncated.

Search is organized as a funnel: 75 vector candidates, reranking with Cohere Rerank 3.5 in the same Retrieve call, the top 12 results, no more than two chunks per document, and the five best chunks passed to the model. The broad initial pool is necessary because the right articles often rank between 20th and 50th in vector search and move up only after reranking. Only public sources may be cited, and that rule is enforced in code rather than in the prompt.

Security and quality control

Bedrock Guardrails checks every incoming message through a separate ApplyGuardrail call that examines only the current turn. Passwords, CVV codes, and cloud keys are blocked. SSNs, ITINs, bank details, and IBANs are masked in place, allowing the agent to help with the process without seeing the actual values. For conversation history, it is critical that the actor ID come from the calling service: the runtime operates under a single service role, so otherwise every user's history would end up in the same partition.

Quality is measured on live traffic without blocking responses. AgentCore online evaluations sample 10% of sessions and assess them with six built-in LLM judges: helpfulness, harmfulness, coherence, instruction following, tool selection, and tool-parameter accuracy. Custom judges run on top of these, separately evaluating faithfulness to sources and the relevance of retrieved material. The distinction is deliberate: an answer can faithfully reflect sources that should never have been retrieved, in which case the search system—not the model—is at fault. A separate classifier watches for responses that drift into tax advice.

What to take away

First, clean the corpus before tuning the model. Marketing pages in the index can damage answers more than any prompt can repair. Second, if a chunk does not know what it belongs to, add a heading and scope. Third, keep everything related to permissions and identity outside the model's decision space. Fourth, separate retrieval and generation metrics, or you will end up fixing the wrong component.

The limitations also matter. The 55% repeat-question figure comes from Taxbit's internal analytics and measures engagement, not accuracy. The solution is tightly coupled to the AWS stack. The agent currently reads nothing from customer data, so it cannot answer questions such as “Why was my calculation produced this way?” Still, as a blueprint for an AI assistant in any regulated field—whether finance, healthcare, or law—the case study is highly practical.