AI Intake Agent Development — When Not to Use AI

The platform's flagship product required collecting sensitive biographical data through a conversational AI agent. The first working version produced a report. A close read found fabricated details, invented astrological placements, incorrectly attributed events, and in one case a fabricated reference to a real person in a sensitive context. These weren't edge cases. They were systemic, and they would have reached paying users.

We scrapped heuristic parsing entirely and rebuilt the intake agent on Claude's tool_use API — making all state transitions deterministic rather than inferred from prose. Every biographical data point routes through validated structured fields. We added a mandatory extraction pass after every message, implemented 11-layer cross-user contamination defense, and documented a hard behavioral contract for the agent: acknowledge without mirroring, channel all data through validated tool calls, never assign meaning to what the user shares.