Self-Directed Product · AI Email Agent · Case Study 03
Governed tool use, hybrid retrieval, persistent traces, and human-controlled consequences
A governed support agent designed to investigate customer questions using live account state and proprietary product knowledge without being allowed to autonomously send consequential messages.
AI AgentsRAGHITL
6Max Tool Iterations
RAGGrounded Knowledge
HITLOutbound Control
01 / Problem & Context
What had to become true
The challenge
A governed support agent designed to investigate customer questions using live account state and proprietary product knowledge without being allowed to autonomously send consequential messages.
My role
I owned the problem framing, system/product decisions, implementation direction, validation, and iteration represented in this case study. The emphasis is on the decisions that changed the system—not a feature inventory.
02 / Decisions & Tradeoffs
Architecture follows the constraint
Key decisions
- Keep the tool registry read-only.
- Let the model choose tools dynamically inside a bounded multi-step loop.
- Ground product answers in an approved hybrid-retrieval corpus with provenance.
- Require explicit human approval before outbound communication.
Working loop
PROBLEM → INVESTIGATE → ARCHITECT → BUILD → VALIDATE → ITERATE
The implementation was treated as a measured loop: diagnose the actual constraint, make the smallest architectural change that resolves it, then verify behavior and quality before moving forward.
03 / System Design & Build
What I built
- OpenAI Responses API drives model-directed function calling with a maximum bounded iteration count.
- Tools retrieve account, subscription, entitlement, login, market-module, evidence, and product-help context.
- Hybrid RAG combines lexical and embedding retrieval over approved internal knowledge.
- Persistent traces record tool use, retrieval evidence, drafts, and agent runs.
- Resend inbound email, signed webhook verification, threading, DNS, and Admin review complete the production workflow.
04 / Validation & Outcomes
Evidence over claims
Validation
- Inbound email successfully reaches the Admin conversation workflow.
- Agent analysis and draft generation remain separate from explicit Send.
- Retrieved evidence remains inspectable through source/section/confidence provenance.
What changed
Useful autonomy does not require unrestricted authority. The agent can investigate dynamically while the system keeps customer-facing consequences under human control.
05 / Skills & Positioning
What this project demonstrates
AI Agents · RAG · HITLThis case sits inside a broader portfolio spanning AI systems and agents, full-stack products and revenue infrastructure, design engineering, and creative AI leadership.
Portfolio Map
Explore the full system of work
Each project is a different proof point. The current project is highlighted so the portfolio reads as one connected capability map rather than a collection of isolated case studies.
