This started as a way to protect a holiday. It turned into the clearest demo I have of the work I want to do next.
Why I cloned myself
I had three weeks of annual leave booked, and I didn’t want to spend any of it watching Teams and Outlook out of the corner of my eye. As the only person who really knows certain systems, leave doesn’t switch off the questions. It just queues them.
Around the same time everyone was talking about “distillation,” in the loose sense: take what’s in your head and press it into something reusable. Clone yourself, sort of. I liked that as a weekend problem, so I gave myself two reasons to build it and started.
One was selfish curiosity. I wanted to take this class of tech apart by building a toy with it rather than reading another thread about it. The other was practical. If a teammate could ask a bot the small stuff, they wouldn’t have to wait three weeks for me to get back.
The obvious question is why they’d ask me at all instead of asking a perfectly good public model. The answer is that the useful questions aren’t in any public model. They’re things like the production config that only lives in one engineer’s memory, the fix for an edge case we hit once at 2am, the bit of trivia about why a strange workaround exists, and the internal context that no general-purpose AI has ever seen. A chunk of that I’d already written down in my own knowledge base. So the real question became: could I stand up a version of me that answers from it?
I built it, and it worked better than I expected. It fielded a real share of the day-to-day “quick question” traffic.
The public version
Then I started interviewing. Almost every Forward Deployed and Applied AI role on my list asked for the same stack: RAG, vector databases, production AI on a cloud platform like GCP. I had the delivery half of that down already. Customer-facing, end to end, integration-heavy work has been the day job for years. The specific artefact those ads want, a grounded retrieval agent on a managed cloud stack, I’d mostly read about.
So I rebuilt the holiday toy in the open. I stripped everything company-specific out of it, the production details, the internal context, anything I can’t share, and grounded a fresh version in my own engineering knowledge instead. That public version is Ask Alex, and it’s live on this site at /ask.
Reading about RAG and shipping one are different things. The interesting problems aren’t in the diagram. They’re in the seams: how you ground answers without hallucinating, what you do when retrieval comes up empty, and what the thing costs you the day after you walk away from it.
What it is
A chat agent that retrieves from a knowledge base of my engineering writing, answers in my voice with inline citations, and hands you to me directly when a question falls outside what it knows. The frontend is a static page and a floating widget on this site. The backend is a small service doing the actual retrieval-augmented generation.
The stack
One request does four things in order: embed the question with a Vertex embedding model, pull the nearest knowledge-base chunks out of pgvector, hand those to Gemini as grounding, then stream the answer back token by token. Auth to Vertex runs through the Cloud Run service account, so there’s no API key sitting in the code. It deploys straight from source with buildpacks, which means no Dockerfile to maintain.
Three decisions that mattered
pgvector, not a managed vector index. A managed vector search service is the resume-friendly choice. For a knowledge base this size it’s mostly idle cost and another thing to operate. Postgres with the pgvector extension gives me a real vector database, the thing the ads actually ask for, on a free-tier instance. Retrieval is one query:
SELECT content, doc_title, embedding <=> $1 AS distance
FROM kb_chunks
ORDER BY embedding <=> $1
LIMIT 5;The decision rule was the one I apply to everything: what does this look like the day after deploy, when something breaks at 11pm. A Postgres table I can query by hand beats a managed index I’d have to learn to debug under pressure.
Grounding and citations over raw generation. The model only sees the retrieved chunks, and it’s told to answer from them and cite them inline. The citations aren’t decoration. They’re the audit trail. If the agent says something about how I work, you can see which source it came from, and I can see when retrieval pulled the wrong passage. Ungrounded answers are where these systems embarrass you, and the citation is the seatbelt.
An escalation tool, not a confident guess. When the knowledge base doesn’t cover a question, the worst outcome is a fluent, wrong answer. So the model gets exactly one tool, and it decides when to call it:
{
name: "escalate_to_human",
description:
"Hand off to the real Alex when the knowledge base does not cover " +
"the question, or the user asks to reach Alex directly. Do NOT call " +
"this if the answer is in the provided context.",
parameters: { /* reason, summary, priority */ },
}When Gemini calls it, the UI stops pretending and shows a handoff card. That’s the conversational-AI pattern that actually matters in the field: knowing the boundary of what you can answer, and routing past it cleanly instead of guessing your way to wrong. It’s also the original idea, intact. The bot covers the small stuff; the genuinely hard question still finds its way to a human.
Making it safe to leave running
A public endpoint backed by a paid model is an invitation. The guardrails are unglamorous: CORS locked to the origins I control, a per-IP rate limit, a hard cap on output tokens, the database connection string in Secret Manager instead of the image, and Cloud Run scaling to zero so an idle demo costs nothing. None of it is clever. It’s the difference between a demo and something you can link from your site and stop thinking about. Which, given the whole point was to stop thinking about work for three weeks, felt appropriate.
What’s not there yet
Being honest about scope is part of the job. There’s no full evaluation harness yet. I log latency and token counts on the server, but I haven’t built the offline eval set that would let me regression-test retrieval quality. There’s no dashboard surfacing cost per request. It’s a single agent with one tool, not a multi-agent system. Those are the next layer, and I know where they’d go. Shipping the grounded, cited, escalating core first, and saying plainly what it doesn’t do, beats a half-built version of everything.
Why this is the work I want
Strip the topic away and this is the shape of forward-deployed work: wire a frontier model into live infrastructure, ground it in real data, put it behind production guardrails, and be honest about where it stops. The connective tissue between the model and everything around it is the actual job. The model is the easy part.
It started as a way to protect three weeks of annual leave from my own systems. It ended as the clearest evidence I have of the work I’d rather be doing full time. Go ask it something. When it hands you off, that’s working as designed.