The Model Is the Easy Part: My First Lessons in AI Engineering

I wanted to build an AI project that shows real AI engineering, not just a chatbot with a nice prompt. Connecting to an AI model is easy. You can do it in an afternoon. The hard part is everything around it: giving it the right data, proving its answers are right, testing it, watching it once it's live, and stopping it from answering things it shouldn't.

So I picked a problem where this really matters: numbers. My rule was simple: every number the AI gives must come from an official record, and a separate program (not the AI)

That became Cited Answers, an assistant that answers questions about Kenya, Uganda and Tanzania using World Bank data.

Why not just ask ChatGPT, Google or Claude?

Good question. For everyday questions, those tools are great. The difference is what this one can prove.

ChatGPT and similar tools answer from what they learned in training or from websites. They might add a link, but nothing checks that the number is really there. If the data is missing, they might guess. You just have to trust them.

Cited Answers only uses one official dataset. Every number is linked to a record and checked by code. If the data isn't there, it says, "I don't have that." And it's tested with 30 questions every time I change something.

The AI engineering behind it

The language model (Claude Haiku) is one piece. These are the skills that turn it into something you can trust:

1. Retrieval-augmented generation (RAG): 1,241 World Bank figures in PostgreSQL with pgvector, so answers come from a database, not memory. The model can search by meaning ("power at home" finds "electricity") or do an exact lookup and knowing when not to use embeddings made it faster and cheaper.
2. Tool calling and structured outputs: The model can't just write a paragraph. It must give the answer plus the source of every number.
3. Verification in code: A small Python program compares every number with its record. A wrong one gets one retry, then it's removed and the reader is told.
4. Testing on every change: 30 practice questions, including trick questions, run automatically on GitHub. If an answer breaks, the change is blocked.
5. Monitoring: every answer traced in Langfuse, with cost, speed and a citation score, tagged by prompt version.
6. Guardrails: off-topic and prompt-injection attempts are refused, and usage limits cap the cost.
7. Model Context Protocol (MCP): the same data tools work in Claude Desktop or any MCP client.
8. Packaging and deployment: Docker runs the database locally and packages the app into one container, with the embedding model built in so it starts without downloading anything. For the public version, the data lives in Neon, a hosted PostgreSQL database that supports pgvector.

The moment the engineering paid off

My checks said almost every number was correct. Then I read the actual answers in Langfuse, and spotted this:

"47.1% of Ugandans had access to electricity in 2022, the latest year available."

The number was right for 2022, but the database had 2024 (55.3%). In 4 out of 5 runs, the model called an older year "the latest". My checker passed it, because it checked numbers, not the words around them.

A general assistant would show you that confidently, and you'd probably never notice. Here, monitoring surfaced it and testing measured it. I gave the model the latest year in every lookup, and stale answers dropped from 4 in 5 to 0, and it used about 11% fewer tokens, which made it cheaper to run, most likely because it no longer needed extra lookups to find the newest year.

The lesson: a good score can still hide mistakes, because a test only catches what it was built to check.

What it's really for

World Bank data is just my example. Cited Answers isn't trying to compete with ChatGPT for the public. It's a blueprint for something general assistants can't offer: an AI assistant on your own data that proves every number.

Different organisations could put one on their websites or internal tools:

• A bank or credit-rating agency: clients ask about ratings and financial figures, and each one links to the published report
• A government statistics office: citizens ask about population, prices or school enrolment, answered from official tables
• A hospital or NGO: staff ask questions over their own reports instead of digging through spreadsheets
• Any business with a dashboard**:** an "ask our data" box where every answer shows its source record

If your colleagues keep asking ‘where did that number come from?’, I can build you an assistant that answers before they do.

Final thoughts

Building this taught me that the model is the easy part. Making it trustworthy is the real work: good data, ensuring it shows proof, testing, monitoring, and letting it say "Not sure." That's the work I want to do, and this project is just the beginning.

That's a wrap! I'm Favour Ludenyo, and I’ll keep building things and noting down the little lessons along the way. Follow along at ludenyowrites.hashnode.dev. Your next favourite bug story is coming.