When ChatGPT came out, everyone wanted to build AI applications. Fair enough — the barrier collapsed almost overnight. Any engineering team can wire up an LLM API and put a working demo in front of stakeholders by the end of the week. LangChain, LangGraph, the model APIs themselves — that path used to take months, and now it takes an afternoon.
Which is genuinely great. But it has also cemented a misconception I keep running into: the idea that building an AI product is mostly about choosing the right model.
In my experience, that's almost never the hard part.
The hard part starts after the first demo works. Production systems need auth, routing, evaluation, observability, governance, deployment pipelines, cost controls, and enough operational tooling that someone other than the person who built it can keep it running at 3am. They need to survive model swaps, prompt rewrites, and the business changing its mind about the use case twice a quarter. None of that gets easier when the underlying model is more capable — it usually gets harder, because now people rely on the thing.
I've spent the last several years bouncing between OCR pipelines, computer vision, enterprise LLM platforms, and the infrastructure underneath. Models have changed enormously in that time. The engineering questions haven't: how do you make this reliable, observable, and cheap enough to keep running? How do you evolve it without breaking everyone who now depends on it?
That's the work I've come to enjoy most. Not the model itself, but everything around it — LLM gateways, agent runtimes, evaluation systems, inference infrastructure, governance layers, developer platforms. The layer that lets a company ship AI features without every team rebuilding the same plumbing.
This blog is where I want to think out loud about that layer. Some posts will be about architecture and distributed systems. Others will get into inference internals, Kubernetes, observability, or the trade-offs behind a specific platform decision I ran into at work. A few will probably just be notes I finally decided to clean up and publish.
I'm not planning to write "how to call an LLM API" posts — there are already plenty of those. I'm more interested in the engineering that begins after the API call succeeds.
The model, in the end, is just the beginning.