LLM Application Engineering

What is actually included

• Retrieval over an existing document and data estate
• Tool use and structured output wired into existing business logic
• Evaluation harnesses and regression suites for non-deterministic systems
• Cost, latency, and failure-mode engineering for production traffic

Features, not chatbots

The interesting work is rarely a chat window bolted onto a homepage. It is the classification step that removes a manual triage queue, the extraction that turns twenty years of PDFs into something queryable, the drafting step that gets a caseworker to a reviewable first version instead of a blank page.

Those features live inside existing applications, so the engineering problem is mostly the ordinary one: how it behaves under load, what it costs per transaction, what happens when the model is unavailable, and how anyone finds out it has quietly degraded.

Retrieval done properly

Retrieval-augmented generation is easy to demonstrate and hard to make reliable. The demo works because the test questions match the documents. Production is different: the corpus is messy, permissions vary by user, documents contradict each other, and the honest answer is often that the information is not there. Handling that last case well is most of the difference between a feature people trust and one they quietly stop using.

Evaluation is the deliverable

A non-deterministic system without an evaluation harness cannot be maintained, because there is no way to tell whether a change improved it. Building the harness alongside the feature is not overhead – it is what makes the thing supportable a year later.

← All capabilities