A quoting screen in an internal system has run for six years. It reads three tables, applies a discount rule nobody wants to touch, and prints a PDF. Somebody now wants it to draft the covering note that goes out with the quote, and the instinct in the room is that this is a new project with a new system next to the old one.
It is a registration. AddChatClient goes into the same startup file that already registers the repository and the mail sender, the client resolves into the same service the screen calls, and it is wrapped in the logging and the failure handling everything else uses. The architecture does not move. One dependency is now a network call that charges by the token, sometimes takes four seconds, and occasionally makes things up.
The Change Lands in Startup
Register the client the way the documentation registers it, with a builder, and then read that builder as the list of things that happen around every model call. Microsoft states the shape directly: IChatClient instances can be layered to create a pipeline of components that each add additional functionality, and those components can come from Microsoft.Extensions.AI, from other NuGet packages, or from code you write. So AddChatClient followed by UseDistributedCache and UseOpenTelemetry is the entire integration surface for caching and tracing, and the distributed cache in question is whatever IDistributedCache the application already registered. Registering the client this way also puts it under checks the host already runs: in the development environment the default service provider verifies that scoped services are not resolved from the root provider and not injected into singletons, so a per-request dependency captured by a long-lived client fails fast on a developer machine.
These are delegating clients. Each one wraps the next, which makes the registration order the nesting order: whatever is registered first sees the request first and the response last. That is ordinary middleware behaviour, and it is why the pipeline sample in the IChatClient documentation carries a comment inviting you to explore changing the order of the intermediate Use calls. In the sample printed there, the cache is registered first, function invocation second, and telemetry third.
What Does a Cache Hit Skip?
Caching is usually the first thing added, because token spend is the first thing noticed. UseDistributedCache layers a DistributedCachingChatClient around the client below it, and the documented behaviour is specific. When a novel chat history is submitted it forwards the request, caches the response, and returns it. The next time the same history is submitted and a cached response is found, it returns the cached response rather than forwarding the request along the pipeline. That last clause is the whole point. On a hit, the request never reaches anything registered after the cache.
Put the cache outside function invocation and a cached answer comes back without the tool underneath it ever running. For a question about what a policy document says, that is exactly right and it is most of the saving. For a question about an order balance it means the answer was generated at some earlier point, and the call that would have read the current balance was skipped. Nothing errors, nothing retries, and no log line says a cache was involved. The number is simply from before.
The second half is what the cache matches on. The page describes the lookup in terms of the chat history that was submitted, so the unit is the whole submitted history and not the last thing a user typed. In a running conversation every turn submits a longer history, which is a different history, so the hit rate on multi-turn chat is close to zero. The hits come from repeated single-shot prompts: classification, extraction, and summarise-this-document work. That happens to be where most of the spend sits in a business application anyway, which is why the cache is worth having and worth placing carefully.
So choose the order deliberately and write down the reason. Cache outside function invocation for prompts that do not read live data. Cache inside it, or skip the cache entirely, for anything that does. If the traces have to account for every request the application made rather than every request that reached the model, the telemetry client belongs outside the cache too. This is a one-line ordering decision in startup that nobody looks at again once the pipeline is written, which is the argument for spending twenty minutes on it while it is still one line.
The Boundary Between the Client and the Business Rule
The injected client belongs behind the service that already owns the discount rule. That rule is the reason the quoting screen exists, and a covering note describing a discount the rule did not apply is worse than no covering note at all. What prevents it is handing the model the computed quote and asking it to write prose about that, instead of handing it the raw rows and an instruction to work the discount out.
The version that skips the service layer is quick to build and hard to see afterwards. Someone injects the chat client into the controller, passes the rows straight through, and the prose is right nine times in ten. The tenth is a quote that reads as though a discount applied when it did not, sent out under the company name, with no failed validation anywhere in the logs because nothing was validated.
Keep the model call downstream of the rule; a task-specific agent is this same boundary with a tool list attached to it. The client reads the output of the calculation, produces text, and hands it back. When the AI service is unavailable the quote still prints without the covering note, because the note is an enrichment on a workflow that already worked, and the only person who notices is the one who was expecting a note.
What Reaches the Trace
UseOpenTelemetry gives the model call the same tracing the rest of the application already has, written against the published OpenTelemetry semantic conventions for generative AI systems. The question that arrives about ten minutes later is what ends up inside the span, because a prompt assembled from a customer record contains the customer record.
The default is narrow and it is worth knowing precisely how narrow. EnableSensitiveData defaults to false, and the reference says what that covers: telemetry includes metadata such as token counts, and excludes raw inputs and outputs, meaning message content, function call arguments, and function call results. That default is also reachable from outside the code. It flips when the OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT environment variable is set to true, case-insensitive, and environment variables are set by whoever configures the environment, which is not always the person who wrote the application. Setting the property explicitly in code overrides the variable, and that is the version of the decision that survives a deployment configured by somebody else.
Where IKRC Fits
On that quoting screen the decisions that took the time were which value the covering note is allowed to read, where the cache sits relative to the balance lookup, and whether message content reaches the traces. None of those is a question about the model. IKRC works at that layer, inside applications that are already in production, through .NET development.
A first conversation that goes anywhere starts from one screen. Say what it does today, name the value on it that would be embarrassing to get wrong, and say who hears about it when the AI service is down for an hour. Call IKRC at 646-783-1441 or email info@ikrc.co.
Related Reading
For scoping what an agent is allowed to do, read Why a Task-Specific Agent Beats a Chatbot Bolted Onto Your App. For the data layer underneath it, read Before You Add AI, Fix the Data Retrieval Layer.
Need this solved in your software?
IKRC builds the custom systems, integrations, and modernization work discussed in this article.