Tag
Production
2 posts filed under Production.
Cost Control for LLM Apps: Caching, Batching, Model Tiers
Cut your Azure OpenAI bill with model-tier routing, the four caches, output discipline, and batching. The levers that move the bill, in order of impact.
The Agent Harness
The harness is the loop around a model that calls tools, manages context, gates actions, and decides when to stop. Its defaults are the product.
