Ci-Cd

Build an eval pipeline with golden datasets, scoring, smoke/full modes, and CI gates to catch prompt regressions before release.

Revised with .NET examples — A newer version of this article, covering both Python and .NET, is available as part of the MAF v1: Python and .NET series: MAF v1 — 23-evaluation-framework. The newer version applies three substantive fixes to the framework below — canonical AgentRunResponse extraction (no more hasattr chain), word-boundary alias matching (the original false-positives "profit" against the "price" alias), and a smoke / full tier split for CI vs nightly runs. Read this article for the conceptual ground; read the new one for the production-grade implementation.

A custom image that only exists on your laptop is not shipped; it is a local experiment. The moment a teammate or a server needs to run it, the image has to live in a registry. Docker Hub is the one most people start with, and the push workflow is the same shape you will later use against Azure Container Registry or Amazon ECR, so it is worth doing once by hand before automating it. This post walks through that manual path with a small nginx-based image.

In the previous article I built a custom nginx image and pushed it to Docker Hub by hand: build, tag, login, push. That works exactly once. The second time you forget the tag, the third time a teammate pushes from a laptop with a stale base image, and soon nobody can say which commit produced the image running in production. The fix is boring and reliable: let GitHub Actions do the build and push on every commit to master.