Agentic ERP: deployment infrastructure
The pipeline that puts the copilot on live infrastructure, plus the 6 checks that have to pass before any release reaches a user.
Two-repo CI/CD architecture
The development repo runs a 6-job GitHub Actions pipeline on every push to
master: backend quality gates (ruff, black, mypy, bandit, pytest unit), frontend
tsc, integration tests, a red-team job (promptfoo), and a build that pushes
images to Azure Container Registry with keyless OIDC auth. A
repository_dispatch event then fires this deployment repo.
This deployment repo picks up the dispatch and runs a deploy job that waits for
manual approval through a GitHub environment. Once approved, the job runs
kubectl apply, then rollout wait, then the 6-check
smoke test. Keeping deployment in its own repo draws the production boundary
where a reviewer can see it: no commit reaches production without a second pair
of eyes.
Infrastructure as code: Bicep modules
| Module | Detail |
|---|---|
| AKS | A system node pool (Standard_D2s_v3) and a GPU node pool (Standard_NC6s_v3), with the OIDC issuer enabled for keyless workload identity. |
| Azure Container Registry | Premium SKU with geo-replication. GitHub Actions pushes over OIDC, so no registry credentials sit in CI secrets. |
| Postgres (Flexible Server) | pgvector enabled, reachable only through a private endpoint, with SSL enforced. |
| Redis Cache | Standard C1 SKU, backing the semantic cache (SHA-256 and cosine lookups) and the human-gate decision UUIDs (24-hour TTL). |
| CosmosDB | Serverless with multi-region read replicas. Holds conversation history and audit logs. |
| Azure Monitor / App Insights | The OTel DaemonSet forwards OTLP traces and metrics. Dashboards cover solver latency, cache hit rate, and human-gate decision volume. |
Kustomize overlays
One base set of Kubernetes manifests (Deployment, Service, ConfigMap, HorizontalPodAutoscaler) is shared across environments. Kustomize overlays patch only what differs between staging and production: replica counts, resource requests and limits, and image tags. No manifest is duplicated.
The NGINX ingress controller routes /api and /ws to the
FastAPI backend, and / to the Vite frontend. WebSocket upgrade
headers are forwarded explicitly, so the human-gate notification channel works
through the ingress instead of silently dropping at it.
Observability: the OTel DaemonSet
An OpenTelemetry Collector DaemonSet runs on every AKS node and receives OTLP traces and metrics from the application pods, then forwards them to Azure Monitor Application Insights over the OTLP/HTTP exporter. The dashboards track solver latency per intent (mcnf_solve, vrp_route, and the rest), Redis cache hit rate, human-gate decision volume with its approval ratio, and red-team probe outcomes from the CI gate.
Smoke test: 6 checks
After kubectl rollout status reports healthy, the
smoke-test.sh script runs 6 checks in sequence and fails fast on
any error:
- Health endpoint.
GET /healthreturns 200. - OpenAPI docs.
GET /docsreturns 200, which confirms FastAPI started correctly. - WebSocket reachability. A short handshake to the human-gate notification channel succeeds.
- Human-gate router. A test solver call above the $10,000 threshold reaches the gate node and returns a pending status rather than an error.
- Injection probe. A known prompt injection string comes back rejected, which confirms the PII sanitizer and the RBAC layer are live.
- Frontend reachability.
GET /returns 200 with the Vite bundle's HTML.
Security posture
Postgres, Redis, and CosmosDB sit behind private endpoints and are unreachable from the public internet. AKS workload identity (OIDC) authenticates pods to Azure resources without static credentials, and the GitHub Actions workflow uses OIDC federation to push images to ACR, so no registry password is stored anywhere. JWT tokens for API access use HS256 with a 60-minute expiry.