Use this checklist when a service is ready to move from development traffic to production traffic through R9S.
Before rollout
- The selected model appears in
GET /models. - The application sends requests through
https://gw.r9s.ai/v1. - Runtime API keys are stored on the server side.
- Streaming behavior has been tested if the product surface is interactive.
- Tool calls, JSON mode, vision input, or audio output have been tested with the exact model family you plan to use.
- Monitoring shows request volume, latency, error rate, token usage, and cost.
Traffic phases
Roll out in phases so you can compare quality, cost, and latency against your baseline:
| Phase | Traffic | Goal |
|---|---|---|
| Smoke test | Internal only | Verify authentication, response shape, and logging |
| Canary | 1% to 5% | Compare latency, errors, and cost |
| Ramp | 10% to 50% | Watch quality feedback and retry behavior |
| Primary | 100% | Keep dashboards and alerts active |
Rollback triggers
Define rollback triggers before ramping traffic:
- Error rate exceeds the service budget.
- Time to first token regresses for interactive flows.
- Token spend or cost per request exceeds the expected envelope.
- Output format compatibility breaks downstream parsing.
- A fallback route changes quality more than the product can tolerate.
Rollback
Keep rollback changes outside application code where possible. The application should continue to target the same R9S base URL while route, provider, model, or fallback changes are handled at the gateway layer.
After rollback, keep the failed route available for diagnosis but stop production traffic until the root cause is understood.
Launch notes
Record these details with each production launch:
- Date and owner.
- Model name and endpoint.
- Expected traffic volume.
- Known fallbacks.
- Retry policy.
- Budget or quota limit.
- Dashboard and log query links.
- Rollback owner and decision threshold.