Best Practices

Practical guidance for secure, reliable, and cost-aware R9S gateway integrations.

Use these practices when running R9S in production applications, internal tools, and agent workflows.

Secure keys

  • Store API keys in server-side secret stores or environment variables.
  • Never hard-code keys in client-side code.
  • Use separate keys for applications, jobs, agents, and reporting tools.
  • Rotate keys regularly.
  • Revoke lost or unused keys.

Keep request compatibility explicit

  • Validate each model’s support for streaming, tools, JSON schema, vision, audio, and reasoning controls.
  • Avoid assuming that two model families support the same optional fields.
  • Test fallback routes with the same payload shape used in production.
  • Keep downstream parsers tolerant of harmless provider differences, but strict about required business fields.

Use streaming where it matters

Streaming improves perceived latency for chat, coding, and agent experiences. Use non-streaming responses when you need:

  • Strict full-response validation.
  • Batch processing.
  • Simpler retry behavior.
  • Features that require non-streaming mode.

Retry carefully

Transient 429 and 5xx errors can be retried, but every retry can add cost and user-visible delay.

Recommended defaults:

  • Exponential backoff.
  • Jitter.
  • A small maximum attempt count.
  • Per-request timeout.
  • Request logging with attempt count.

Do not retry unsafe tool side effects unless your application provides idempotency keys or deduplication.

Control cost

  • Track cost by API key, model, endpoint, and app attribution.
  • Set budgets before production rollout.
  • Use lower-cost models for classification, extraction, routing, and drafts when quality allows.
  • Cache repeated prompts or retrieval results in your application layer.
  • Review cached-token metrics when a model supports caching.

Observe every launch

Before a new route reaches primary traffic, confirm dashboards and logs show:

  • Request volume.
  • Latency and time to first token.
  • Error rate.
  • Retry count.
  • Token usage.
  • Spend.
  • Model and channel identifiers.

Keep request IDs available in support tickets and incident notes.

Attribute application traffic

For app traffic, pass:

HTTP-Referer: https://example.com
X-Title: Example App

These headers help connect usage, cost, and ecosystem reporting to the product that generated the traffic.

Plan rollback

Every production route should have a rollback decision before launch:

  • Which metric triggers rollback.
  • Who can approve rollback.
  • Which model or route is the previous known-good path.
  • How long the team will observe after rollback.

Keep the application pointed at R9S while gateway routes and provider choices change behind the stable endpoint.