I’ve been running agentgateway for a couple of days as the main gateway in my cluster for AI requests and MCP traffic. I previously used OpenRouter as the upstream backend.

Then I came across DigitalOcean Serverless Inference. It looked interesting enough to try, so I wanted to add it as a second upstream and make it the one actually serving traffic.

agentgateway ships with named provider shapes for OpenAI, Anthropic, Gemini, Bedrock, and a few others. DigitalOcean is not one of them. What it does offer is an OpenAI-compatible chat-completions API at https://inference.do-ai.run/v1/chat/completions. We can treat it like any other OpenAI-shaped upstream and redirect host, port, and path to DigitalOcean’s endpoint instead of api.openai.com.

Use the OpenAI provider shape

On AgentgatewayBackend, spec.ai.provider accepts exactly one provider block: openai, azureopenai, azure, anthropic, gemini, vertexai, bedrock, or custom. There is no digitalocean variant, and we do not need one here.

DigitalOcean did not invent its own request format. It adopted OpenAI’s. That means the generic openai shape is enough, with host, port, and path pointing the request at inference.do-ai.run instead of api.openai.com.

There is also a custom provider for APIs that do not match an existing dialect. I skipped it here.

Create the API key Secret

policies.auth.secretRef reads the Authorization key from a Secret in the same namespace and forwards it as the outbound header. Store the full value, Bearer <token>, not just the bare token.

The token is a DigitalOcean model access key from the control panel under Inference / GenAI.

kubectl create secret generic digitalocean-api-key \
  -n agentgateway-system \
  --from-literal=Authorization="Bearer <your-do-model-access-key>"

Define the AgentgatewayBackend

host, port, and path sit next to openai under provider. They change where the OpenAI-shaped request goes without changing the request itself. I left model unset under openai: so the model field from each inbound request passes through. Any model from DigitalOcean’s model list stays reachable without editing this resource again.

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: digitalocean
  namespace: agentgateway-system
spec:
  ai:
    provider:
      openai: {}
      host: inference.do-ai.run
      port: 443
      path: /v1/chat/completions
  policies:
    auth:
      secretRef:
        name: digitalocean-api-key
    tls:
      sni: inference.do-ai.run

Set policies.tls.sni to inference.do-ai.run as well. Without it, the upstream TLS handshake fails even when host and path look correct. I hit that before adding sni to the manifest.

kubectl apply -f backend.yaml
kubectl get agentgatewaybackend digitalocean -n agentgateway-system
NAME           ACCEPTED   AGE
digitalocean   True       2s

Update the HTTPRoute

The HTTPRoute that already matched /v1 and pointed at the old backend is the one to edit. Keep the same metadata.name and namespace when you apply the update so Kubernetes replaces backendRefs in place instead of creating a second route. Substitute <your-route-name> below with whatever you already use.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: <your-route-name>
  namespace: agentgateway-system
spec:
  parentRefs:
  - name: agentgateway-proxy
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /v1
    backendRefs:
    - group: agentgateway.dev
      kind: AgentgatewayBackend
      name: digitalocean

That one backendRef change moves every client hitting /v1/chat/completions from the old backend to DigitalOcean across the cluster.

The previous AgentgatewayBackend is still there. Nothing points at it anymore. Switching back is a one-line revert in the HTTPRoute without recreating the backend.

Verify it works

Accepted status on the AgentgatewayBackend and HTTPRoute only means the gateway liked the config. It does not prove a request actually reaches DigitalOcean.

I checked with a kubectl port-forward to the proxy Service:

kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80 &

curl -s -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"<model-name>","messages":[{"role":"user","content":"Say OK if you can hear me."}]}'
{
  "model": "<model-name>",
  "usage": { "prompt_tokens": 96, "completion_tokens": 127, "total_tokens": 223 },
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "OK! I can hear you. How can I help you today?"
    },
    "finish_reason": "stop"
  }]
}

Swap model for any ID from DigitalOcean’s model list. I tried a few different ones with the same curl and got completions back each time.