Skip to main content

Configure the AI Gateway

Enable the AI Gateway through the platform chart, then apply its custom resources in the order described below. See AI Gateway for ongoing configuration.

Prerequisites​

  • A corporate identity provider, configured once at global.stacklok.primaryIdp. The gateway ties every request to an identity from that provider. See Configure identity.
  • PostgreSQL, which the platform already requires. Budgets, pricing, and recorded spend live there.
  • Redis or Valkey, only if you intend to enable the detection result cache. It is optional and off by default. See PCI/PII controls.

Enable it​

Set the install toggle in your platform values and upgrade:

values.yaml
global:
  stacklok:
    aiGateway:
      enabled: true

This installs the AI Gateway operator and custom resource definitions. Apply an AIGateway resource to create a gateway instance.

Bring it up in this order​

Complete the following sequence before sending production traffic:

  1. Apply an AIGateway resource with at least one provider and one route. See Connect model providers.

  2. Create an AIPolicy that targets it. The operator adds budget enforcement only to gateways with a matching policy. The policy can omit screening controls when you only need budget enforcement.

  3. Create budgets for every user or group that will send traffic, before you point the gateway at the budget service. A caller with no applicable budget is refused. See Budgets and pricing.

  4. Verify. Confirm the gateway reports its providers ready and that budget enforcement probed successfully:

    kubectl get aigw -n <NAMESPACE>
    kubectl get aigw <NAME> -n <NAMESPACE> \
      -o jsonpath='{.status.webhooks}' | jq .

Content screening posture​

Detection failures deny requests by default. An experimental waiver can allow traffic during a rollout or incident, but it is unavailable on the stable release channel.

Next steps​