LiteLLM Helm Chart

Deploys the LiteLLM proxy: an OpenAI-compatible gateway in front of every model provider, plus the management API and the Admin UI. One Deployment serves all three.

This chart is a clean-room replacement for the two upstream charts (helm/litellm-helm and helm/litellm). It carries neither as a dependency.

The reason it exists is routing. Both upstream charts publish through a networking.k8s.io/v1 Ingress and nothing else, so a cluster running Envoy Gateway, Contour, or OpenShift has to bolt its own object on afterwards, usually as a hand-written manifest outside the release. Here each implementation is a first-class, schema-validated template.

Implementation API Typical controller
Ingress networking.k8s.io/v1 ingress-nginx, HAProxy, Traefik
HTTPRoute gateway.networking.k8s.io/v1 Envoy Gateway, Contour, Istio, NGINX Gateway Fabric
HTTPProxy projectcontour.io/v1 Contour
Route route.openshift.io/v1 OpenShift

More than one may be enabled at a time, which is how you move from Ingress to Gateway API without a gap in service.

Contents

Prerequisites

  • Kubernetes 1.25 or later
  • Helm 3.19 or later
  • The CRDs for whichever routing implementation you enable
  • PostgreSQL 14 or later, for the database flow only
  • Redis 7.0 or later, for more than one replica — or valkey.enabled, which deploys one
  • The Altinity ClickHouse operator and cert-manager, for langfuse.deploy.enabled only

This chart deploys no database. Both upstream charts bundle the Bitnami subcharts, whose versioned tags were withdrawn from docker.io/bitnami and now need a bitnamilegacy repoint to install at all. A proxy that stores virtual keys and spend data belongs in front of a database you back up and patch, so this chart points at one you already run instead.

Two optional subcharts exist, both off by default, and neither is Bitnami:

Dependency Chart Switched on by What it is for
Valkey valkey-io/valkey-helm valkey.enabled The response cache and the coordination store
Langfuse langfuse/langfuse-k8s langfuse.deploy.enabled Tracing, where nothing else runs Langfuse yet

A conditional dependency whose condition is false renders nothing at all, so a release that leaves both off is byte-identical to one built before they existed. They are still fetched into charts/ by helm dependency build, because Helm refuses to render a chart whose declared dependencies are missing from the directory regardless of condition — which is why CI builds dependencies before it lints.

Install

From a source checkout:

helm upgrade --install litellm helm-charts/litellm \
  --namespace litellm \
  --create-namespace \
  --values my-values.yaml

From GHCR:

helm upgrade --install litellm \
  oci://ghcr.io/sassoftware/sas-retrieval-agent-manager-examples/charts/litellm \
  --namespace litellm \
  --create-namespace \
  --values my-values.yaml

Omit --version to install the newest published chart. Add --version <x.y.z> to pin an exact version.

The default values install a working proxy with one mock model, so a bare helm install starts and passes its health probes with nothing else configured. Replace models before you send real traffic.

Reach it without routing:

kubectl -n litellm port-forward svc/litellm 4000

The OpenAI-compatible API is then at http://localhost:4000, and the Admin UI at http://localhost:4000/ui.

Versions

appVersion in Chart.yaml is the upstream LiteLLM release this chart targets, and it is the default container tag. image.tag is empty by default and falls back to it, so one edit moves the proxy and the migration Job together. Nothing can pin them apart: both resolve the image through the same template helper.

Set image.tag only to run a release other than the one the chart targets.

The chart rejects latest and main-latest. The proxy and the migration Job resolve the tag independently at pull time, so a tag that moves between the Job finishing and a pod restarting runs a different proxy against the schema that Job just migrated. main-stable, LiteLLM’s documented floating production tag, is allowed.

version in Chart.yaml is the chart’s own version and is managed by CI, which resolves the next free patch from the registry on every publish.

Model management

modelManagement.mode decides where the model catalog lives. It is the single most consequential value in this chart, and it is worth choosing deliberately. Everything else works the same either way.

  config database
Catalog lives in config.yaml, from models in your values PostgreSQL
Adding a model edit values, helm upgrade, pods roll Admin UI or POST /model/new, no redeploy
Reproducible from Git yes no — the database is the source of truth
Needs PostgreSQL no yes
Migration Job not rendered runs prisma migrate deploy before the proxy
Virtual keys, teams, budgets unavailable available
Spend tracking unavailable available
Audit of who changed what Git history the database

Those last three rows usually decide it. Virtual keys and spend tracking have no config.yaml representation, so any deployment that hands out per-team keys or bills usage back needs database.

Config flow 

modelManagement:
  mode: config

models:
  - model_name: gpt-4o
    litellm_params:
      model: azure/gpt-4o
      api_base: https://example.openai.azure.com/
      api_key: os.environ/AZURE_OPENAI_API_KEY
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

environmentSecrets:
  - litellm-provider-keys

models is appended to proxyConfig.model_list and rendered into a ConfigMap. The pod template carries a checksum of that ConfigMap, so helm upgrade restarts the proxy whenever the catalog changes. Without the checksum the new file would sit in the ConfigMap unread until an unrelated change happened to roll the pods.

Everything that is not a model goes in proxyConfig, which is rendered verbatim:

proxyConfig:
  litellm_settings:
    drop_params: true
    request_timeout: 600
    callbacks: ["prometheus"]
  router_settings:
    routing_strategy: simple-shuffle
  general_settings:
    proxy_batch_write_at: 60

To manage config.yaml entirely outside this chart, set config.existingConfigMap.name. models and proxyConfig are then ignored, and no ConfigMap is rendered.

Database flow 

modelManagement:
  mode: database

database:
  host: litellm-pg.postgres.database.azure.com
  name: litellm
  sslMode: verify-full
  existingSecret:
    name: litellm-db        # keys: username, password

redis:
  host: litellm-redis.example.com
  existingSecret:
    name: litellm-redis     # key: password

The chart sets general_settings.store_model_in_db, renders the migration Job, and starts the proxy with DISABLE_SCHEMA_UPDATE=true so that N replicas do not race one database on every rollout.

config.yaml is still mounted and still owns everything that is not a model: callbacks, router settings, guardrails, and the rest of general_settings. Only the catalog moves.

Set modelManagement.seedFromConfig: true to also render models into config.yaml as an always-present floor, for example a house model that must exist even if someone empties the database. Config entries are read-only in the UI.

Turning the database on without moving the catalog

database.enabled is independent of modelManagement.mode. Setting it while staying in config mode gives you virtual keys, teams, budgets, and spend tracking, while the catalog stays in Git:

modelManagement:
  mode: config
database:
  enabled: true
  host: litellm-pg.example.com
  existingSecret:
    name: litellm-db

This is a good middle ground for a platform team that wants reviewable model changes and per-team keys at the same time.

Switching flows later

config to database is additive: point database.* at a server, flip the mode, upgrade. The migration Job creates the schema, and the catalog you already had keeps loading if you set seedFromConfig.

database to config drops every runtime-created model, key, team, and spend record from the serving path. Export anything you need first; the chart cannot migrate it for you.

Provider credentials

Never put a literal API key in models or proxyConfig. Both are rendered into a ConfigMap, which is world-readable to anything with get configmap in the namespace, and both end up in the Helm release secret in plain text.

Put keys in a Secret, list it under environmentSecrets, and reference them as os.environ/<KEY>:

kubectl -n litellm create secret generic litellm-provider-keys \
  --from-literal=AZURE_OPENAI_API_KEY=... \
  --from-literal=ANTHROPIC_API_KEY=...

environmentSecrets:
  - litellm-provider-keys
models:
  - model_name: gpt-4o
    litellm_params:
      model: azure/gpt-4o
      api_key: os.environ/AZURE_OPENAI_API_KEY

Every key in a listed Secret becomes an environment variable in the pod, so the names in the Secret are exactly what os.environ/ resolves against. environmentConfigMaps works the same way for values that are not secret.

In the database flow, credentials added through the Admin UI are encrypted in the database with LITELLM_SALT_KEY. Set it once through environmentSecrets and never change it: it encrypts stored provider keys, and changing it makes them unreadable.

Routing

Shared settings live under routing; each implementation adds its own block. routing.hosts, routing.path, routing.tls, routing.annotations, and routing.labels apply to all of them.

The proxy serves the OpenAI API, the management API, and the UI from one port, so a single path prefix covers everything and no path-based dispatch is needed.

routing.path defaults to /litellm and must match serverRootPath, which is the prefix the proxy actually serves under — see Sharing a hostname.

Forwarded headers

TLS terminates at the ingress, so the request arriving at the pod is plaintext. Left to itself the proxy takes that at face value and emits http:// URLs — the trailing-slash redirect on /ui sends the browser to a plaintext address, downgrading a release this chart published over HTTPS.

uvicorn already parses the X-Forwarded-* headers that say otherwise, but ignores them unless the sender is trusted, and its default is 127.0.0.1 — which in Kubernetes is never the ingress. trustedProxies supplies that list, and defaults to *:

trustedProxies: ["*"]

* trusts any client that can reach the pod. The Service is ClusterIP, so that means in-cluster callers rather than the internet, and the only header they could forge is consulted for the client IP in logs and for LiteLLM’s optional allowed_ips. On a cluster without NetworkPolicy, narrow it to whatever fronts the release:

trustedProxies: ["10.244.0.0/16"]   # pod CIDR

CIDR notation needs uvicorn 0.30 or newer, which the chart’s pinned image carries. Setting trustedProxies: [] leaves the variable unset and restores uvicorn’s own default.

Ingress 

routing:
  hosts: [litellm.example.com]
  tls:
    enabled: true
    secretName: litellm-tls
  ingress:
    enabled: true
    className: nginx
    annotations:
      nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
      nginx.ingress.kubernetes.io/proxy-body-size: "32m"

ingress-nginx rejects a dot in an Exact or Prefix path through its admission webhook, and serves ImplementationSpecific as a plain prefix anyway. The chart therefore downgrades a dotted path to ImplementationSpecific when className is nginx, so a path such as /v1.0 is accepted instead of being rejected at apply time.

Raise the read timeout. A streaming completion holds the connection open far longer than the 60-second default, and the symptom of leaving it alone is truncated responses under load rather than an obvious error.

routing.ingress.extraPaths adds further paths to each host, for example to send one prefix to a different Service.

HTTPRoute (Gateway API) 

routing:
  hosts: [litellm.example.com]
  httpRoute:
    enabled: true
    parentRefs:
      - name: eg
        namespace: envoy-gateway-system
        sectionName: https
    timeouts:
      backendRequest: 0s

routing.tls is ignored here on purpose. In Gateway API the certificate belongs to the Gateway’s listener, not to the route attached to it. Put the Secret in the listener’s tls.certificateRefs; a chart that also rendered it here would silently do nothing.

timeouts.backendRequest: 0s disables the request timeout. Without it a long streaming completion is cut off at the implementation’s default, which for Envoy Gateway is 15 seconds.

filters and extraRules are passed through untouched, so a URLRewrite, a RequestHeaderModifier, or an extra rule for a second hostname all work without the chart knowing about them.

HTTPProxy (Contour) 

routing:
  hosts: [litellm.example.com]
  tls:
    enabled: true
    secretName: litellm-tls
  httpProxy:
    enabled: true
    ingressClassName: contour
    timeoutPolicy:
      response: infinity
      idle: 600s
    retryPolicy:
      count: 2
      retryOn: ["5xx"]

Prefer this over a plain Ingress on Contour. The retry policy, response timeout, and load balancer policy become schema-validated fields the API server checks, instead of controller-specific annotations that nothing validates and that fail silently when misspelled.

timeoutPolicy.response: infinity is the Contour equivalent of the timeout note above.

One HTTPProxy carries one virtual host. The chart fails the render rather than silently publishing only the first entry when routing.hosts has more than one.

Sharing a hostname another release already owns

Contour allows one root HTTPProxy per FQDN, and a root is any proxy that declares a virtualhost. When another application already owns the hostname this release has to answer on, a second root is rejected — and it is undefined which of the two loses, so the attempt risks taking the incumbent down.

The way through is delegation: that application includes this one under a path prefix, and this release renders a child proxy with no virtualhost of its own.

In the owning release’s root HTTPProxy:

includes:
  - name: litellm
    namespace: litellm
    conditions:
      - prefix: /litellm

And here:

serverRootPath: /litellm      # must match the delegated prefix
routing:
  path: /litellm
  httpProxy:
    enabled: true
    delegated: true

routing.hosts and routing.tls are not needed and are ignored: the child inherits both from the parent, so it needs no certificate of its own. TLS passthrough is rejected, since it is a property of a virtualhost the child does not have.

The generated route matches /, not routing.path. Contour concatenates the parent’s condition with the child’s, so repeating the prefix would publish /litellm/litellm. serverRootPath must still match the delegated prefix, because that is what the proxy itself serves under.

A child is reachable only while the parent includes it. Delete or reconfigure the parent and Contour reports the child as orphaned and ignores it — the release keeps running and serves nothing. kubectl get httpproxy <name> -o jsonpath='{.status.description}' is what tells you.

Delegation also makes the mount root useful. The proxy answers its own root with a plain "LiteLLM: RUNNING" string, so a browser opening /litellm lands on nothing helpful; requestRedirectPolicy on an exact match sends it to the Admin UI instead:

routing:
  httpProxy:
    extraRoutes:
      - conditions:
          - exact: /
        requestRedirectPolicy:
          path: /litellm/ui/
          statusCode: 302

An exact match is what keeps this to the bare prefix — a prefix condition would redirect /litellm/v1/... as well. This works for a root HTTPProxy too; it is only unavailable on a plain Ingress, where Contour ignores the redirect annotations.

Route (OpenShift) 

routing:
  hosts: [litellm.example.com]   # omit to let OpenShift generate one
  tls:
    enabled: true
  route:
    enabled: true
    tls:
      termination: edge
      insecureEdgeTerminationPolicy: Redirect

Leave routing.hosts empty and the router generates a hostname from the route name, the namespace, and the cluster’s apps domain.

A passthrough Route cannot carry a path, so the chart omits spec.path when termination is passthrough and sends the whole host to the proxy.

Migrating between implementations

Enable both, cut traffic over at DNS, then disable the old one:

routing:
  hosts: [litellm.example.com]
  ingress:
    enabled: true          # still serving
    className: nginx
  httpRoute:
    enabled: true          # new path, warming up
    parentRefs:
      - name: eg

Both objects point at the same Service, so neither takes the proxy down.

Sharing a hostname

At the domain root LiteLLM claims /ui, /v1, /docs, /health, /metrics and more. That is an awkward neighbour for an application that owns the site, and it is the usual reason to deploy this chart as an add-on rather than as the main tenant.

The chart therefore ships mounted under /litellm, via serverRootPath:

serverRootPath: /litellm
routing:
  path: /litellm

The Admin UI is at /litellm/ui and the OpenAI-compatible base URL is https://<host>/litellm/v1. Nothing outside the prefix is served, so every root path stays available to the other application.

To claim the domain root instead, clear both:

serverRootPath: ""
routing:
  path: /

Point clients at the prefix. With the default, OPENAI_BASE_URL is https://<host>/litellm/v1; a client configured for https://<host>/v1 gets a 404 from the proxy rather than a connection error, which tends to read as a missing model.

API docs

LiteLLM serves the Swagger UI at its mount root — upstream’s DOCS_URL defaults to / — so opening the release in a browser lands on the API reference rather than the Admin UI. This chart moves it to /docs:

docs:
  enabled: true
  path: /docs
  URL with the chart defaults
Swagger UI https://<host>/litellm/docs
Admin UI https://<host>/litellm/ui

Set path: / to restore upstream’s behaviour, or enabled: false to drop the docs page and the OpenAPI schema together — both, because Swagger reads the schema, and removing one without the other leaves a page that renders an error. A path under /ui fails the render, since it would be shadowed by the Admin UI.

With the docs moved, the mount root itself returns LiteLLM’s plain "LiteLLM: RUNNING" health string. The proxy has no route that redirects its root to the Admin UI, so link people to /litellm/ui directly.

The health endpoints keep answering at the root as well, which is why the probes in this chart are not prefixed. Verified against the image: with SERVER_ROOT_PATH set, /health/readiness and /litellm/health/readiness both return 200 while /ui returns 404 and /litellm/ui returns 200. The UI’s own asset URLs are rewritten by the image at runtime.

The chart fails the render if routing.path does not match the prefix, because publishing / anyway would defeat the point, and publishing a different prefix would serve 404s.

This changes the client-facing API base URL, so existing callers need updating.

Single sign-on

Signs in to the Admin UI with an OpenID Connect provider instead of the shared master key, through LiteLLM’s generic OIDC support.

SSO is a LiteLLM enterprise feature. Since v1.76.0 it is free for up to five users; past that it needs a licence in license.existingSecret.

sso:
  enabled: true
  clientId: litellm
  authorizationEndpoint: https://idp.example.com/realms/r/protocol/openid-connect/auth
  tokenEndpoint: https://idp.example.com/realms/r/protocol/openid-connect/token
  userinfoEndpoint: https://idp.example.com/realms/r/protocol/openid-connect/userinfo

The client secret is generated on first install and reused on every later upgrade, so upgrading never invalidates the provider-side registration. Supply your own with sso.clientSecret.existingSecret.

sso.proxyBaseUrl is derived from routing.hosts[0], routing.tls.enabled and routing.path when left empty, including any serverRootPath prefix. Deriving it keeps the redirect URI from drifting out of step with the Ingress. The provider must accept <proxyBaseUrl>/sso/callback.

Why the three endpoints are separate

They are not called by the same party, and they are frequently not interchangeable.

Setting Called by Must be reachable from
authorizationEndpoint the browser the user’s machine
tokenEndpoint the proxy pod inside the cluster
userinfoEndpoint the proxy pod inside the cluster

A provider published through a cloud load balancer is often unreachable from inside the same cluster, because the load balancer does not hairpin — yet it still advertises its public URL in its discovery document. Copying all three from that document produces a sign-in that redirects correctly and then fails at the token exchange. Point the browser-facing one at the public address and the other two at the in-cluster Service.

Keycloak bootstrap

sso.keycloakBootstrap adds a one-shot Job that configures the Keycloak side to match:

sso:
  keycloakBootstrap:
    enabled: true
    url: http://keycloak.identity.svc.cluster.local:8080
    realm: myrealm
    admin:
      existingSecret:
        name: keycloak-admin     # keys: username, password

It creates, in an existing realm:

  1. A confidential OIDC client with the right redirect URIs, including LiteLLM’s /sso/debug/callback claim-dumping route.
  2. Two client roles carrying LiteLLM’s own role names — proxy_admin and internal_user by default.
  3. Two groups, litellm-admins and litellm-users, granted those roles.
  4. A protocol mapper putting the client roles into a flat roles claim.
  5. Optionally an llmadmin account in the admin group.

Step 4 is the one that is easy to miss. Keycloak nests client roles under resource_access.<client>.roles, which LiteLLM does not read, so without the mapper every user signs in carrying no role at all.

Client roles rather than realm roles, and dedicated groups rather than existing ones, so nothing this chart creates appears at realm scope or changes another application’s authorization. Point groups at existing group names if you would rather reuse them.

Every step reads before it writes, so the Job is safe to re-run and repairs a half-applied earlier run rather than duplicating it. It never creates realms.

Realms usually carry a password policy. The generated llmadmin password ends in a fixed Aa1! so it satisfies the common upperCase(1) lowerCase(1) digits(1) specialChars(1) rules by construction — relying on chance would fail roughly one deploy in four hundred. If your policy is stricter, set sso.keycloakBootstrap.adminUser.password.value; the Job names the policy in its error output when Keycloak rejects one.

Read the account password with:

kubectl -n litellm get secret litellm-sso \
  -o jsonpath='{.data.admin-user-password}' | base64 -d

Database

The chart reads database credentials from a Secret only and accepts no inline password.

It emits the discrete DATABASE_HOST, DATABASE_PORT, DATABASE_USERNAME, DATABASE_PASSWORD, and DATABASE_NAME variables and lets the proxy assemble and percent-encode the URL. This matters: a password containing @, /, or ? corrupts a hand-assembled connection string, and the resulting failure is an opaque Prisma connect error rather than anything that names the password as the cause.

kubectl -n litellm create secret generic litellm-db \
  --from-literal=username=litellm \
  --from-literal=password='<password>'

database:
  enabled: true
  host: litellm-pg.example.com
  port: 5432
  name: litellm
  sslMode: verify-full
  existingSecret:
    name: litellm-db
    usernameKey: username
    passwordKey: password

Set database.existingSecret.urlKey instead to supply a whole connection URL from one Secret key, for parameters the discrete fields cannot express. The discrete host, port, and name are then not emitted.

Workload identity

For a managed database with IAM authentication, no password is read at all. The proxy mints a token at startup.

database:
  auth:
    mode: azureEntra     # or awsIam
serviceAccount:
  create: true
  annotations:
    azure.workload.identity/client-id: <client-id>

The Secret is still required for the username. migrationJob.serviceAccountName gives the Job its own identity when it needs one distinct from the proxy’s.

TLS to the database

sslMode: verify-full with a CA bundle is what a managed PostgreSQL service expects. Mount the bundle and point at it:

database:
  sslMode: verify-full
  sslRootCert: /certs/ca.pem
volumes:
  - name: db-ca
    configMap:
      name: rds-ca
volumeMounts:
  - name: db-ca
    mountPath: /certs
    readOnly: true

volumes and volumeMounts are applied to the migration Job as well as the proxy, so migrations verify the certificate on the same terms.

Connection budget

Each worker opens its own pool, so a pod holds numWorkers × connection_limit upstream connections and the total scales with maxReplicas, not with the replicas running today. An autoscaler at the default maxReplicas: 100 can ask for far more connections than a stock PostgreSQL accepts.

Either size autoscaling.maxReplicas from what the database can serve, or turn on the in-pod pooler:

database:
  connectionPool:
    enabled: true
    maxDbConnections: 20
    maxClientConn: 1000

With the pooler a pod holds at most maxDbConnections upstream connections however many workers run. Set database.disablePreparedStatements: true in front of an external transaction-mode pooler such as PgBouncer or RDS Proxy.

Read replica 

database:
  readReplica:
    enabled: true
    host: litellm-pg-reader.example.com

Read-only queries go to the replica and writes continue to the writer. Fields left empty fall back to the writer’s values, including the credentials Secret.

Creating the database

Everything above assumes the database, the login role, and the schema already exist — the usual case, because a DBA provisioned them. When they do not, database.init adds a one-shot Job that creates them.

It is off by default. Turning it on does not change how the proxy connects; it only adds a step in front.

database:
  host: litellm-pg.example.com
  name: litellm
  schema: litellm
  existingSecret:
    name: litellm-db        # keys: username, password

  init:
    enabled: true
    admin:
      existingSecret:
        name: litellm-pg-admin   # keys: username, password

Which credential is which

There are two, and conflating them is the easy mistake:

  Used by Lifetime
database.existingSecret the proxy, every day permanent
database.init.admin this Job, once erased when the Job succeeds

database.existingSecret is the input to the Job, not something it reads back. The Job creates a login role with that username and password, makes it the owner of database.name and database.schema, and exits. The proxy then authenticates as that role. One Secret, written once by you, used by both.

The admin credential is separate because creating a database and a role needs privileges the proxy must never hold.

What it runs

Against the maintenance database named by admin.database (postgres by default, because CREATE DATABASE cannot run inside the database being created):

  1. CREATE ROLE for the service user, or ALTER ROLE to reset its password to match the Secret.

  2. CREATE DATABASE owned by that role.

    Then, connected to the new database:

  3. GRANT ALL PRIVILEGES ON DATABASE to the role.
  4. CREATE SCHEMA ... AUTHORIZATION the role, and GRANT ALL ON SCHEMA.
  5. ALTER ROLE ... SET search_path, so an interactive psql session as that role lands in the right schema.

Each step is skipped when the object already exists, so the Job is safe to re-run. Step 4’s GRANT runs even on an existing schema: PostgreSQL 15 stopped granting CREATE on public to everyone, and without it prisma migrate deploy fails with permission denied for schema.

Turn individual steps off with createDatabase, createUser, and createSchema when a DBA has already done part of the work.

Password rotation

The ALTER ROLE in step 1 is unconditional, so changing the password in database.existingSecret and upgrading resets the role to match. Without that, rotating the Secret would leave the proxy unable to log in until someone fixed the role by hand.

Erasing the admin credentials

With inline admin.username / admin.password, the chart renders a Secret, the Job uses it, and a second container erases both keys from it the moment the Job succeeds.

The ordering is structural rather than best-effort. The psql step is an init container and the cleanup is the only ordinary container, so Kubernetes will not start the cleanup until the init container has exited 0. A failed initialization therefore never reaches the erase step, and the credentials survive for the retry.

The erase is a JSON merge patch setting both keys to null. That removes them whether or not they are still there, which makes it idempotent with no conditional logic — and that in turn is what lets it invoke kubectl directly, with no shell. The upstream registry.k8s.io/kubectl image is distroless and has no /bin/sh, so a shell script here would not run at all.

The Secret object itself is kept, with its data keys gone. The Role carries get and patch on that one Secret name and deliberately no delete, so this chart never creates a Role that could delete Secrets in your namespace.

Erasing the Kubernetes Secret does not remove an inline password from the Helm release secret, which helm get values can read. It is also restored for the length of the hook window on every upgrade, because re-running the Job needs it.

To keep the admin password out of the release entirely, create the Secret yourself and name it under admin.existingSecret. The chart then modifies nothing — it only destroys what it created — and rotating that Secret afterwards is yours to do.

The cleanest sequence for a one-time bootstrap is therefore: install with init.enabled: true, confirm the Job succeeded, then set init.enabled: false for every later upgrade.

Ordering

The Job is a Helm hook at weight -5, ahead of the migration Job at 1. The chart rejects a weight that is not below the migration’s, because the failure it prevents surfaces as a confusing Prisma error about a database that does not exist rather than as a misconfigured value.

Under a GitOps controller, turn hooks.helm.enabled off and hooks.argocd.enabled on; the sync waves keep the same order.

The wave is the hook weight. They express the same thing in two dialects, so the chart does not ask for both — there is no syncWave value to keep in step with the weight beside it.

A wave is only meaningful relative to everything else in the same ArgoCD Application, which the chart cannot see: how the surrounding Applications are numbered, or whether the database is provisioned by this chart at all. Where the defaults do not fit, set the annotation directly — it is applied after the chart’s own and so takes precedence:

migrationJob:
  annotations:
    argocd.argoproj.io/sync-wave: "12"

Ordinary failures

  • password authentication failed for the admin user: admin.username is not a login that may CREATE DATABASE and CREATE ROLE. On a managed service this is the server administrator, not postgres.
  • The Job waits and then times out: the server is not reachable from the cluster. waitTimeout bounds this at 300 seconds.
  • permission denied to create database: the admin user exists but lacks CREATEDB.

Read what it actually did:

kubectl -n litellm logs job/litellm-db-init -c db-init

Redis

Redis is the proxy’s coordination store: cross-pod rate limits, spend tracking, and the pod lock manager that elects a single owner for shared background jobs.

redis:
  host: litellm-redis.example.com
  port: 6379
  existingSecret:
    name: litellm-redis
    passwordKey: password

The chart wires REDIS_HOST, REDIS_PORT, and REDIS_PASSWORD, and renders a general_settings.coordination_redis block. A coordination_redis block you write in proxyConfig always wins. Set redis.coordination: false to wire only the environment variables, for example when Redis is there for response caching alone.

Without Redis, rate limits and spend counters are per-pod. A tpm budget is then enforced at roughly replicaCount times what you configured, and budget resets run once per pod instead of once. The chart prints this warning on install whenever replicaCount is above one or autoscaling is on and the release has no Redis.

For Sentinel, set redis.sentinel.enabled and redis.sentinel.masterSet; the coordination block is then rendered with sentinel_nodes and service_name instead of host and port.

Letting the chart deploy one

valkey.enabled: true deploys a Valkey — the Redis fork, same wire protocol — through the Valkey project’s own chart, and points everything above at it:

valkey:
  enabled: true
cache:
  enabled: true

That is the whole configuration. The chart derives, from the subchart’s own values so the two cannot drift:

Variable Derived from
REDIS_HOST the Valkey primary Service, fully qualified with valkey.clusterDomain
REDIS_PORT valkey.service.port
REDIS_PASSWORD the Secret the subchart reads the default user’s password from

valkey.nameOverride and valkey.fullnameOverride are honoured, because the address is built with the same logic the subchart names its Service with rather than by assuming <release>-valkey.

redis.host always wins. Set it and the bundled Valkey is not consulted for the address, even if it is also deployed — a subchart default must never be able to repoint a release at a different server than the one someone typed. That is also how you put the bundled Valkey behind an explicit name.

Every other key under valkey is passed through to the subchart untouched; see its values reference. This chart’s schema accepts them without listing them, which is the one place it is not strict about unknown keys: the subchart’s value surface is not ours to enumerate.

The subchart ships with authentication off, so any pod that can reach the Service can read and write the cache. On a shared cluster, turn it on:

valkey:
  enabled: true
  auth:
    enabled: true
    usersExistingSecret: litellm-valkey-acl   # key `default`
    aclUsers:
      default:
        permissions: "~* &* +@all"

With usersExistingSecret the password is yours to create and rotate, and the chart reads the default user’s key from it. With an inline aclUsers.default.password the subchart writes the password into its own <release>-valkey-auth Secret — and into the Helm release secret — and the chart reads default-password from there. Either way the proxy receives a secretKeyRef, never a literal. Enabling auth with only an aclConfig block fails the render: the chart cannot read a password out of an ACL file it does not parse.

What it is and is not for: a response cache is reconstructible by definition, so losing the pod costs latency and provider spend, never data. Spend records and virtual keys are not — those live in PostgreSQL. Treat the bundled Valkey as a cache unless you enable persistence (valkey.replica.persistence) and back it up like anything else.

Sentinel and the bundled Valkey are mutually exclusive, and the chart says so rather than rendering a sentinel_nodes block pointing at a server that has no Sentinel in front of it.

Response cache

Cache completion responses in Redis, so an identical request is answered from the cache instead of being billed and rate-limited by the provider again. The cache is shared by every replica — one pod’s hit serves the next pod’s request, which is the reason to put it in Redis rather than in each worker’s memory.

cache:
  enabled: true
  ttl: 3600
  namespace: litellm-prod
  supportedCallTypes: [acompletion, aembedding]
redis:
  host: redis.example.com
  existingSecret:
    name: litellm-redis

This renders into config.yaml:

litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: os.environ/REDIS_HOST
    port: os.environ/REDIS_PORT
    password: os.environ/REDIS_PASSWORD
    ttl: 3600
    namespace: litellm-prod
    supported_call_types: [acompletion, aembedding]

The address never appears in config.yaml as a literal: the same REDIS_* variables the coordination store uses are referenced, so one Redis setting feeds both and there is one place to change it.

Value Effect
cache.ttl Seconds an entry stays valid. Empty leaves LiteLLM’s default, which never expires entries and relies on Redis eviction
cache.namespace Key prefix, so several releases can share one Redis without reading each other’s entries
cache.supportedCallTypes Which call types are cached. Empty caches every type LiteLLM knows

Embeddings are the usual reason to turn caching on: the same document chunk is embedded identically every time it is re-indexed, and each re-index is billed again without a cache.

What you have written always wins, whole-key. A cache or cache_params block in proxyConfig is left exactly as it is rather than merged into, because a half-merged cache_params would match neither what you asked for nor what this chart documents. That is also the escape hatch for the backends the chart does not model:

cache:
  enabled: false
proxyConfig:
  litellm_settings:
    cache: true
    cache_params:
      type: s3
      s3_bucket_name: litellm-cache

cache.type accepts redis only. It is the one backend the chart can wire end to end, and the one that behaves correctly above a single worker: LiteLLM’s in-memory cache is per worker process, so a two-replica, four-worker deployment would hold eight unrelated caches and hit on roughly one request in eight.

cache.enabled with no Redis at all — neither redis.host nor valkey.enabled — fails the render. A proxy in that state starts happily and then answers every request from the provider while logging connection errors, which is an expensive way to find out.

Check it is working after the first repeated request:

kubectl -n litellm exec deploy/litellm -- curl -s localhost:4000/cache/ping

Migration Job

In the database flow the chart runs prisma migrate deploy in a Job ahead of the proxy, and starts the proxy with DISABLE_SCHEMA_UPDATE=true for as long as that Job owns the schema. Without this, every replica races to push the schema on every rollout.

DISABLE_SCHEMA_UPDATE is emitted last in both workloads, after envVars and extraEnvVars, so a user-supplied value cannot silently shadow it under last-wins duplicate-environment semantics.

A Job’s pod template is immutable, so a stable-named Job cannot be updated in place: the first helm upgrade that changes the image tag or any environment variable would be rejected by the API server. migrationJob.hooks.helm.enabled is on by default and deletes and recreates the Job ahead of each install and upgrade, which is what makes repeated upgrades work.

Under a GitOps controller that runs the Job itself, turn the Helm hook off and migrationJob.hooks.argocd.enabled on. Running both migrates twice.

migrationJob.activeDeadlineSeconds defaults to 1800 and is a wall-clock budget for the whole Job, shared across every retry. Without it a migration blocked on a lock never fails, and a release waiting on the hook never returns.

Secrets

Master key

The master key authenticates admin API calls and is the Admin UI password.

It is generated once as sk-... on first install and stored in <release>-masterkey. Every later upgrade reads the value back out of the live Secret, so upgrading never rotates the key and never invalidates the virtual keys derived from it.

kubectl -n litellm get secret litellm-masterkey \
  -o jsonpath='{.data.masterkey}' | base64 -d

Point masterKey.existingSecret.name at a Secret you manage to take it over. The chart then renders no Secret of its own, which is the right choice when an external secrets operator owns the namespace.

masterKey.value sets it inline. It ends up in the Helm release secret in plain text, so prefer existingSecret outside development.

License and UI login 

license:
  existingSecret:
    name: litellm-license   # key: license
ui:
  credentialsSecret:
    name: litellm-ui        # keys: username, password

Without ui.credentialsSecret the Admin UI is reached with the master key, which means sharing the credential that also authorises every admin API call.

Scaling and availability 

replicaCount: 3
numWorkers: "1"
resources:
  requests:
    cpu: "1"
    memory: 4Gi
  limits:
    cpu: "1"
    memory: 4Gi
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 20
  targetCPUUtilizationPercentage: 60
pdb:
  enabled: true
  maxUnavailable: 1
topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: litellm

resources is unset by default so the chart installs on a small cluster, and so an upgrade never leaves a pod Pending. Production should set it. One worker at database-connected steady state needs roughly 1 CPU and 4Gi, and both scale with numWorkers.

4Gi is a floor rather than a target. The Prisma query engine’s resident memory is a high-water mark that grows to the largest statement it has ever run and is not returned to the OS, so a pod’s floor ratchets up to its worst-ever write. Provision below that and one large write is enough to get the pod OOM-killed, which surfaces as a crash loop under traffic that looks unremarkable on every other metric.

One worker per pod is the right default on Kubernetes: it keeps the HPA reading CPU against a single process and makes rolling restarts drain cleanly.

autoscaling.targetMemoryUtilizationPercentage is deliberately unset. For the same high-water-mark reason, a memory target reads the largest write a pod ever did rather than its current load, so replicas ratchet up after one large write and never scale back in. Scale on CPU, or on the per-pod request and token metrics.

60 percent rather than something higher because a new replica is not useful the moment it is created: the startup probe allows up to 300 seconds. A target that only trips near saturation adds capacity minutes after it was needed.

targetRequestsPerSecond and targetTokensPerSecond render autoscaling/v2 Pods metrics on the proxy’s own counters. Both need a Prometheus Adapter serving those names on custom.metrics.k8s.io.

Graceful shutdown 

terminationGracePeriodSeconds: 90
lifecycle:
  preStop:
    httpGet:
      path: /health/drain
      port: 4000
      httpHeaders:
        - name: X-Drain-Token
          value: <same value as drain_endpoint_token>
proxyConfig:
  general_settings:
    enable_drain_endpoint: true
    drain_endpoint_token: <token>

/health/drain marks the pod NotReady and blocks only until in-flight requests actually finish, instead of the fixed sleep that a preStop hook usually contains. Keep terminationGracePeriodSeconds a few seconds above GRACEFUL_SHUTDOWN_TIMEOUT (30 by default) to leave room for teardown before SIGKILL.

Observability 

metricsServer:
  enabled: true
  port: 4001
serviceMonitor:
  enabled: true
  interval: 15s
proxyConfig:
  litellm_settings:
    callbacks: ["prometheus"]

metricsServer moves Prometheus /metrics onto a separate process and port with its own ClusterIP Service, so a scrape never lands on an inference worker. The ServiceMonitor then scrapes that Service instead of the proxy port.

The proxy port serves /metrics behind virtual-key auth and answers an unauthenticated scrape with 401, so enable metricsServer whenever serviceMonitor is on. The metrics port itself carries no auth: keep it off public routing. The chart rejects a metrics port equal to the service port.

For production logging, turn the proxy’s own verbosity down and emit structured logs:

logLevel: ERROR
proxyConfig:
  litellm_settings:
    set_verbose: false
    json_logs: true

Tracing with Langfuse

Prometheus answers how much and how fast. Langfuse answers what was actually asked and answered: prompt, response, model, latency, token counts, and cost, grouped into traces and attributable to a virtual key.

langfuse:
  enabled: true
  host: http://langfuse-web.observability.svc.cluster.local:3000
  existingSecret:
    name: litellm-langfuse

kubectl -n litellm create secret generic litellm-langfuse \
  --from-literal=public-key=pk-lf-... \
  --from-literal=secret-key=sk-lf-...

That sets LANGFUSE_HOST, LANGFUSE_PUBLIC_KEY, and LANGFUSE_SECRET_KEY on the proxy and appends langfuse to litellm_settings.success_callback and failure_callback.

Appends. A callback list you wrote in proxyConfig keeps every entry it had — callbacks are a set of independent sinks, and adding Langfuse to a release that already ships traces to Datadog must not switch Datadog off. Listing langfuse yourself is also fine: it is not added twice.

The keys are project keys (pk-lf-… / sk-lf-…), issued in the Langfuse UI under Settings → API keys, and they come from a Secret in both modes below. This chart accepts no inline Langfuse key, for the same reason it accepts no inline database password: a project key grants write access to every trace in the project, and an inline value would sit in the Helm release secret as well as in the pod spec.

existingSecret.publicKeyKey and .secretKeyKey name the keys inside that Secret, and default to public-key and secret-key. Point them at whatever an existing Secret already uses rather than copying it.

Callbacks are on the request path. A Langfuse that is unreachable costs every request the callback’s timeout, so prefer an address in the same cluster, and treat langfuse.host as production infrastructure rather than a best-effort sink.

Deploying Langfuse as part of this release

langfuse.deploy.enabled: true adds Langfuse itself to this release and defaults langfuse.host to its in-cluster web Service. It is off by default, and off is the recommendation.

Langfuse is a platform, not a sidecar. Turning this on adds, into a release whose own workload is one stateless proxy:

Component Why Langfuse needs it
Langfuse web + worker Deployments the UI/API, and the ingestion pipeline
PostgreSQL transactional store
ClickHouse cluster analytics store for traces and observations
Valkey ingestion queue — a second one, unrelated to valkey.enabled
SeaweedFS S3-compatible storage for event and media blobs

That is roughly ten workloads and several PersistentVolumeClaims, with their own upgrade and backup story, sharing a release lifecycle with a proxy you will want to upgrade far more often.

It also has cluster-level prerequisites this chart neither installs nor checks for you:

  • the Altinity ClickHouse operator (clickhouse.com/v1alpha1), and
  • cert-manager, which that operator requires.

Without them the subchart fails its own render with a message naming the missing CRD. For an offline render — helm template, or a GitOps diff against a cluster whose CRDs are not visible — set langfuse.clickhouse.crdCheck: false.

Deploying the server does not create project keys: nothing issues them at install time. Sign in, create an organization and a project, issue a key pair, and put it in the Secret. langfuse.existingSecret.name is required either way, and traces are rejected until the keys in it are real.

kubectl -n litellm port-forward svc/litellm-langfuse-web 3000

Every other key under langfuse is passed through to the subchart untouched — langfuse.postgresql, langfuse.clickhouse, langfuse.s3, and langfuse.langfuse for the server’s own settings; see its values reference.

Use it for an evaluation cluster, or where Langfuse genuinely has no other home. For anything you intend to keep, run Langfuse as its own release and point langfuse.host at it: upgrading this proxy should not be able to take the observability stack down with it.

Running read-only

readOnlyRootFilesystem is supported. The image writes to the filesystem on startup, so the chart adds emptyDir volumes for /tmp, /.cache, and /.npm automatically when you enable it:

securityContext:
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1000
  allowPrivilegeEscalation: false
  capabilities:
    drop: ["ALL"]
envVars:
  LITELLM_MIGRATION_DIR: /tmp/migrations
  PRISMA_BINARY_CACHE_DIR: /tmp/prisma
  XDG_CACHE_HOME: /tmp/cache

The same volumes are added to the migration Job. The environment variables above point the components that still need a writable path at the emptyDir.

Values reference

Defaults are the shipped values.yaml. Every value is validated by values.schema.json, which rejects unknown keys, so a typo is caught at install time rather than ignored.

Model management

Key Description Default
modelManagement.mode config or database config
modelManagement.seedFromConfig Also render models into config.yaml in database mode false
models Model catalog, appended to proxyConfig.model_list one mock model
proxyConfig The rest of config.yaml, rendered verbatim {general_settings: {}}
config.existingConfigMap.name Mount this ConfigMap instead of rendering one ""
config.existingConfigMap.key Key holding the config file config.yaml
config.mountPath In-container path of the config file /etc/litellm/config.yaml

Workload

Key Description Default
replicaCount Proxy replicas, ignored when autoscaling is on 1
numWorkers Passed as --num_workers; empty lets the image decide ""
listen Bind address inside the pod 0.0.0.0
serverRootPath Prefix the proxy is mounted under; "" claims the root /litellm
trustedProxies Senders whose X-Forwarded-* headers are believed ["*"]
docs.enabled Serve the Swagger UI and the OpenAPI schema true
docs.path Path for the Swagger UI; / is upstream’s mount root /docs
image.repository Proxy image ghcr.io/berriai/litellm
image.tag Empty follows appVersion ""
image.pullPolicy   IfNotPresent
imagePullSecrets   []
serviceAccount.create Needed only for a cloud IAM binding false
serviceAccount.annotations Where the IAM binding goes {}
resources Requests and limits; set these in production {}
strategy Deployment update strategy {}
terminationGracePeriodSeconds   90
topologySpreadConstraints   []
nodeSelector, tolerations, affinity   {}, [], {}
podSecurityContext, securityContext   {}
lifecycle Prefer the /health/drain preStop hook {}
volumes, volumeMounts Also applied to the migration Job []
extraContainers, extraInitContainers Run through tpl []
command, args Override the image entrypoint []
deploymentAnnotations, deploymentLabels   {}
podAnnotations, podLabels podLabels cannot set a selector key {}
commonLabels, commonAnnotations Applied to every rendered object {}

Environment and secrets

Key Description Default
environmentSecrets Secret names mounted wholesale as env vars []
environmentConfigMaps ConfigMap names mounted wholesale as env vars []
logLevel Sets LITELLM_LOG; "" to source it elsewhere INFO
envVars Plain key/value env entries {}
extraEnvVars Full Kubernetes EnvVar list, for valueFrom []
masterKey.value Inline key; prefer existingSecret ""
masterKey.existingSecret.name Take over the master key Secret ""
license.existingSecret.name Enterprise license, into LITELLM_LICENSE ""
ui.credentialsSecret.name Admin UI login ""

Service and probes

Key Description Default
service.type   ClusterIP
service.port Also the in-pod listen port 4000
service.annotations   {}
service.loadBalancerClass LoadBalancer only ""
service.loadBalancerSourceRanges LoadBalancer only []
metricsServer.enabled Separate metrics process and Service false
metricsServer.port Must differ from service.port 4001
serviceMonitor.enabled Needs the Prometheus operator CRDs false
serviceMonitor.interval, .scrapeTimeout   15s, 10s
livenessProbe.* /health/liveliness see values.yaml
readinessProbe.* /health/readiness see values.yaml
startupProbe.* Allows up to 300s to first ready see values.yaml

Routing

Key Description Default
routing.hosts Hostnames; required for ingress and httpProxy []
routing.path Published prefix; must match serverRootPath /litellm
routing.tls.enabled   false
routing.tls.secretName kubernetes.io/tls Secret ""
routing.annotations, routing.labels Merged into every routing object {}
routing.ingress.enabled   false
routing.ingress.className   ""
routing.ingress.pathType Auto-downgraded for a dotted path on nginx Prefix
routing.ingress.extraPaths Further paths per host []
routing.httpRoute.enabled   false
routing.httpRoute.parentRefs Required; the Gateways to attach to []
routing.httpRoute.matches, .filters, .timeouts Passed through [], [], {}
routing.httpRoute.extraRules Appended after the generated rule []
routing.httpProxy.enabled   false
routing.httpProxy.delegated Render a child proxy for a parent that includes it false
routing.httpProxy.ingressClassName   ""
routing.httpProxy.tls.* Falls back to routing.tls.secretName see values.yaml
routing.httpProxy.timeoutPolicy, .retryPolicy, .loadBalancerPolicy Contour policies {}
routing.httpProxy.conditions, .extraRoutes, .includes   []
routing.route.enabled   false
routing.route.tls.termination edge, reencrypt, or passthrough edge
routing.route.tls.insecureEdgeTerminationPolicy   Redirect
routing.route.wildcardPolicy   None

Single sign-on

Key Description Default
sso.enabled Sign in to the Admin UI with OIDC. Enterprise; free to five users false
sso.proxyBaseUrl Browser-facing URL; derived from routing when empty ""
sso.clientId   litellm
sso.scope   openid profile email
sso.clientState, .usePkce Required by some providers "", false
sso.authorizationEndpoint Browser-facing; must be publicly reachable ""
sso.tokenEndpoint, .userinfoEndpoint Called by the pod; may be in-cluster ""
sso.logoutUrl   ""
sso.clientSecret.value Generated on first install when empty ""
sso.clientSecret.existingSecret.name Use a Secret you manage ""
sso.keycloakBootstrap.enabled Configure the Keycloak side automatically false
sso.keycloakBootstrap.url Admin API base URL, reachable in-cluster ""
sso.keycloakBootstrap.realm Must already exist; the Job creates no realms ""
sso.keycloakBootstrap.admin.existingSecret.name Required. Keycloak administrator ""
sso.keycloakBootstrap.admin.realm Realm holding the admin account master
sso.keycloakBootstrap.roles.admin, .user Must be names LiteLLM recognises proxy_admin, internal_user
sso.keycloakBootstrap.groups.admin, .user Groups granted those roles litellm-admins, litellm-users
sso.keycloakBootstrap.adminUser.enabled Create a named administrator account true
sso.keycloakBootstrap.adminUser.username   llmadmin
sso.keycloakBootstrap.adminUser.password.value Generated when empty ""
sso.keycloakBootstrap.adminUser.permanentPassword False forces a change at first sign-in true
sso.keycloakBootstrap.extraRedirectUris Beyond the two the chart registers []
sso.keycloakBootstrap.image.* Needs kcadm.sh; match the server’s minor version quay.io/keycloak/keycloak:26.3.2
sso.keycloakBootstrap.waitTimeout Seconds to wait for Keycloak 300
sso.keycloakBootstrap.hooks.helm.weight Runs after the database Jobs "2"

Database

Key Description Default
database.enabled Forced on by modelManagement.mode: database false
database.host, .port, .name, .schema   "", 5432, litellm, ""
database.sslMode verify-full for a managed service ""
database.sslRootCert In-container path to a CA bundle ""
database.auth.mode password, awsIam, or azureEntra password
database.existingSecret.name Required; no inline password is accepted ""
database.existingSecret.usernameKey, .passwordKey   username, password
database.existingSecret.urlKey Whole URL from one key ""
database.existingSecret.hostKey Host from a Secret key ""
database.readReplica.enabled Route read-only queries to a replica false
database.readReplica.host, .port Empty falls back to the writer ""
database.connectionPool.enabled In-pod PgBouncer, transaction mode false
database.connectionPool.maxDbConnections Upstream cap per pod 20
database.connectionPool.maxClientConn   1000
database.disablePreparedStatements Required in front of an external pooler false
database.maxIdleConnectionLifetime Seconds; the proxy defaults to 60 ""

Creating the database

Key Description Default
database.init.enabled Create the database, role, and schema before anything else runs false
database.init.admin.username Administrator login. Recorded in the Helm release secret ""
database.init.admin.password Administrator password. Recorded in the Helm release secret ""
database.init.admin.existingSecret.name Read the admin credentials from a Secret you own. The chart then erases nothing ""
database.init.admin.existingSecret.usernameKey, .passwordKey   username, password
database.init.admin.database Maintenance database to connect to for CREATE DATABASE postgres
database.init.admin.sslMode Falls back to database.sslMode ""
database.init.createDatabase   true
database.init.createUser Create the login role, and reset its password to match the Secret true
database.init.createSchema Needs a non-empty database.schema true
database.init.cleanup Erase the admin credentials from the Secret the chart created true
database.init.waitTimeout Seconds to wait for the server to accept connections 300
database.init.image.* Needs psql and pg_isready postgres:17-alpine
database.init.kubectlImage.* Cleanup container only registry.k8s.io/kubectl:v1.34.0
database.init.serviceAccount.create ServiceAccount, Role, and RoleBinding for the cleanup container true
database.init.hooks.helm.weight Must be below migrationJob.hooks.helm.weight. Also used as the ArgoCD sync wave "-5"
database.init.hooks.argocd.enabled   false
database.init.backoffLimit, .ttlSecondsAfterFinished, .activeDeadlineSeconds   4, 120, 900
database.init.resources, .nodeSelector, .tolerations, .affinity   {}
database.init.podSecurityContext, .securityContext   {}

Redis

Key Description Default
redis.host, .port   "", 6379
redis.existingSecret.name, .passwordKey   "", password
redis.sentinel.enabled, .masterSet   false, mymaster
redis.coordination Render general_settings.coordination_redis true
valkey.enabled Deploy a Valkey and wire redis.* to it false
valkey.* Passed through to the valkey subchart its defaults

Response cache

Key Description Default
cache.enabled Needs a Redis: redis.host or valkey.enabled false
cache.type redis only; write cache_params yourself for anything else redis
cache.ttl Seconds an entry stays valid; empty leaves LiteLLM’s default ""
cache.namespace Key prefix, so releases can share one Redis ""
cache.supportedCallTypes Call types to cache; empty caches all of them []

Langfuse

Key Description Default
langfuse.enabled Wire this proxy to a Langfuse false
langfuse.host Base URL including the scheme; empty uses the bundled server ""
langfuse.existingSecret.name Project keys; required, in both modes ""
langfuse.existingSecret.publicKeyKey Key holding pk-lf-… public-key
langfuse.existingSecret.secretKeyKey Key holding sk-lf-… secret-key
langfuse.deploy.enabled Also deploy Langfuse; see the footprint warning false
langfuse.* Passed through to the langfuse subchart its defaults

Migration Job

Key Description Default
migrationJob.enabled   true
migrationJob.hooks.helm.enabled Required for repeated upgrades true
migrationJob.hooks.helm.weight   "1"
migrationJob.hooks.argocd.enabled Turn on only with the Helm hook off false
migrationJob.backoffLimit   4
migrationJob.activeDeadlineSeconds Whole-Job budget; null to disable 1800
migrationJob.ttlSecondsAfterFinished   120
migrationJob.serviceAccountName For a Job-specific cloud identity ""
migrationJob.resources   {}
migrationJob.extraContainers, .extraInitContainers   []

Scaling

Key Description Default
autoscaling.enabled   false
autoscaling.minReplicas, .maxReplicas maxReplicas sets the DB connection budget 1, 100
autoscaling.targetCPUUtilizationPercentage   60
autoscaling.targetMemoryUtilizationPercentage Deliberately unset; see above ""
autoscaling.targetRequestsPerSecond, .targetTokensPerSecond Need a Prometheus Adapter ""
autoscaling.behavior   {}
pdb.enabled   false
pdb.minAvailable, .maxUnavailable Set exactly one null
extraResources Extra manifests, run through tpl []

Validation

The chart fails the render, naming the value, rather than producing a workload that cannot start. A template-time error is readable; the same mistake found at runtime is an opaque CrashLoopBackOff or a route that silently publishes nothing.

Condition Why it fails
config mode with no models The proxy needs at least one model to start
seedFromConfig in config mode Every model is already in config.yaml; the value would do nothing
Database flow with no credentials Secret Credentials come from a Secret only
Unknown database.auth.mode Caught by the schema before templates render
httpRoute with no parentRefs A route with no parent attaches to no Gateway
ingress or httpProxy with no hosts Would publish nothing
TLS enabled with no certificate Secret Would serve the controller’s default certificate
More than one host on httpProxy or route Both carry exactly one
metricsServer.port equal to service.port The container cannot bind both
image.tag of latest or main-latest Splits the proxy and the migration Job across versions
podLabels setting a selector key The Deployment selector is immutable; the apply would be rejected
pdb with neither or both bounds Ambiguous
autoscaling.minReplicas above maxReplicas Cannot be satisfied
database.init with no admin credentials, or with two sources for them Nothing to authenticate as, or ambiguity about which gets erased
database.init under awsIam or azureEntra The cloud issues the login, so there is no password to set on a role
database.init with existingSecret.urlKey A connection URL carries no separate user to create
database.init.createSchema with an empty database.schema Nothing to create
database.init hook weight at or above the migration’s The migration would run before the database exists
serverRootPath with routing.path: / The release would still claim the whole host
serverRootPath and routing.path disagreeing The published prefix would return 404
sso.enabled with no host and no proxyBaseUrl There is no redirect URI to register
sso.proxyBaseUrl without a scheme Providers reject such a redirect_uri
keycloakBootstrap with no admin Secret Nothing to authenticate to Keycloak as
keycloakBootstrap roles outside LiteLLM’s own set The role is ignored and users get no privileges
keycloakBootstrap admin and user groups the same One group cannot carry two roles
keycloakBootstrap without sso.enabled It would register a client nothing uses
cache.enabled with no Redis The proxy would log connection errors and bill every request
cache.type other than redis Caught by the schema; the chart wires no other backend
redis.sentinel.enabled with valkey.enabled The bundled Valkey has no Sentinel in front of it
valkey.auth.enabled with no readable password The proxy would connect unauthenticated and fail with NOAUTH
langfuse.enabled with no Secret Project keys come from a Secret only
langfuse.enabled with no host and no bundled server There is nowhere to send traces
langfuse.host without a scheme The SDK uses it as a base URL verbatim
langfuse.deploy.enabled without langfuse.enabled It would deploy the whole platform and send it nothing

Upgrading 

helm diff upgrade litellm helm-charts/litellm -n litellm -f my-values.yaml
helm upgrade litellm helm-charts/litellm -n litellm -f my-values.yaml

The master key is never rotated by an upgrade, so virtual keys survive. In the database flow the migration Job runs first, as a pre-upgrade hook, and the release waits on it.

To move to a new LiteLLM release, bump appVersion in Chart.yaml rather than setting image.tag. That keeps the proxy and the migration Job on one version by construction.

Read the upstream release notes before crossing a minor version. Migrations are one-way, and rolling the chart back does not roll the schema back.

Troubleshooting

Pods crash-loop straight after install. Check kubectl -n litellm logs deploy/litellm. A bad proxyConfig shows as a startup parse error naming the key. In the database flow, check the migration Job first: kubectl -n litellm logs job/litellm-migrations.

helm upgrade hangs. Usually the migration Job waiting on a database lock. migrationJob.activeDeadlineSeconds bounds this at 1800 seconds by default; if it was set to null, the release waits forever.

Rate limits allow more than configured. redis.host is unset and each pod is counting on its own. See Redis.

Streaming responses are truncated. A timeout at the routing layer. See the timeout guidance for your implementation.

The Admin UI shows no models in database mode. The catalog is in the database, not in config.yaml. An empty list is expected on a fresh install; add models through the UI or POST /model/new.

Model changes do not take effect in config mode. Confirm the ConfigMap changed and the pods rolled. The pod template carries checksum/config, so a real change always restarts the pods; no restart means the rendered config was identical.

A routing object exists but nothing serves. Check the controller accepted it:

kubectl -n litellm get httproute litellm -o jsonpath='{.status.parents[*].conditions[?(@.type=="Accepted")]}'
kubectl -n litellm get httpproxy litellm -o jsonpath='{.status.currentStatus}'
kubectl -n litellm describe ingress litellm

Development 

helm dependency build helm-charts/litellm
helm lint helm-charts/litellm --strict
helm unittest helm-charts/litellm
bash helm-charts/litellm/ci/validate.sh
helm package helm-charts/litellm --destination /tmp

helm dependency build comes first and is not optional: Helm refuses to render a chart whose declared dependencies are missing from charts/, even when every condition is false. It installs the versions in Chart.lock; use helm dependency update only when you mean to move a pin, and commit the rewritten lock with it — CI fails a run whose Chart.lock changes.

tests/ holds per-template assertions. ci/validate.sh holds whole-release render checks, where the question is which objects exist together: both flows, every routing implementation, and that the proxy and the migration Job agree on one image. .github/workflows/helm-charts.yml runs all of it for every chart under helm-charts/, discovering them by listing the directory.

Pass --kube-version to any helm template you run by hand. Helm 3 and Helm 4 assume different Kubernetes versions when no cluster is reachable, which changes whether this chart’s kubeVersion constraint is met.

For schema validation:

helm template litellm helm-charts/litellm -n litellm --kube-version 1.31.0 | \
  helm kubeconform --summary --ignore-missing-schemas -f - helm-charts/litellm

The three CRD-based routing objects are skipped unless you pass --schema-location for their schemas.

Not included

  • A bundled PostgreSQL. See Prerequisites. database.init creates a database on a server you run; it does not run the server. A Redis is different: valkey.enabled can deploy one, because a response cache holds nothing that is not reconstructible.
  • The split gateway / backend / UI deployment from the upstream helm/litellm chart. One Deployment keeps the baseline at one pod and avoids path-prefix dispatch that has to track the image’s route allowlist. The templates carry app.kubernetes.io/component labels and per-component helpers, so the split can be added later without breaking any existing value path.
  • KEDA autoscaling. The HPA covers CPU and the two per-pod workload metrics.
  • Enterprise billable-request metering.

Back to top

Open source under the Apache License 2.0. Provided AS IS, without warranties, responsibility, or support from SAS except where applicable law requires otherwise.