1. Docs
  2. Changes and releases
  3. AI Agents

AI Agents

  • AI Agents agent-platform v4.35.0

    Added

    • Model and runtime image signatures are verified at admission: modelServing.imageVerification renders a Kyverno verifyImages ClusterPolicy over every model pod of the serving namespace (giantswarm/agent-platform#552). A model pod ran whatever the registry served under its references — the modelcar sidecar that carries the weights, the pre-fetch init container KServe runs from the same image and the runtime container were admitted unverified — and an installation that curates its models as signed OCI images had no way to have the platform refuse an unsigned or tampered one. Off by default; on, the connectivity chart renders <release>-model-serving-image-verification where the cache policies render (Kyverno served) and nowhere else: one rule per pod shape — the classic InferenceService predictor and the LLMInferenceService workload pod, Pods at CREATE and UPDATE, since a pod’s container images are mutable and a CREATE-only rule would admit an image swapped into a running pod — verifying every container image that matches images (Kyverno’s imageReferences, wildcards as Kyverno reads them: gsoci.azurecr.io/giantswarm/models/* covers every curated model image) against attestors, one entry per trusted signer passed through as written into one attestor set with count: 1 (keyless — issuer and subject or subjectRegExp, Rekor — a public key or certificates), mutateDigest: true pinning the verified tag to its digest, required: true, failureAction: Enforce (or Audit). Enabled with an empty images list, no attestor, an entry that is no Kyverno attestor entry, a failureAction outside the two or an unknown key fails the render naming the key. The meta chart mirrors the block (make verify-meta holds every leaf equal); make verify-model-serving-policies asserts the rendered rule of both shapes, runs the policy through kyverno apply over both fixture pods (every rule skipped, the policy accepted by Kyverno’s schema, a misspelt field refused), nothing while disabled or without Kyverno, the guards, and the meta chart’s forwarded render equal. README “Verifying model images”.
  • AI Agents agent-platform v4.34.1

    Fixed

    • The classic vLLM runtime runs its container through the llm-d template’s argument grammar, so every shipped preset serves on the classic InferenceService path too (giantswarm/agent-platform#549). The chart’s kserve-vllm ClusterServingRuntime — the path model-manager composes into wherever the LLMInferenceService API is not served — put a preset’s arguments straight into argv, while the presets write every JSON-valued vLLM flag single-quoted inside one argument (--limit-mm-per-prompt='{"image": 1, "video": 0}', giantswarm/agent-platform#532) because the llm-d runtime template re-parses its arguments with a shell; vLLM received the single quotes literally and refused the value (invalid loads value), so the six presets with such a flag could not start on the classic path, and a bare JSON that would have worked there fails the chart’s argument guard. The runtime container now has the command /bin/sh -c 'eval "exec vllm serve $@"' -- ahead of its base arguments (modelServing.runtime.args, unchanged: --model /mnt/models --port 8080 --served-model-name {{.Name}}, KServe still expands {{.Name}} before the container starts); KServe appends the preset’s arguments after them, so $@ is base + preset and a single-quoted JSON value reaches vLLM as one word with the quotes gone — exactly as on the llm-d template, one grammar for presets. The image must carry /bin/sh and vllm on PATH (the upstream vLLM images do; an installation’s own modelServing.runtime.image must too). make verify-serving-slice now also runs the RENDERED classic command over the rendered base arguments and every shipped preset’s, with a vllm stub dumping argv, and asserts the base arguments (the {{.Name}} placeholder expanded the way KServe does, an installation’s extra ones) and the preset’s reach vLLM one word each with every JSON value parsing on both paths, so the two cannot drift again; the fixture pod of verify-model-serving-policies carries the command. Not changed: a preset’s argument rule (single-quote a value with whitespace or JSON; the render refuses the rest), the base arguments, the port, the image.
    • The shipped presets serve on the llm-d runtime: a JSON-valued vLLM argument is one argument with the JSON single-quoted inside it, and the chart refuses a preset argument the runtime’s shell would re-split (giantswarm/agent-platform#532; proof 1 of giantswarm/giantswarm#37639). The well-known runtime template’s entrypoint (kserve-config-llm-template, from the kserve-runtime-configs chart) ends in eval "exec vllm serve … $@": every argument is re-parsed by the shell, so --default-chat-template-kwargs followed by {"enable_thinking": false} reached vLLM as {enable_thinking: and false} — invalid loads value, CrashLoopBackOff — and qwen3-8b-fp8 could not be served; qwen3-14b, qwen3-5-27b, qwen3-5-35b-a3b, nemotron-3-super-nvfp4 and qwen3-8-27b (three such arguments) carried the same shape. The six presets now write each such value as --flag='{…}', which the shell’s re-parse yields as one word with the JSON intact. The connectivity chart’s resolvePreset fails the render of any preset — shipped or modelServing.presets — whose argument carries whitespace, a quote or a shell metacharacter outside single quotes, naming the preset and the argument; the preset schema’s spec.args states the contract. make verify-serving-slice runs the template’s exact eval over every shipped preset’s arguments (argv dumped in place of vLLM) and asserts one word per argument and a parsing JSON value, and that seven bad argument shapes fail the render naming the guard. The classic InferenceService path gets the same grammar in giantswarm/agent-platform#549 (below).
  • AI Agents agent-platform v4.34.0

    Added

    • The serving runtime image is pre-pulled on the pool’s GPU nodes at join (giantswarm/agent-platform#545). A predictor pulls its runtime image only when its main container starts, after the storage-initializer has finished the weights, so on a pool scaling from zero the two longest steps of a cold start ran one after the other — measured on a fresh node: weights 116 s, then 115 s for the 8.8 GB llm-d-cuda image, while with prewarm the node had been Ready for two minutes. modelServing.prepull (on by default) renders a DaemonSet in the serving namespace with one /bin/true init container per image of prepull.images — the runtime image of the well-known LLMInferenceServiceConfig, gsoci.azurecr.io/giantswarm/llm-d-cuda:v0.8.0, pinned here because the connectivity chart has no other value for it and following the kserve-runtime-configs line, never ahead of it — and a pause main container (prepull.pauseImage, prepull.resources): the kubelet pulls the images the moment the node joins, in parallel with the weights, and containerd keeps them for the predictor. It runs on the nodes prepull.nodeSelector (Karpenter’s karpenter.k8s.aws/instance-gpu-manufacturer: nvidia) and the pool’s own label select, tolerates the pool’s taint first and every taint after it (prepull.tolerations, so the pull starts under a node’s start-up taints), holds no GPU, mounts no token, is denied all traffic by a policy of its own in both flavours (the pull is the kubelet’s, not the pod’s) and passes the fleet’s restricted Pod Security Standard with no exception. The meta chart mirrors the block (make verify-meta holds every leaf equal); make verify-model-serving-policies holds the rendered pod to the fleet’s restricted-PSS policies, the deny-all’s selection and the meta chart’s forwarded render.
  • AI Agents agent-platform v4.33.0

    Added

    • Preset qwen3-8-27b-l40s: Qwen3.8 27B NVFP4 on one 48 GB GPU (giantswarm/agent-platform#544). The shipped qwen3-8-27b is tuned for a 128 GB unified-memory Blackwell node and hosts on no size of the curated AWS accelerator families (giantswarm/agent-platform#535). The new preset serves the same Inferact NVFP4 repack on one L40S — g6e.2xlarge or g6e.4xlarge — with the flags a live probe proved: --gpu-memory-utilization=0.92, --max-model-len=65536, --max-num-seqs=4, --max-num-batched-tokens=8192, chunked prefill, prefix caching, one image per prompt, the qwen3 reasoning parser, tools on with the qwen3_coder parser, thinking off by default, no speculative decoding, no CUTE_DSL_ARCH, the model’s own chat template; requests 4 vCPU / 32 GiB, limits 8 / 48 GiB; weightsGiB: 25 (the Hub’s size), overheadGiB: 16; predictor.timeout: 1800. Ada has no native FP4: vLLM takes the Marlin weight-only FP4 path, loads 24.17 GiB of weights and keeps 14.65 GiB for the KV cache (222,822 tokens, 3.4 concurrent 64k requests). cluster-manager’s presetFit hosts it on 2xlarge and up of the L40S family (g6e.xlarge leaves a predictor 3 vCPU / 27.1 GiB) and on no L4, A10G or T4 size (25 + 16 = 41 GiB > 24). make verify-wiring expects ten shipped presets; verify-preset-weights and the eval check of verify-serving-slice cover the new file.
  • AI Agents klaus-operator v0.0.163

    Fixed

    Full Changelog: https://github.com/giantswarm/klaus-operator/compare/v0.0.162...v0.0.163

  • AI Agents klaus v0.1.22

    Changed

    Full Changelog: https://github.com/giantswarm/klaus/compare/v0.1.21...v0.1.22

  • AI Agents klausctl v0.4.8

    Fixed

    Full Changelog: https://github.com/giantswarm/klausctl/compare/v0.4.7...v0.4.8

  • AI Agents agent-platform v4.32.1

    Fixed

    • vLLM’s cache is a subPath of the claim’s own — .vllm-cache at the claim root, mounted at /mnt/vllm-cache on the runtime container with VLLM_CACHE_ROOT set in the same rule — and no longer a path under the model’s directory (giantswarm/agent-platform#541). 4.31.0 set VLLM_CACHE_ROOT=/mnt/models/.cache/vllm through modelServing.policies.env, and every cold start crash-looped: the storage-initializer’s Hugging Face client creates <model>/.cache/huggingface as uid 1000, mode 755, during the download, so .cache exists before vLLM starts, owned by uid 1000 and not group-writable, and the runtime (another uid, gid 1000 through the claim’s fsGroup) failed os.makedirs('/mnt/models/.cache/vllm') with PermissionError — Engine core initialization failed; a model whose weights were already cached hit the same directory. The redirect rule now mounts the claim a second time on the runtime container of both pod shapes, at /mnt/vllm-cache from the claim-wide subPath .vllm-cache (the kubelet creates a missing subPath directory with the claim root’s group and mode, root:1000 2775, which gid 1000 writes; compiled graphs are keyed by model and configuration inside it, so one directory serves every model), and sets VLLM_CACHE_ROOT=/mnt/vllm-cache on that container in the same patch — the env is never set without the directory behind it, and a pod without the cache keeps vLLM’s default cache root. modelServing.policies.env is back to HF_HUB_DISABLE_XET alone in both charts, and the runtime’s /mnt/models mount keeps the attributes KServe declares again (the classic predictor’s read-only): the runtime writes nothing into the model’s directory. make verify-model-serving-policies asserts the cache mount and the env together on the runtime container of both shapes and on the storage-initializer neither, the model mount’s readOnly unchanged, that no mount or env value of the pod names a path under /mnt/models, and that a workload pod without a model name gets neither the mount nor the env. An installation on 4.31.0 with a crash-looping model: the next pod after the upgrade starts clean; the <model>/.cache/huggingface directory the initializer created stays and is harmless.
  • AI Agents agent-platform v4.32.0

    Added

    • The connectivity chart ships the PriorityClass the gpu-node-pool chart’s prewarm placeholder runs at (giantswarm/agent-platform#539). A pool created with prewarm: true launches its first node with the release: the pool chart’s one-shot placeholder Job holds one GPU at negative priority until the first predictor preempts it. A PriorityClass is cluster-scoped, and a pool release is delivered under Flux multi-tenancy as the organisation’s tenant ServiceAccount, whose rights are namespaced — a pool chart that rendered the class failed InstallFailed on it and the pool never came up. The connectivity chart now renders PriorityClass agent-platform-prewarm-placeholder — value: -1000 (below the default priority 0, so any workload preempts the placeholder), preemptionPolicy: Never, globalDefault: false, the cluster-manager component label, a description naming its use — from clusterManager.prewarmPriorityClass (enabled: true, name, value), mirrored in the meta chart; the pool chart’s pool.prewarm.priorityClassName defaults to the name and renders no class of its own (giantswarm/gpu-node-pool#15). The class renders with the cluster-manager component, which composes the pool releases and is on in exactly one release per installation (the platform’s) — not with the model-serving switch, which on an installation is on in the serving slice cluster-manager creates beside the pool: a class gated there would arrive after the pool’s Job and leave with the slice. enabled: false renders none (a placeholder pod naming a class the cluster lacks is rejected at admission; the pool itself comes up without a prewarmed node); value is immutable on the API, so a change is a rename. Guards fail the render naming the key on a value at or above 0 or fractional, a name that is not a DNS-1123 subdomain, and the reserved system- prefix. tests/verify-mirrored-values.py holds the block equal between the charts (a forwarded copy under another name would rename the class every pool expects), with a negative control in make verify-meta; make verify-cluster-manager asserts the class, its knobs, the guards and that nothing renders while the component is off. README: “The prewarm placeholder’s PriorityClass”.
  • AI Agents agent-platform v4.31.0

    Added

    • The model-serving cache outlives a pool teardown, its claim gets a fast StorageClass by default, and vLLM’s compile cache lives on the claim (giantswarm/agent-platform#537). Measured on a GPU pool scaling from zero: every new pool downloaded the weights again (the Helm-owned serving namespace took the kept hf-cache claim down with it on uninstall), the weights loaded at the default gp3 baseline (8.8 GiB in 71 s = 124 MiB/s, the class’s cap), and torch.compile ran on every pod (its cache in an emptyDir). The connectivity chart now: (1) marks the serving namespace helm.sh/resource-policy: keep while modelServing.cache.enabled is on — an uninstall leaves the namespace, the claim and its volume behind, nothing else; a re-install adopts them; (2) renders a StorageClass of the claim’s own (modelServing.cache.storageClass, default create: true: <chart>-<claim> = agent-platform-connectivity-hf-cache, provisioner: ebs.csi.aws.com, gp3 at throughput: "1000" / iops: "4000", WaitForFirstConsumer, expansion allowed, reclaimPolicy: Delete — the claim is the durable object) and the applied claim references it; create: false with name references an existing class and renders none; pvc.storageClassName next to create: true is refused naming both; the default is AWS’s, another cloud sets provisioner and parameters; (3) adds VLLM_CACHE_ROOT=/mnt/models/.cache/vllm to modelServing.policies.env and mounts the runtime container’s /mnt/models read-write in both pod shapes, so a model’s compiled graphs persist in its cache directory. The meta chart mirrors every new default; tests/verify-mirrored-values.py now holds the whole modelServing.cache and modelServing.policies blocks equal between the charts (a forwarded env without the entry would drop it from every installation), with a second negative control in make verify-meta. An installation whose claim already exists on another class keeps it with storageClass.create: false and name: <its class> (storageClassName is immutable; the hook fails naming the field otherwise).