agent-platform release v4.32.1
Fixed
- vLLM’s cache is a subPath of the claim’s own —
.vllm-cache at the claim root, mounted at /mnt/vllm-cache on the runtime container with VLLM_CACHE_ROOT set in the same rule — and no longer a path under the model’s directory (giantswarm/agent-platform#541). 4.31.0 set VLLM_CACHE_ROOT=/mnt/models/.cache/vllm through modelServing.policies.env, and every cold start crash-looped: the storage-initializer’s Hugging Face client creates <model>/.cache/huggingface as uid 1000, mode 755, during the download, so .cache exists before vLLM starts, owned by uid 1000 and not group-writable, and the runtime (another uid, gid 1000 through the claim’s fsGroup) failed os.makedirs('/mnt/models/.cache/vllm') with PermissionError — Engine core initialization failed; a model whose weights were already cached hit the same directory. The redirect rule now mounts the claim a second time on the runtime container of both pod shapes, at /mnt/vllm-cache from the claim-wide subPath .vllm-cache (the kubelet creates a missing subPath directory with the claim root’s group and mode, root:1000 2775, which gid 1000 writes; compiled graphs are keyed by model and configuration inside it, so one directory serves every model), and sets VLLM_CACHE_ROOT=/mnt/vllm-cache on that container in the same patch — the env is never set without the directory behind it, and a pod without the cache keeps vLLM’s default cache root. modelServing.policies.env is back to HF_HUB_DISABLE_XET alone in both charts, and the runtime’s /mnt/models mount keeps the attributes KServe declares again (the classic predictor’s read-only): the runtime writes nothing into the model’s directory. make verify-model-serving-policies asserts the cache mount and the env together on the runtime container of both shapes and on the storage-initializer neither, the model mount’s readOnly unchanged, that no mount or env value of the pod names a path under /mnt/models, and that a workload pod without a model name gets neither the mount nor the env. An installation on 4.31.0 with a crash-looping model: the next pod after the upgrade starts clean; the <model>/.cache/huggingface directory the initializer created stays and is harmless.