agent-platform release v4.34.0
Added
- The serving runtime image is pre-pulled on the pool’s GPU nodes at join (giantswarm/agent-platform#545). A predictor pulls its runtime image only when its main container starts, after the storage-initializer has finished the weights, so on a pool scaling from zero the two longest steps of a cold start ran one after the other — measured on a fresh node: weights 116 s, then 115 s for the 8.8 GB
llm-d-cuda image, while with prewarm the node had been Ready for two minutes. modelServing.prepull (on by default) renders a DaemonSet in the serving namespace with one /bin/true init container per image of prepull.images — the runtime image of the well-known LLMInferenceServiceConfig, gsoci.azurecr.io/giantswarm/llm-d-cuda:v0.8.0, pinned here because the connectivity chart has no other value for it and following the kserve-runtime-configs line, never ahead of it — and a pause main container (prepull.pauseImage, prepull.resources): the kubelet pulls the images the moment the node joins, in parallel with the weights, and containerd keeps them for the predictor. It runs on the nodes prepull.nodeSelector (Karpenter’s karpenter.k8s.aws/instance-gpu-manufacturer: nvidia) and the pool’s own label select, tolerates the pool’s taint first and every taint after it (prepull.tolerations, so the pull starts under a node’s start-up taints), holds no GPU, mounts no token, is denied all traffic by a policy of its own in both flavours (the pull is the kubelet’s, not the pod’s) and passes the fleet’s restricted Pod Security Standard with no exception. The meta chart mirrors the block (make verify-meta holds every leaf equal); make verify-model-serving-policies holds the rendered pod to the fleet’s restricted-PSS policies, the deny-all’s selection and the meta chart’s forwarded render.