AI Agents

  • Fixed

    • The kagent controller metrics Service the connectivity chart renders (<release>-kagent-controller-metrics, the target of its ServiceMonitor) selected the controller pods with the connectivity release’s own app.kubernetes.io/instance label. Under the retired standalone umbrella every subchart shared one release name, so that matched; under the meta chart each component is its own release — the kagent pods carry app.kubernetes.io/instance: kagent, the component key the meta chart fixes as the release name — and the Service had no endpoints: the controller was never scraped on any installation with kagent and the monitors on (#305). The selector now names kagent’s release through the new connectivity helper agent-platform.kagent.releaseName; the fallbacks of the controller route’s backend host, the UI route’s Service and the oauth2-proxy Service (kagent.fullnameOverride / kagent.oauth2-proxy.fullnameOverride, pinned in values.yaml, else .Release.Name) go through agent-platform.kagent.fullname / the release-name helper too — the same wrong assumption, latent behind the pins. No compatibility knob: the meta chart is the chart’s only consumer. No installation needs to act: the Service’s selector changes (a Service update, no pod rolls) and kagent controller scraping starts working; every other object is byte-identical. make verify-global asserts the selector and that no kagent template falls back to .Release.Name.
    • A first helm install with components.kagent.enabled on a cluster without the kagent namespace deadlocked: the kagent chart installs into kagent.namespaceOverride while its HelmRelease targets the platform namespace (so install.createNamespace never creates kagent), and the one chart object that does create it — the connectivity chart’s Namespace — sits in a release that dependsOn kagent. Every kagent attempt failed with namespaces "kagent" not found until the retries were exhausted, everything behind kagent waited, helm install --wait timed out (#306). With the bundled engine the meta chart now runs one pre-install,pre-upgrade hook Job, <release>-kagent-namespace (weight -8, alpine/k8s, as the hook ServiceAccount, whose own hook events gain pre-install,pre-upgrade while this hook renders): it creates the namespace when it is missing, waits out one that is still terminating (the reinstall right after an uninstall), and leaves an existing one alone. The Namespace is deliberately not an object of the meta release — the connectivity release renders and tracks it, adopting the hook-created one exactly as it adopts the one Giant Swarm’s bases pre-create, and two Helm releases must never track one object. Not rendered with the engine off (the fleet’s pure render is byte-identical; a cluster’s own Flux gets the namespace out of band), with kagent off, with kagent.namespaceOverride empty or equal to the HelmReleases’ target namespace. Measured on kind (kagent on, no namespace): helm install --wait deployed, helm upgrade idempotent, helm uninstall --wait clean with the namespace gone through the connectivity release, reinstall clean. make verify-engine asserts the hook and its absences; the ATS kind smoke can turn kagent on from this version.
  • Fixed

    Full Changelog: https://github.com/giantswarm/muster/compare/v5.14.1...v5.14.2

  • Fixed

    Full Changelog: https://github.com/giantswarm/klaus-oci/compare/v0.0.72...v0.0.73

  • Fixed

    • The shared RemoteMCPServer agent-platform/muster reported Accepted=False (ReconcileFailed … Unauthorized) on every installation: the kagent controller lists a server’s tools with its own request, which carries no bearer, and muster (an OAuth resource server) answers 401 (#299). While muster runs with OAuth on (muster.muster.oauth.server.enabled, the default) the connectivity chart now labels the CR kagent.dev/discovery: disabled, the opt-out from controller-side tool discovery contributed upstream (kagent-dev/kagent#2752, 0.10.x backport kagent-dev/kagent#2753): a controller that knows the label reports Accepted=True (DiscoveryDisabled, empty inventory); an older one ignores the label and behaves as before. Agents are unaffected — they resolve tools at run time as the caller. A controller credential through spec.headersFrom was rejected on purpose: the kagent runtime applies static headers after the propagated caller token, so every agent would have acted as the controller’s ServiceAccount (docs/authentication.md, “Tool discovery by the kagent controller”). make verify-kagent-discovery asserts the label follows the OAuth toggle and stays off the operator-defined kagent.remoteMcpServers.
  • Added

    • Self-management through the bundled Flux, on by default (roadmap#4348, PRD Q5/Q5b): where the chart brings the engine, the engine also holds the chart. gitops.self.enabled (auto | true | false, default auto = follows components.flux.enabled; true with the engine off fails the render) renders the release’s own OCIRepository + HelmRelease (templates/self/release.yaml, both named after the release, in the release namespace, run as agent-platform-flux — gitops.serviceAccountName overrides it — chartRef to the OCIRepository, releaseName/targetNamespace the release’s, install/upgrade.disableWait, values from Secret agent-platform-values key values.yaml with optional: false). The OCIRepository follows <gitops.self.repository>/agent-platform (default oci://gsoci.azurecr.io/charts/giantswarm; gitops.self.insecure for a plain-HTTP lab registry) at >=<installed version> <next major>.0.0, derived from the running chart at render time (gitops.self.versionRange overrides; an offline helm template derives it from the placeholder Chart.yaml version), on gitops.self.interval (default 10m). The bundled helm-controller adopts the CLI-installed release with one extra revision and rolls the chart forward inside its major without a helm upgrade. Measured on kind: install --wait 87 s, adoption in the second the release went deployed, a pushed version applied 47 s after the push with the new chart default in effect and the user’s values intact, a rewritten values Secret applied within 20 s, uninstall --wait with the self HelmRelease present in under a minute, the upgrade from an engine-on 3.18.0 into this version adopted with the bracket held.
    • The install-time bracket: the self HelmRelease is rendered suspend: true on helm install and whenever it does not exist yet (a guarded lookup — the upgrade of an engine-on installation into this version), never on a controller-driven upgrade; the post-install/post-upgrade hook <release>-self-values (weight 0) writes the user-supplied values of the revision in flight (helm get values, never the merged tree) into Secret agent-platform-values, labelled reconcile.fluxcd.io/watch=Enabled, by server-side apply, and — only while the HelmRelease is suspended — starts the detached Job <release>-self-resume, which polls helm status and resumes the HelmRelease once the release is deployed (or failed); still pending after 300 s it exits 1 without resuming. Neither is a chart object Helm’s kstatus --wait would wait on. helm template renders suspend: true.
    • The Helm CLI is day-0 only — a ValidatingAdmissionPolicy (templates/self/policy.yaml: <release>-self-managed-<namespace> + a binding of the same name scoped to the release namespace, failurePolicy: Fail) on the CREATE of Secrets of type helm.sh/release.v1 labelled name=<release>: admitted for system:serviceaccount:<namespace>:agent-platform-flux (the identity the bundled helm-controller impersonates) or while the namespace carries agent-platform.giantswarm.io/helm-cli=allow. helm upgrade and helm rollback fail in about a second with the reason in Helm’s own error output (the message names Secret agent-platform-values and the hand-back) and write no revision; UPDATE/DELETE pass, so does helm uninstall. Needs Kubernetes ≥ 1.30: the render refuses an older cluster while self-management is on. A render guard also refuses the annotation together with self-management on (two writers on one release). The guard-Job alternative (a pre-upgrade hook reading the pending storage Secret’s managedFields manager) stays documented in UPGRADE.md as the fallback should the policy ever be ruled out.
    • The hand-back / escape hatch: kubectl annotate namespace <ns> agent-platform.giantswarm.io/helm-cli=allow, then helm upgrade … --set gitops.self.enabled=false --force-conflicts (once; helm-controller co-owns the fields) hands the release back to the CLI without an uninstall — the new pre-upgrade hooks suspend the self HelmRelease before Helm deletes it, so helm-controller drops its finalizer; the value stays in the values file (README “Self-management”).
    • Teardown extended (templates/hooks/self.yaml, rendered whenever the engine is on): weight -6 <release>-self-stop-resumer (one plain kubectl delete of the resumer Job, --ignore-not-found) and weight -5 <release>-self-suspend (suspends the self HelmRelease if present, removes the hook-written values Secret by its labels) at pre-delete — and at pre-upgrade too while self-management is off (the hand-back and lab renders). They run as the new namespaced identity <release>-self (ServiceAccount + Role + RoleBinding, templates/self/rbac.yaml, regular objects: the detached resumer runs after a hook-managed account is gone), as do the values hook and the resumer. gitops.hooks.helmImage (docker.io/alpine/k8s:1.36.4 — kubectl, helm, jq, a shell; Renovate-tracked like gitops.hooks.image) runs the hooks that need helm or have to tolerate a missing object; templates/hooks/_job.tpl takes script, image and serviceAccountName.
    • Lab shape: gitops.self.enabled: false renders none of it (no self objects, no values hook, no policy) but the -6/-5 hooks and their identity; tests/test-values.yaml sets it — the ATS installs an unreleased chart, which a self HelmRelease would replace with the published one — and tests/ats/test_smoke.py asserts the lab shape on the cluster. make verify-self (tests/verify-self.py, in CI with Helm 3.17.3) asserts the three shapes, the guards and the knobs; make verify-engine renders its engine-on shape with self-management off. No installation needs to act: the fleet (engine off) renders byte-identically; an engine-on CLI installation becomes self-managed on its next helm upgrade — see UPGRADE.md.
  • Added

    • The standalone umbrella’s wiring, ported into the connectivity chart and gated on the component toggles the meta chart forwards (roadmap#4348, PRD Q6): with components.backstage on, the portal’s Agent Platform app-config ConfigMap agent-platform-backstage-app-config (the login provider from global.identity, one installation backstage.installationName, the in-cluster Kubernetes and muster entries, agentPlatform.fluxServiceAccountName from the one kagent.fluxServiceAccountName value, the skill repositories, the kagent / model-manager API routes when on, the pg block for backstage.database.engine: postgresql), the public HTTPRoute backstage.<domain> and a post-install/post-upgrade config-reload hook that stamps the rendered app-config’s checksum into the Backstage Deployment’s pod template (kubectl set env by label: a no-op while unchanged, a rolling update on a change, nothing on a first install) with its network policy in both flavors (templates/backstage/); with components.mcp-kubernetes on and muster on, the MCPServer CR that registers the server with muster — OAuth, forwardToken, requiredAudiences: [mcp-kubernetes.kubernetesAudience], the infrastructure tool-group label (templates/mcp-kubernetes/); with the new feature switch components.modelServing on (an entry without chart:: the loop renders no release for it and forwards its flag like every other; a top-level block named after a switch — modelServing: — travels with forwardAllValues only while the switch is on, so a live connectivity chart that predates the block never sees it), model serving on KServe with vLLM from the modelServing: block — the ClusterServingRuntime, the seven shipped serving presets and their discovery ConfigMap, the Hugging Face cache PVC, the Kyverno cache policies (modelServing.policies.enabled: auto, resolved with the other cluster-shape knobs) and the serving namespace’s network policies in both flavors (templates/model-serving/, files/model-serving/), plus the guard that a model-manager on the kserve backend agrees with the layer; the render fails with a clear message when components.modelServing is on without components.kserve-crd and components.kserve-resources (or a cluster that serves the serving.kserve.io APIs; modelServing.kserve.requireApi: false skips the check); and with components.kserve-resources / components.kserve-llmisvc-resources on, the controllers’ network policies and guards (templates/kserve/). The wiring’s own keys keep the standalone’s names minus components. — backstage.<hostname, parentRefs, installationName, extraScopes, startUrlSearchParams, enabledExtensions, disabledExtensions, skillsRepositories, catalogs, configReload> and mcp-kubernetes.kubernetesAudience live in the chart blocks and are dropped from the values forwarded to the strict component charts (components.backstage.omitKeys, components.mcp-kubernetes.omitKeys); modelServing.* is the standalone’s components.modelServing.* one-to-one; components.kserve.* maps onto the four kserve component toggles (README, “Turning on the standalone’s extras”). The seven values blocks of the extras are no longer held back from the connectivity release (components.agent-platform-connectivity.omitKeys is empty), the connectivity release dependsOn muster (the MCPServer CRD) and backstage dependsOn agent-platform-connectivity (its pod mounts the ConfigMap rendered there). The kagent UI route guard from the standalone lands too: a kagent.uiRoute.hostname override with an oauth2-proxy redirect-url still derived from global.domain fails the render. global.identity.ca (secretName, key) is declared by the connectivity chart and accepted by the meta chart (global.identity is additionalProperties: true there — no default, so the global block injected into every component release is unchanged). make verify-wiring (in CI) asserts off = no object, the objects and guards on, both flavors and the vanilla shape, and the meta chart’s forwarding. No installation needs to act: every toggle is off on the fleet; the connectivity HelmRelease’s values gain the seven blocks and the modelServing roster line, its dependsOn gains muster, and the connectivity chart renders byte-identical objects from them — one Helm revision of that release, no object change, no pod rolled.
  • Fixed

    • A fresh install with components.kagent.enabled and postgres.enabled deadlocked: the kagent controller mounts the CNPG Secret kagent-pg-app that the connectivity release renders, while connectivity dependsOn kagent for its CRDs, so the kagent install waited for a controller that could never start (FailedMount), timed out, was uninstalled and retried, and connectivity never installed. The kagent child HelmRelease now carries install.disableWait: true (new per-component key components.<name>.installDisableWait): the first install is reported once the manifests (CRDs, Deployment) are applied, connectivity proceeds, the controller starts as soon as the Cluster exists; upgrades keep waiting for the rollout. make verify-meta asserts it. Existing installations see only the changed HelmRelease spec, no Helm action.
  • Added

    • The chart brings its own Flux engine (roadmap#4348, PRD Q1/Q2/Q3/Q9): helm install of the meta-package on a cluster without Flux yields a running platform. The new flux-engine subchart (helm/agent-platform/charts/flux-engine, released with this chart, the only dependencies: entry of Chart.yaml, condition: components.flux.enabled, default true) carries the seven Flux CRDs of source-controller and helm-controller and the four Flux Operator CRDs in its crds/, and renders the Flux Operator (ghcr.io/controlplaneio-fluxcd/flux-operator 0.59.0 as a Renovate-managed pin, web UI off, no network policy — the upstream chart’s Deployment/ServiceAccount/cluster-admin binding/metrics Service, lifted verbatim) and one FluxInstance named flux in the release namespace (distribution.version: "2.x" from ghcr.io/fluxcd, components: [source-controller, helm-controller], cluster.multitenant: true with tenantDefaultServiceAccount: default, no network policy). The operator upgrades Flux — controllers, CRDs, stored objects — inside the range on its own. Its values live under flux-engine:. components.flux is a feature switch, not a component: an entry without a chart key takes part in the roster and the enablement logic but renders no release (templates/components.yaml). Measured on a fresh kind cluster: helm install --wait deployed in about two minutes, the FluxInstance Ready at Flux v2.9.5 in 15 s.
    • The tenant identity agent-platform-flux: a ServiceAccount in the release namespace bound to cluster-admin, rendered by the subchart; gitops.serviceAccountName defaults to it whenever the engine is on, so every platform HelmRelease names it (under the lockdown a HelmRelease without one runs as the rights-less default account and fails). The fleet keeps "" and its exempt namespace and renders none.
    • Render guards for the engine (agent-platform.validateEngine): with the engine on, a cluster that already runs Flux — a helm-controller Deployment (app.kubernetes.io/component=helm-controller) or a FluxInstance outside the release namespace, looked up live under the Helm CLI, --dry-run=server and helm-controller — fails the render with this cluster runs Flux; set components.flux.enabled=false or install the chart through it; gitops.namespace set to another namespace fails (the tenant identity lives in the release namespace); with the engine off, turning it off on an installation that runs it fails (an upgrade would delete the operator together with the FluxInstance it finalizes and hang). helm template sees no cluster and renders the engine.
    • The ordered teardown: pre-delete hook Jobs (templates/hooks/, rendered only with the engine on) — a hook ServiceAccount + ClusterRoleBinding <release>-hooks to cluster-admin at weight -10, <release>-teardown-releases at weight 0 deleting the platform HelmReleases by name in gitops.namespace and waiting, <release>-teardown-engine at weight 5 deleting the FluxInstance and waiting; one plain kubectl command each in the distroless registry.k8s.io/kubectl (gitops.hooks.image, a Renovate-managed pin), restricted pod security, --ignore-not-found, hook-delete-policy: before-hook-creation,hook-succeeded; weights -6/-5 are reserved for the self-management hooks. helm uninstall --wait returns clean in well under a minute; the operator removes Flux including its CRDs (every HelmRelease object with them, the agents’ included; workloads and Helm storage stay), the four operator CRDs remain, a reinstall is clean. Without the hooks Helm deletes the operator together with the FluxInstance and hangs for its timeout (measured).
    • A kind install smoke in the ATS (.ats/main.yaml runs the smoke step with app-tests-skip-app-deploy: true; tests/ats/test_smoke.py): the Gateway API CRDs, then helm install --wait of the candidate with tests/test-values.yaml (the kind shape: engine on, muster + dicebear + connectivity, valkey off, muster’s OAuth server and the network policies off — the auto knobs resolve the rest), asserting the release deployed, the FluxInstance Ready at a Flux 2.x, every component HelmRelease Ready as agent-platform-flux, one Helm revision, the operator as field manager of the Flux CRDs; then helm uninstall --wait in under a minute with no Flux CRD left and the operator CRDs kept, and a clean reinstall. execute-chart-tests runs it on every PR.
    • make verify-engine (tests/verify-engine.py, in CI with Helm 3.17.3): engine off renders no CRD, no hook, no operator, no FluxInstance, no identity and no serviceAccountName, the fleet shape stays clean; engine on renders exactly the eleven CRDs, the operator objects, the FluxInstance (two controllers, multitenant), the identities and the hooks at their weights, every HelmRelease names agent-platform-flux, the teardown Job names every rendered HelmRelease; the two shapes’ OCIRepository/HelmRelease documents are identical but for serviceAccountName and the roster; Chart.yaml’s only dependency is flux-engine, conditional, a local directory, its version and appVersion in step with the subchart. make verify-meta asserts the pure-renderer rule with the engine off and that the one dependency is not a component pin. README: a quick start (domain, identity, components — nothing about Flux), the engine, clusters that run Flux, uninstalling, Helm versions. Management clusters need not act: they carry components.flux.enabled: false already (management-cluster-bases#721), and the fleet render with it is byte-identical. Other installations that run their own Flux set the value before upgrading — see UPGRADE.md.
  • Fixed

    Full Changelog: https://github.com/giantswarm/muster/compare/v5.14.0...v5.14.1

  • Changed

    Full Changelog: https://github.com/giantswarm/klaus/compare/v0.1.0...v0.1.1