Argo CD App-of-Apps with Sync Waves

Argo CD makes Git the source of truth for what runs in a cluster, but the layout of the Git repository decides whether that promise survives contact with a real platform. Two patterns carry most production deployments: the app-of-apps (one root Application whose directory contains other Applications) and sync waves (annotations that force ordering, so CRDs exist before the operators that manage them). Together they let a cluster bootstrap itself from an empty state.

The reconciliation model

An Application declares the source (repository, path, target revision), the destination (cluster, namespace) and the sync policy. The controller compares rendered manifests to live state on a loop - every three minutes by default - and with syncPolicy.automated set it applies the diff, pruning and self-healing as configured. There is no push step and no build agent: the cluster reconciles toward Git, continuously.

The root application

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: root-app
  namespace: argocd
  finalizers: ["resources-finalizer.argocd.argoproj.io"]
spec:
  project: default
  source:
    repoURL: git@github.com:example/argo-config.git
    targetRevision: main
    path: bootstrap
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true

The root application adds one important property: the cluster's application inventory is itself reconciled from Git. Adding a platform component means committing a new Application manifest into bootstrap/, not running kubectl. The resources-finalizer.argocd.argoproj.io finalizer ensures deleting the root application cascades to its children.

Ordering with sync waves

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-3"

Applications (and resources inside them) are synced in ascending wave order, and Argo CD waits for a wave to become healthy before moving on. A layout that works well in practice:

• Wave -5: CRDs and namespaces.
• Wave -3: operators and controllers (cert-manager, GPU operator, storage drivers).
• Wave -1: shared infrastructure - ingress controller, external DNS, monitoring stack.
• Wave 0: shared services - databases, message brokers, caches.
• Wave 1+: application workloads, in dependency order.

Without waves, a fresh cluster sync produces the classic CRD race condition: an Application fails because the custom resource it needs is not served yet, then succeeds on the next reconciliation - noisy, occasionally alarming, and entirely avoidable.

Scaling from one cluster to a fleet

generators:
  - matrix:
      generators:
        - list:
            elements:
              - {name: cert-manager, wave: "-3"}
              - {name: kube-prometheus, wave: "-1"}
        - clusters: {}
template:
  metadata:
    name: 'platform-{{name}}-{{cluster.name}}'
    annotations:
      argocd.argoproj.io/sync-wave: '{{wave}}'

An ApplicationSet with a matrix generator multiplies the platform component list by the registered clusters, so onboarding a cluster means registering it in Argo CD, not writing another set of manifests. Waves in the template keep per-cluster ordering identical to the single-cluster case.

Drift, self-heal and verification

kubectl -n argocd get applications
argocd app get root-app --show-operation
argocd app sync platform-cert-manager-prod --prune
kubectl -n argocd get applications -o json | jq '.items[] | {name:.metadata.name, sync:.status.sync.status, health:.status.health.status}'

Self-heal reverts manual changes, which is the point, but it also means an emergency hotfix applied by hand disappears at the next reconcile. If a manual change must survive, put it in Git or add an ignoreDifferences entry with a comment explaining why. Alert on applications that are OutOfSync or Degraded rather than on the underlying workload - Argo CD already knows both.

Related: NetworkPolicy default-deny patterns, Kubernetes CNI comparison, and Proxmox VE quorum loss recovery.

原文链接：https://kubernetes.recipes/recipes/deployments/argocd-app-of-apps-sync-waves