Architecture area

GitOps delivery

Every change to what runs is a commit in one repository. Each kind of file in its four directories is read by its own Argo CD ApplicationSet, and only Argo CD applies anything to a cluster.

The shape

Four rows, one per directory of application-repositories. People open pull requests into platform/, which the taskapp-platform ApplicationSet applies to the management cluster. The operators there react: release-operator and schema-operator commit into components/, which taskapp-catalog (the services) and taskapp-schemas (their database schemas) deploy to dev and prod. People and CI write infra/ and values/, which taskapp-infra deploys to the cluster each file names. People write packages/, which taskapp-packages installs as Crossplane packages. WHO WRITES DIRECTORY APPLICATIONSET LANDS ON People PR from Backstage platform/ Component · Release · … taskapp-platform management applied as-is; operators react operators commit back Operators only release · schema components/ derived deployment files taskapp-catalog taskapp-schemas dev · prod services and their schemas People + CI PR · version bumps infra/ + values/ cluster infrastructure taskapp-infra each file's cluster incl. the platform itself People PR · one-line version packages/ Crossplane package versions taskapp-packages each file's cluster infrastructure APIs
Scroll sideways to see all of it. The dashed box is application-repositories. Each row is one directory and the ApplicationSets that read it.

How it works

  1. One file per thing, per environment

    The path names what it is and where it runs; the environment in the path selects the cluster.

    platform/registry/payments.yaml                 # Component: identity
    platform/environments/dev/payments-release.yaml  # Release: what runs in dev
    components/payments/environments/dev.yaml        # derived: chart source
    components/payments/values/dev.yaml              # derived: image tag
    components/payments/schema/dev.yaml              # derived: schema package
    infra/backstage/management.yaml                  # infra: chart source
    values/backstage/management.yaml                 # infra: Helm values
    packages/crossplane-compositions/management.yaml # package version
  2. Files × clusters

    Each ApplicationSet crosses the files it reads with the registered clusters, keeping only the pairs where the file's environment matches the cluster's environment label. One file in, one Argo CD Application out, pointed at the right cluster.

    generators:
      - matrix:
          generators:
            - git:
                files: [path: "components/*/environments/*.yaml"]
            - clusters:
                selector:
                  matchLabels: {environment: "{{ .environment }}"}
  3. Intent in, deployment state out

    Onboarding a service adds a Component and a Release to platform/. They are applied as real Kubernetes resources on the management cluster, where release-operator derives the components/ files from the Release, pinning the chart and the image to the same commit of the service's repository.

  4. Every writer makes a commit

    People change things through pull requests opened by Backstage's GitHub App. Automation commits directly, as its own bot: release-operator for services, schema-operator for their database schemas, and each platform component's CI for its own version bump. A GitHub webhook brings every push to Argo CD within seconds.

Dev and prod

The two environments move differently, and both moves are commits to a Release.

dev: follows main

With autoDeploy on, release-operator moves dev's Release to every green build of main, as a direct commit. There's no rollback: fix forward, and the fix deploys itself.

Card action: Promote to prod coming

prod: moves only by pull request

Promote opens a PR setting prod's version to what dev runs right now. Rollback opens a PR back to the version that ran before, keeping the environment's bindings. The chart is pinned to the same commit as the image, so it rolls back too. The database schema doesn't: it only moves forward, by its own pull request. See Database schemas.

Card action: Rollback

Design choices

Intent separate from deployment state

platform/ holds what people ask for; components/ holds what the operators derived from it. To change what runs you change the Release or the DatabaseSchema, never the derived files.

Each file has one kind of writer

Automation never overwrites a human decision. Where a chart comes from (reviewed, rarely changes) sits in a different file from its image tag (changes on every deploy).

Trade-off: more, smaller files than one values file per app.

Platform resources aren't Helm releases

Every Release, Database and DatabaseSchema is applied as-is, one Application each, so each has its own sync and health status, and Backstage finds them by label.

The platform ships itself the same way

The operators and Backstage bump their own versions in infra/ after a green build, through a shared workflow. It stores no secret: GitHub's OIDC token assumes an AWS role that can read only the bot's key.

Push first, polling as backup

Webhooks reach both Argo CD and the ApplicationSet controller, so a merge syncs in seconds. A missed webhook costs at most one 3-minute poll.

One ApplicationSet per kind of file

Adding a service or an infra component is adding a file; nothing in the Argo CD repository changes.

Trade-off: Argo CD doesn't order the Applications one ApplicationSet generates. On a fresh cluster, apps that need another app's CRDs fail first and converge through retries.

Not built yet

  • The Promote button. Rollback is live; until Promote exists, a new prod version goes through the portal's Create deployment template.
  • Ordered bring-up. Infra apps on a fresh cluster converge by retry, not by declared order.

Code