Onboarding a new MDA node

Bringing an MDA onto its own back office is a mix of paperwork, platform work and federation steps. Some of it is automated by the deploy scripts; the trust decisions always stay with people, under maker-checker. In this lesson you walk through the whole sequence, from the data-sharing agreement to the first signed ping, and learn how to suspend or retire a node.

The references are docs/deploy/per-mda-backoffices.md, docs/deploy/estate.md and the runbook deploy/runbooks/federation-node.md in the platform repository.

1. People and paperwork first

Before anything is deployed:

The MDA's data-sharing agreement is signed, its record of processing (RoPA) entry is made and the
federation DPIA update is approved. Templates are in docs/federation/; they are drafts for legal review.
A NODE_KEY is agreed: lower-case and permanent. It becomes the node key, the Helm release name gsb-<key> and the
host label. A retired NODE_KEY is never reused.
The MDA's ZamPay merchant is set up for the node: its own service registration, revenue mappings, destination
accounts and a callback destination that points to the node's API, not the hub's. Until that is in place the node refuses to raise invoices (503 to the citizen and an alert to administrators); nothing else on the node is affected. Invoices the hub raised for the MDA before the carve-out stay on the hub and are finalised there.

2. Describe the MDA in the estate

The deployment estate is described in one manifest, deploy/estate/estate.yaml. Add the MDA with:

key (the NODE_KEY), codes, name, group and placement;
its modules;
its ZamPay identifiers and its sizing (small, medium or large);
an optional branding block when the derived names are not right.

Then render and validate:

node scripts/estate/render.js           # writes deploy/estate/generated/<env>/...
node scripts/estate/render.js --check   # the manifest holds and generated/ is current
npm test

Validation refuses a key or host label that collides with another instance or a legacy host, an unbundled module, and a release name over 40 characters. Commit the estate and the generated files.

The generated values tell each node who it is: PLATFORM_ROLE=node, NODE_KEY, HUB_URL, INSTALLATION_MDAS (the ministries it serves; its back office shows only their cases), MODULES_AVAILABLE, MODULES_BY_MDA and MODULES_BOOTSTRAP, and its branding.

3. Platform resources

The platform team creates, from the generated node-<key> files: the namespace, the database (its own cluster for medium and large nodes, a database on a shared cluster for small ones), the object-storage bucket, DNS, the secrets and the federation certificate. Secrets are created by an operator with values built outside git; the deploy scripts only check that the names and keys exist and never read a value.

The node's signing key is generated where it will live, put in the secret store and never e-mailed. Its key id is usually the NODE_KEY plus a date, so rotations are easy to follow.

4. Deploy

Deploy the hub first, then the back offices, then repeat on the next environment: test, stage, then production.

deploy/scripts/deploy-backoffices.sh tst --mda <key> --dry-run --tag <git-sha>   # the plan, no cluster needed
deploy/scripts/deploy-backoffices.sh tst --mda <key> --tag <git-sha>
deploy/scripts/deploy-backoffices.sh stg --mda <key> --tag <git-sha>
deploy/scripts/deploy-backoffices.sh prod --mda <key> --yes                       # promotion only

On Windows the PowerShell equivalents (deploy-backoffices.ps1) take the same arguments. A run plans, runs a read-only preflight (tools, cluster identity, secrets by name, images by digest, database, bucket, ZamPay identifiers), asks for confirmation, deploys with Helm (or Argo CD), waits for the rollout and runs a read-only smoke test: health, branding, module gate both ways, the case list refusing an anonymous caller, and, with a staff token from the secret store, the node listing only its own ministries' cases.

For nodes it then checks the discovery documents over mTLS from inside the pods, both ways (node to hub and hub to node), and writes a registration request file holding the node key, ministries, base URL and scope ping. The script never registers or approves anything.

5. Register the node on the hub (maker-checker)

Check the node from the hub's network: /.well-known/gsb-federation shows role: node and the right NODE_KEY;
/.well-known/jwks.json shows its key id.
Hub administrator A proposes the node under System administration → Federation (or POST /api/federation/nodes
with a reason): NODE_KEY, name, ministries served, base URL, scopes (start with ping), shared claims (a new node gets name only), and a certificate fingerprint only if you choose to pin it. By default the certificate is recognised by its CN.
Hub administrator B approves (POST /api/federation/changes/:id/approve). The proposer is refused with
SEPARATION_OF_DUTIES.

6. Prove it

on the hub:   POST /api/federation/ping {"target":"<key>"}   -> ok: true
on the node:  POST /api/federation/ping {"target":"hub"}     -> ok: true

Both audit trails show the same federation id (federation.sent on the caller, federation.received on the receiver). Then:

give the node its first administrators, with MFA, and check its modules;
open the network paths: the node's egress range in the hub's federation ingress allow-list and network policy, and
the reverse on the node;
finish the ZamPay merchant steps and raise a test invoice on ZamPay's test environment: the callback must arrive on
the node;
later, as the flows go live, propose the catalogue:write and events:write scopes (each approved by a second
administrator). Once the catalogue scope is active the node's published services appear on the national portal.

Suspending, re-activating and retiring

All through the registry, with maker-checker:

Suspend (POST /api/federation/nodes/:id/suspend): takes effect on the next message. The hub refuses the node's
messages (FED_UNKNOWN_ISSUER) and stops calling it (FED_UNKNOWN_PEER). Use it for a compromised key or a node that must be stopped.
Activate restores trust.
Retire keeps the row for the audit trail. A retired NODE_KEY is never reused.

Rolling back a node's image never changes its NODE_KEY, signing key or registry entry.

Check your understanding

Which three documents must be in place before a node holds citizens' data?
What does the node refuse to do until its ZamPay merchant is configured, and what still works?
Which parts of onboarding do the deploy scripts automate, and which must people do?
Why does a new node start with only the ping scope?
What happens on the next message after a node is suspended?
Why is a retired NODE_KEY never reused?