Run the re-wrap of subject keys

The re-wrap moves every person's data key that is not under the deployment's wrapping key onto it. It runs by itself on every pnpm stack:up, before credential starts, and it changes nothing when every key is already in place. This page shows how to run it by hand, how to read what it reports, and how to see it move a key.

Everything here is for a local stack. Run every command from the repository root.

What it moves

Each person's personal data is encrypted under a key of their own, the subject key. A subject key is never stored in the clear: it is stored wrapped (encrypted) under a wrapping key, and each stored subject key records which wrapping key it is under, as kek_id.

The wrapping key is an AES key on the security module's oktoid token, created by the first pnpm stack:up and never extractable. Its name on the token is a handle of the form hsm:v1:oktoid:<32 hex digits>. A handle names a key and is not a secret. The stack keeps it in infra/compose/.state/local/wrapping-key.json.

credential holds no software key at all, so it can only read a subject key that is under a hsm: wrapping key. The re-wrap is what guarantees that before credential starts: it moves every live subject key that is under any other key, a soft: one or an older hsm: one, onto the wrapping key it is given.

Run it by hand

The Compose service is credential-rewrap. It needs values that the stack's own script supplies, such as the wrapping key's handle and the paths to the stack's state, so a bare docker compose run does not start it. Run it through the script's Compose helper instead:

node --input-type=module -e "import { compose, stackPaths } from './infra/scripts/compose.ts'; process.exitCode = await compose(stackPaths('local'), ['run', '--rm', '--no-deps', 'credential-rewrap']);"

--no-deps leaves the running stack alone; without it Compose would run the migration and token jobs again first. Through this wrapper a run takes 9 to 13 seconds of wall time on a local stack, and ends with one line of JSON on standard output. On a stack where every subject key is already under the wrapping key, or that holds none:

{"rewrapped":{"outcome":"complete","count":0,"sources":0,"records":0,"unwrapFailed":0,"changed":0,"batches":0}}

and the exit status is 0. Before it, the command logs the same counts as the event subject_key.rewrap.finished.

Read the report

Member	Means
outcome	What to do next; the table below
count	Subject keys this run moved onto the wrapping key
sources	How many different wrapping keys they were moved from
records	subject-key.rewrap records written to the audit trail. Their counts add up to count
unwrapFailed	Subject keys left untouched because their stored value did not authenticate
changed	Subject keys another writer changed first, such as an erasure. Nothing was owed to them
batches	Batches that finished and were stored. A batch is up to 100 keys, and a batch that meets a fault writes nothing

The report and the log carry counts only. Which wrapping keys were involved is written in the audit records and nowhere else.

The exit status tells a job what to do without reading the report:

Exit	outcome	Means	What to do
0	complete	Every live subject key the run could see is under the wrapping key	Nothing
75	retry or interrupted	The token could not be reached, or the run was told to stop	Run it again
65	damaged	Everything that could be moved was moved; some stored keys did not authenticate. They are logged by subject id and left alone	Treat it as a security event first. Running again changes nothing
1	operator	The token refused or could not serve a key that a stored subject key names, or failed its own check	Look before retrying: a retry changes nothing
137	none	The process was killed (signal 9), for example by docker kill or for lack of memory, and printed no report. Each batch is stored in its own transaction, so the batches that finished are kept and only the one in hand is lost	Run it again: it moves only the keys still under another wrapping key, so it resumes where the killed run stopped

pnpm stack:up stops on 65 and 1, because credential does not start until the re-wrap has completed.

Check the result

List the wrapping keys the stored subject keys are under:

node --input-type=module -e "import { compose, stackPaths } from './infra/scripts/compose.ts'; process.exitCode = await compose(stackPaths('local'), ['exec', '-T', 'postgres', 'psql', '-U', 'postgres', '-d', 'platform', '-c', 'SELECT kek_id, count(*) FROM identity.subject_key GROUP BY kek_id']);"

After a complete run there is one kek_id, the handle in wrapping-key.json, or no row at all on a stack where nobody has been registered. pnpm stack:verify makes the same check in two rows: no live subject key is under a soft: or an unknown wrapping key, and every row, destroyed or live, is under an hsm:v1: one.

See it move a key

To watch the re-wrap move something on a stack where every key is already in place, give it a second wrapping key for one run. Do this on a local stack you can reset.

1. Have one subject key. Register a person through the API (POST /persons, with invite false). The query above then shows one row under the handle in wrapping-key.json.

2. Make a second wrapping key on the same token. The command prints its handle and nothing else:

   node --input-type=module -e "import { compose, stackPaths } from './infra/scripts/compose.ts'; process.exitCode = await compose(stackPaths('local'), ['run', '--rm', 'credential-wrapping-key']);"

   {"handle":"hsm:v1:oktoid:<32 hex digits>"}

   The command does not change wrapping-key.json.

3. Re-wrap onto it, naming the new handle for this run only:

   node --input-type=module -e "import { compose, stackPaths } from './infra/scripts/compose.ts'; process.exitCode = await compose(stackPaths('local'), ['run', '--rm', '--no-deps', '-e', 'OKTOID__KEY_WRAP__HSM__HANDLE=<the new handle>', 'credential-rewrap']);"

   The report says "count":1,"sources":1,"records":1,"batches":1, and the query shows the row under the new handle. The same command again reports "count":0.

4. Move it back by running the command from Run it by hand, which uses the handle in wrapping-key.json. It reports "count":1 again, and the row is under the original handle. The next pnpm stack:up would do the same, because it re-wraps onto the kept handle.

The second wrapping key stays on the oktoid token afterwards, unused. Nothing on this page removes it; pnpm stack:reset removes the token with every key on it.

This is not how to replace the wrapping key

The override in step 3 lasts one run, and credential keeps running with the kept handle as its wrapping key throughout. Replacing the wrapping key for good also means changing the handle the stack hands to credential and to the re-wrap, and that procedure is not described here because it has not been performed. Generating a second key is not a rotation by itself: the stored subject keys stay under the first until a re-wrap moves them.

What this page does not show

• A move from a soft: wrapping key to the token. A local stack has no way to create a subject key under a software wrapping key, so this was not produced by hand; the platform's integration tests cover it.
• A run that ends damaged, retry or operator, or that is killed. The exit statuses above are read from the command, not provoked; 137 is the shell's status for a process ended by signal 9, and what survives it is read from the re-wrap's batch transactions (apps/credential/src/modules/identity/handlers/subject-key-rewrap.ts).