Skip to content
quest'n'click

Clever CloudHow-toFor adminsApplies from 4.2.3

Rotating keys

Cette page en françaisThis page on the GitHub wiki

How to replace each secret of a hosted Studio, and what each change does to the people logged in: the session key, the worker's job key, the provider's client secret, the Git deploy key and the metrics token.

What you get

A new secret in place, the old one retired, and no one logged out unless you choose to (a compromised session key).

Context and versions

  • A hosted Studio on Clever Cloud, from 4.2.3 (STUDIO_GIT_KEY and STUDIO_METRICS_TOKEN arrived in 4.2.3).
  • Verified on: not verified on a live installation. No rotation was exercised on the reference deployment. The effects on sessions are read from the code; the session key's are backed by the test named below.

Prerequisites

  • Configuration done, the current secrets in ~/secrets/my-game/ (0600).
  • openssl, and the engine repository with npm ci done.

Where to run the commands

On your machine, in the engine repository where both apps are linked. A variable change applies at the next deploy or restart, and a redeploy is the overlap of Redeploys: plan its minute of slow writes. Never show clever env's output: it prints values (Configuration).

Steps

STUDIO_SESSION_KEY

The session cookie is a random token stored by its hash: it does not depend on this key. The key seals each session's refresh token and id_token (AES-256-GCM). It is a ring, comma-separated: the first key seals, any key opens.

  1. Make a new key and put it first, the old one after:

    # on your machine, in the engine repository
openssl rand -base64 48 > ~/secrets/my-game/session-key.new && chmod 600 ~/secrets/my-game/session-key.new
clever env set --alias my-game-studio -- STUDIO_SESSION_KEY "$(cat ~/secrets/my-game/session-key.new),$(cat ~/secrets/my-game/session-key)"
clever restart --alias my-game-studio

    Nobody is logged out. New logins are sealed with the new key; existing sessions still open with the old one.

  1. After the sessions' absolute lifetime (24 h by default, sessions.absoluteHours), every session sealed with the old key has ended. Remove it from the ring, and keep the new file as the current one:

    # on your machine, in the engine repository
mv ~/secrets/my-game/session-key.new ~/secrets/my-game/session-key
clever env set --alias my-game-studio -- STUDIO_SESSION_KEY "$(cat ~/secrets/my-game/session-key)"
clever restart --alias my-game-studio

If the key leaked, skip the ring: set the new key alone and restart. Each session's next check by the provider (within 5 min) cannot open its refresh token, and the session ends: the member sees "the session can no longer be checked: log in again" (401) and logs in again. The Bearer tokens of the CLI and the MCP are not sealed with this key and are not affected.

STUDIO_WORKER_KEY and WORKER_PUBLIC_KEY

The Studio signs every job with the private key; the worker accepts only jobs its public key checks. The worker holds one public key at a time, so there is a window during which jobs are refused.

  1. Make a new pair in a new folder (the command refuses to overwrite a key):

    # on your machine, in the engine repository
mkdir -p ~/secrets/my-game/keys-new && npm run studio-server -- worker-keys --out ~/secrets/my-game/keys-new

  1. Set both variables, then deploy the worker and the studio one right after the other:

    # on your machine, in the engine repository
clever env set --alias my-game-worker -- WORKER_PUBLIC_KEY "$(cat ~/secrets/my-game/keys-new/studio-worker.pub)"
clever env set --alias my-game-studio -- STUDIO_WORKER_KEY "$(cat ~/secrets/my-game/keys-new/studio-worker.key)"
clever restart --alias my-game-worker
clever restart --alias my-game-studio

    Between the two, the worker refuses the old Studio's jobs (403): writes that need a check are refused with a 503 ("the edit cannot be judged now: …"), previews and the health fail and are tried again. Sessions are not affected.

STUDIO_OIDC_CLIENT_SECRET

  1. Regenerate the studio client's secret at the provider (in Keycloak: the client's Credentials tab), save it to the 0600 file, set it, and restart the studio at once:

    # on your machine, in the engine repository
clever env set --alias my-game-studio -- STUDIO_OIDC_CLIENT_SECRET "$(cat ~/secrets/my-game/oidc-client-secret)"
clever restart --alias my-game-studio

    Until the restart, new logins fail. The provider then answers the sessions' checks invalid_client, which the Studio treats as its own misconfiguration, not the user's: a session lives on up to R (15 min) after its last check, then is suspended (503) until the secret matches again. Nobody is logged out once the restart is done in time.

STUDIO_GIT_KEY

  1. Add a new deploy key to the game's repository at your Git host, set it, restart, then remove the old key at the host:

    # on your machine, in the engine repository (ssh-keygen writes the pair; the .pub goes to the Git host)
ssh-keygen -t ed25519 -N '' -C my-game-studio -f ~/secrets/my-game/git-deploy-key-new
clever env set --alias my-game-studio -- STUDIO_GIT_KEY "$(cat ~/secrets/my-game/git-deploy-key-new)"
clever restart --alias my-game-studio

    No effect on sessions. A push or an import during the switch may fail and is run again.

STUDIO_METRICS_TOKEN

  1. Make a new token (32 characters or more), set it, restart, and give it to your scraper. Between the restart and the scraper's update, /metrics answers 401: expect a gap in the graphs.

Expected result, and how to check it

  • Session key: members keep working after step 1; after step 2, a session older than the ring's change would have ended anyway.
  • Worker key: a write that needs a check goes through again, and a preview builds; the worker logs no worker-job-refused.
  • OIDC secret: a new login works; /auth/me answers.
  • Git key: a push from the Studio succeeds.
  • Metrics token: the scraper's target is up again.

Common errors

SymptomCauseFix
a session key is 32 characters or more at startA short key in the ringopenssl rand -base64 48
Everyone logged out within 5 minThe old session key removed at once (no ring)Expected for a leaked key; otherwise use step 1 first
Writes 503 "the edit cannot be judged now: the processing worker refused the job (403…)"The worker and the studio hold different keysFinish step 4 on both apps
503 "the identity provider cannot be reached: the session is suspended"The OIDC secret changed at the provider, not yet in the Studio for more than RSet the new secret and restart
clever env set reads the key as an optionNo -- before the nameThe commands' form

Files or data changed

The apps' variables; the key files in ~/secrets/my-game/; at the provider and the Git host, the client secret and the deploy keys. Sessions end only in the leaked-key case.

Recovery or going back

Each step is undone by setting the previous value back and restarting, as long as you kept its file. A session key removed from the ring cannot be brought back for the sessions it sealed: they have ended.

References