Key management — Horcrux threshold signing

Run a Krypton validator's block signing through Horcrux (Strangelove's CometBFT remote/threshold signer) so that no single host ever holds the validator's priv_validator_key.json. The key is sharded 2-of-3 across cosigners and reconstructed nowhere — a signature is a multi-party computation needing ≥ 2 of 3 cosigners to agree.

The deploy artifacts are in l1/from-source/deploy/horcrux/. This page is the operator-facing version of that directory's README.md.

Deploy artifact — not yet run on real infra

Validated with docker compose ... config -q only; no live signing test was performed, and the public testnet is not live yet. Read the caveats — especially the privval-compatibility dry-run — before production.

Why threshold signing

A plaintext priv_validator_key.json on the node is the #1 slashing risk:

• Double-sign hazard. Run the same key on two hosts (botched failover, a forgotten old box) and they emit conflicting votes at one height — equivocation, slashable under the W3 SlashingModule.
• Single-key compromise. A plaintext key on an internet-facing beacond host is one container escape from theft → impersonation/equivocation.

Threshold signing fixes both: compromising one host yields one useless shard, and the cosigners run a Raft cluster holding a shared high-watermark (last signed height/round/step) so the cluster refuses to sign at/below a height it already signed — double-signing becomes structurally hard, not just a matter of discipline.

Topology

docker-compose.yml defines a 2-of-3 cosigner cluster (cosigner-1/2/3). A commented single-signer fallback exists for test/small setups.

            ┌─────────────── private link (WireGuard / VPC) ───────────────┐
            │                                                                │
  ┌─────────┴────────┐  raft 2222  ┌──────────────────┐  raft 2222  ┌────────┴─────────┐
  │   cosigner-1     │◀── gRPC ───▶│    cosigner-2     │◀── gRPC ───▶│   cosigner-3     │
  │   shard 1        │    5703     │    shard 2        │    5703     │   shard 3        │
  └─────────┬────────┘             └─────────┬────────┘             └────────┬─────────┘
            │                                 │                               │
            └──────────── privval 1234 (Horcrux DIALS beacond) ──────────────┘
                                            │
                                 ┌──────────┴──────────┐
                                 │  beacond (cl)       │  priv_validator_laddr = tcp://0.0.0.0:1234
                                 │  NO local key       │  (the privval LISTENER)
                                 └─────────────────────┘

Sign-flow direction — get this right

beacond is the privval LISTENER; the cosigners dial in to it.

• beacond sets priv_validator_laddr = tcp://0.0.0.0:1234 — it listens on 1234.
• each cosigner's config chainNodes[].privValAddr is the address it connects to.
• cosigners do not publish 1234; they only mesh with each other on 2222 (Raft) and 5703 (gRPC nonce exchange) and need outbound reachability to beacond:1234.

This is the opposite of beacond dialing out to a signer. Wiring it backwards means no signatures.

Run each cosigner on a separate host

The single-host layout in the artifact co-locates all three shards — that is for compose config validation only and delivers no security guarantee. In production, run each cosigner in a separate failure domain so one compromised host never holds ≥ 2 shards.

Ports (all private, never public)

Port	Direction	Purpose
1234	beacond → cosigner	privval (Horcrux dials beacond's listener)
2222	cosigner ↔ cosigner	Raft (leader election + high-watermark)
5703	cosigner ↔ cosigner	gRPC nonce exchange (MPC signing)

Migration ceremony (from a local key)

Do this once, on a secure offline machine with the horcrux binary (or docker run --rm ... ghcr.io/strangelove-ventures/horcrux:v3.3.2 <cmd>). Inputs: your existing priv_validator_key.json and the chain-id 473374.

# 1. Cosigner-to-cosigner encryption keys (ECIES, secp256k1).
horcrux create-ecies-shards --shards 3 --output-dir ./ceremony
#    -> ./ceremony/cosigner_{1,2,3}/ecies_keys.json

# 2. Shard the validator key 2-of-3. (horcrux enforces threshold > shards/2,
#    so 2-of-3 is the minimum valid config.)
horcrux create-ed25519-shards \
  --chain-id 473374 \
  --key-file ./priv_validator_key.json \
  --threshold 2 \
  --shards 3 \
  --output-dir ./ceremony
#    -> ./ceremony/cosigner_{1,2,3}/473374_shard.json

Flag name changed in v3.x

The sharding command is create-ed25519-shards (older docs say create-shards, since renamed). create-rsa-shards is an alternative comms scheme to ECIES — pick one and use it consistently.

3. Distribute — one shard set per cosigner host. For each cosigner i, place under HORCRUX_DIR/cosigner-i/ (mapped to the container's /home/horcrux/.horcrux/):

• 473374_shard.json — that host's single Ed25519 shard
• ecies_keys.json — that host's comms key
• config.yaml — from config/config.threshold.yaml (identical on all three; fill in the cosigner p2pAddrs and the beacond privValAddr)

Move them over a secure channel; never email them or leave copies lying around.

4. Destroy the plaintext key — but only after the dry-run below. See caveats.

beacond side: switch to a privval listener

beacond (Krypton's CL, a beacon-kit / CometBFT fork) must be told to listen for a remote signer instead of reading a local key. In the beacond home config/config.toml:

# Before (default): empty -> beacond loads priv_validator_key.json locally
priv_validator_laddr = ""

# After: beacond opens a privval listener; Horcrux dials in and signs.
priv_validator_laddr = "tcp://0.0.0.0:1234"

When priv_validator_laddr is set, CometBFT serves the privval socket and no longer needs the local key file. The cl service mounts the CL home, so editing config.toml on the host is sufficient — beacond start reads it from --home, so no compose change is strictly required for the config flip itself.

You must, however, make the listener reachable by the cosigners on a private address:

• add 1234 to the cl service ports: bound to beacond's private/WireGuard IP (e.g. "<private IP>:1234:1234") — never 0.0.0.0 on a public host — or put the cosigners on the same docker network and use the service name;
• set each cosigner's chainNodes[].privValAddr to that address. The templates use tcp://host.docker.internal:1234 for the single-host artifact; replace it with beacond's private IP for real deployments.

priv_validator_state.json stays — Horcrux keeps its own high-watermark, and a stale local state file is harmless once signing goes through the listener.

Running

cp .env.example .env          # set HORCRUX_IMAGE (pin a digest), HORCRUX_DIR
# place per-cosigner config.yaml + 473374_shard.json + ecies_keys.json under
#   HORCRUX_DIR/cosigner-{1,2,3}/
chmod -R 0700 "$HORCRUX_DIR" && chown -R 2345:2345 "$HORCRUX_DIR"  # image uid:gid
docker compose -f docker-compose.yml up -d
docker compose -f docker-compose.yml logs -f   # watch for RaftStore / leader election + sign events

Bring the cosigner cluster up before (or together with) flipping beacond to the listener, so beacond's privval listener gets a signer as soon as it starts.

Single-signer fallback

For a test/small setup without threshold: comment out cosigner-1/2/3 and uncomment the signer: service. Init its home with config/config.single.yaml (signMode: single) and place the whole priv_validator_key.json in HORCRUX_DIR/signer/. Start requires --accept-risk (horcrux refuses single-signer silently — there is no double-sign protection).

Never run two single-signers for one validator

That re-creates the double-sign hazard. The single signer keeps the key off the beacond container but is a single point of compromise and failure.

Security & caveats

Security model. Key never reconstructed (2-of-3 MPC); structural double-sign defense (Raft high-watermark); blast radius bounded — lose 1 cosigner and the cluster keeps signing, lose 2 and signing halts safely (liveness fails closed, no equivocation). Even an owned beacond host yields only a signer-client socket, not signing material.

Confirm privval compatibility on a multi-epoch testnet dry-run BEFORE destroying any plaintext key

Krypton's CL is a fork of CometBFT (a berachain/cometbft bera-v1.x fork, not upstream). Horcrux speaks the standard CometBFT privval protocol (SignVote, SignProposal, PubKeyRequest over the secret-connection privval socket), which is stable across CometBFT versions, so this should work — but it has not been verified against the exact fork build. Before mainnet:

1. confirm the fork did not change the privval wire protocol / message types;
2. confirm beacond honors priv_validator_laddr (standard CometBFT does);
3. do a testnet dry-run where the validator signs through Horcrux for several epochs, then cross-check the fork's CometBFT version against Horcrux's vendored cometbft dependency for protocol drift.

Only after the dry-run passes:

# On the OLD validator host, once the cluster is confirmed signing:
shred -u ${DATA_DIR}/cl/config/priv_validator_key.json   # or `rm -P` on macOS/BSD
shred -u ./priv_validator_key.json                       # the ceremony machine's copy

Keep an encrypted, air-gapped backup only if your recovery plan requires re-sharding.

• Pin the image. .env.example pins v3.3.2; for production replace the tag with the @sha256 digest (re-verify it before use). Mutable tags are a supply-chain/consensus risk.
• Firewall the mesh. 2222 and 5703 carry cosigner secrets; 1234 is beacond↔cosigner. All private only (WireGuard / VPC / private VLAN), default-deny, allow only the specific peer IPs. Nothing here is ever reachable from a public interface. See Ports & firewall.
• Run cosigner hosts hardened — non-root (uid 2345), HORCRUX_DIR 0700, encrypted disk, no inbound except the cosigner mesh + SSH-over-VPN.
• No metrics wired — debugAddr is empty; set it (private bind) to add the cluster to Monitoring.
• Config schema matches horcrux v3.x; if you bump the image, re-generate with horcrux config init and diff against the templates.

See also

• Secrets (Vault / KMS) — encrypted-at-rest custody for the shards / ECIES keys (Vault/KMS store, Horcrux signs)
• Validator node — the beacond node that flips to the privval listener
• Ports & firewall — the private 1234 / 2222 / 5703 ports