AYOUB.*
← EMERGING SYSTEMS ENGINEERINGREPOS ↗

Appendix C · Part IV: Reference

Migration and Adoption Playbooks

Six programmes, each with its trigger, its sequence, its exit conditions, and the decision that most often goes wrong. Every step names an observable condition rather than a date, because a date with no condition gets moved.

In this chapter
  1. How to use these
  2. Playbook 1: Post-quantum cryptographic migration
  3. Playbook 2: Confidential computing adoption
  4. Playbook 3: Supply chain hardening
  5. Playbook 4: Agentic system to production
  6. Playbook 5: Internal developer platform
  7. Playbook 6: Efficiency and carbon reduction
  8. The one rule under all six

How to use these

Each playbook has the same five parts. The trigger says when the programme is worth starting. The prerequisites say what must exist first. The stages give an ordered sequence with an exit condition for each. The failure modes name what goes wrong and how it is visible. And the decision point names the one choice that determines whether the programme succeeds.

Skip a stage only after writing down why, because a skipped stage is a decision and a forgotten one is a future incident.

Playbook 1: Post-quantum cryptographic migration

Trigger: any data with a confidentiality lifetime beyond about five years, any long-lived trust anchor, or any regulatory deadline that applies to you.

Prerequisites: a named owner with budget, and executive awareness that this is a multi-year programme rather than a project.

Stage 1. Inventory. Record every use of public-key cryptography with its algorithm, key lifetime, data confidentiality lifetime, code path, and owner. Use static analysis, binary scanning, network scanning, and certificate inventory together, because each finds what the others miss. Exit when coverage is complete and the gaps are documented rather than unknown.

Stage 2. Prioritize by confidentiality lifetime, not by sensitivity label. Long-lived archives and root keys first, session-scoped data last. Exit when every inventory item has a priority and a target stage.

Stage 3. Instrument. Every cryptographic operation emits its algorithm as telemetry. Exit when you can answer "how many connections negotiated the classical group in the last hour, and which clients" without an investigation.

Stage 4. Enable hybrid key exchange on public endpoints, on a small fraction first. Exit when handshake failure rate by client network is unchanged for a full business cycle.

Stage 5. Make the code agile. Application code names a purpose rather than an algorithm, every artifact records its algorithm, and every buffer is variable-length. Exit when an algorithm can be added through configuration.

Stage 6. Re-wrap data at rest under post-quantum key encryption keys. Exit when the archive inventory shows no classical wrapping.

Stage 7. Migrate signatures and certificates, on the longer timetable that their size and coordination cost require. Exit when the classical chain is below a threshold in the telemetry.

Stage 8. Deprecate. Move the old algorithm from preferred, to accepted, to an exception list, to removed. Exit when the code path is deleted.

Failure modes: an inventory that is never completed, so nothing is prioritized. Adding the new algorithm and never removing the old one, which leaves a downgrade target. Attempting key exchange and certificates in one change, which makes the two failure modes indistinguishable. And a protocol that offers both algorithms without authenticating the negotiation transcript.

Decision point: whether crypto-agility is built in stage 5 or skipped. A team that skips it completes this migration and repeats the full cost for HQC, and again for whatever follows.

Playbook 2: Confidential computing adoption

Trigger: a contractual or regulatory requirement that the infrastructure operator cannot access the data, a multi-party computation where neither party will send data to the other, or a model and a prompt that must not see each other.

Prerequisites: a reproducible build, because an attestation policy needs predictable measurements. Start there if it does not exist.

Stage 1. Choose the boundary. A VM-level TEE lifts an existing workload with no code change and trusts the guest kernel. A process enclave gives a smaller trusted base and requires splitting the application. Exit when the choice is recorded with its reasoning.

Stage 2. Minimize what goes inside. Key material, decryption, the sensitive transformation, and result encryption inside. Web server, parsing, logging, metrics, and orchestration outside. Exit when the interface between them is a narrow, versioned, authenticated message contract.

Stage 3. Build the attestation verification path before anything depends on it. Signature chain, revocation, minimum firmware version, policy flags, measurement allowlist, and nonce binding. Exit when a deliberately wrong measurement is refused in a test.

Stage 4. Automate the measurement allowlist from the build pipeline, keyed to the provenance attestation. Exit when a release promotion updates the allowlist with no human step.

Stage 5. Wire secure key release. Ephemeral key inside, hash of its public key in the report, policy on the service side. Exit when a key is released only under a passing attestation, demonstrated by a failing case.

Stage 6. Build and test the break-glass recovery path: a second wrapping under a separately controlled offline key, with its own approval process. Exit when the team performed a recovery in a drill.

Failure modes: lifting a whole application into a confidential VM and getting the compliance checkbox with little security property. A permissive policy adopted because measurement churn broke a manual allowlist. Omitting the nonce binding, which allows replay of a captured report. And no recovery path, which turns a lost image into permanent data loss.

Decision point: whether stage 6 happens before production. A confidential system with no tested recovery is a delayed data loss event.

Playbook 3: Supply chain hardening

Trigger: a regulatory requirement, a customer questionnaire, or the observation that you cannot currently answer which running services contain a given component version.

Prerequisites: a single build system for the majority of artifacts. Hardening several divergent pipelines costs several times as much.

Stage 1. Generate a bill of materials from the built artifact, including the base image, and attach it to the artifact as a signed attestation. Exit when every deployed artifact has one, keyed by digest.

Stage 2. Store and re-evaluate continuously against current advisory data. Exit when the question "which running services contain this component at this version" is answered in seconds.

Stage 3. Add reachability analysis before adding a second scanner. Exit when the finding count that reaches an engineer falls by an order of magnitude.

Stage 4. Emit provenance from a hosted build service, reaching SLSA level 2, then harden to level 3 so that the build cannot forge its own provenance. Exit when provenance exists for every artifact digest.

Stage 5. Adopt keyless signing with a workload identity. Exit when no long-lived signing key exists.

Stage 6. Enforce at admission: signature valid, identity policy satisfied including the workflow path, and provenance present. Exit when an artifact signed by the wrong identity is refused in a test.

Stage 7. Make builds reproducible: normalize timestamps, strip paths, sort every emitted list, and pin the build environment. Exit when two independent builds of the same commit produce identical digests.

Stage 8. Add runtime controls: seccomp, dropped capabilities, read-only root, default-deny egress, and detection rules in reporting mode first.

Failure modes: a bill of materials generated from the manifest, which omits vendored and statically linked code. A policy gate set at a threshold the organization will not hold, which trains everyone that gates are advisory. Checking that a signature is valid without checking who signed. And enabling blocking runtime rules before a full business cycle of detection.

Decision point: whether the identity policy in stage 6 names the workflow path. Without it, an attacker with repository write access adds a workflow that builds a malicious artifact from legitimate source, and every check passes.

Playbook 4: Agentic system to production

Trigger: a prototype agent works in a demonstration and someone proposes exposing it to users or giving it write access.

Prerequisites: a durable execution engine, because an agent loop in process memory loses everything on restart after its tools already had side effects.

Stage 1. Replace the free-form loop with an explicit state graph. Named states, guarded transitions, per-state timeouts, and every path reaching a terminal state. Exit when a test suite drives every transition with a stubbed model.

Stage 2. Put the budgets in the graph: step limit, wall-clock limit, token limit, and a bounded replan count. Exit when exceeding each one is demonstrated to reach a terminal state.

Stage 3. Classify every tool by risk and apply the matching control. Exit when every irreversible action passes through an approval state.

Stage 4. Scope the credentials. The agent acts with the authority of the end user, never with a service account that can reach everything. Exit when a tool called on behalf of one user cannot reach another user's data, demonstrated by a test.

Stage 5. Sandbox code execution behind a hypervisor boundary, with no network, no credentials, no persistence, and a fresh instance per session. Exit when a deliberately hostile program in the sandbox reaches nothing.

Stage 6. Instrument per step: model and version, token counts, latency, cost, tool and arguments, and the semantic outcome. Exit when a bad run can be diagnosed from telemetry alone.

Stage 7. Build the three evaluation layers: stubbed unit tests, an offline case set scored against a rubric, and online sampling that over-samples budget limits, low retrieval scores, and immediate user rephrasing.

Stage 8. Run the adversarial review. Assume the model is fully controlled by an attacker and enumerate what it reaches.

Failure modes: budgets stated in the prompt rather than enforced in the graph. A shared service account, which turns one prompt injection into a full data breach. A pooled sandbox carrying a previous user's files. Cost discovered in a monthly bill rather than in a per-run budget. And a system that only has the first evaluation layer, which appears to work until a user says otherwise.

Decision point: stage 4. Everything else is recoverable, and an agent with a broad service account is one persuasive document away from an incident.

Playbook 5: Internal developer platform

Trigger: engineers spend a meaningful fraction of their time on infrastructure rather than on product, or every team has built a different deployment pipeline.

Prerequisites: a platform team with a product owner rather than an operations queue, and a mandate to say no.

Stage 1. Measure before building. Instrument the full commit-to-production path per stage, and survey engineers on where the friction is. Exit when the top three delays are identified with evidence, which is usually not where the team assumed.

Stage 2. Identify the dominant service shape and build one golden path for it. Template, pipeline, runtime defaults, and task-oriented documentation. Exit when a new service of that shape reaches production in under an hour.

Stage 3. Make the abstraction declarative and reconciled, with a visible escape hatch. Exit when a team with an unusual requirement is served rather than blocked.

Stage 4. Generate the service catalog from platform state rather than asking teams to maintain a page. Exit when the catalog is accurate without human effort.

Stage 5. Add ephemeral environments with time-to-live sweepers, hard quotas, and daily reconciliation against open pull requests. Exit when no orphaned environment survives a week.

Stage 6. Add scorecards in reporting mode. Checks that are automatic, owned, and fixable. Exit when teams improve their scores without being asked.

Stage 7. Tie enforcement to the level of change rather than to a global cutoff. Exit when a below-threshold service is blocked from a new critical dependency rather than from deploying.

Stage 8. Track escape hatch usage as a feature backlog, and convert anything done manually three times into self-service.

Failure modes: building the portal before the control plane, giving a directory of links that goes stale. A golden path that is safer and slower, which loses. A scorecard that blocks deployment on day one, which gets an exemption process within a week. And the platform team becoming the operations queue it was created to remove, visible as a growing ticket backlog.

Decision point: stage 1. A platform team that skips measurement optimizes the part it owns, and the largest delays are usually process decisions elsewhere.

Playbook 6: Efficiency and carbon reduction

Trigger: a cost target, a sustainability commitment, or an accelerator fleet whose utilization nobody measured.

Prerequisites: per-workload resource metrics with at least a quarter of history.

Stage 1. Measure requested against used, per workload, and publish the ratio per team. Exit when the ratio is visible to the teams that own it, because visibility alone closes a surprising share of the gap.

Stage 2. Right-size. Set requests from the p95 or p99 of measured use with headroom, memory request equal to memory limit, and omit CPU limits unless a hard cap is required. Exit when the fleet-wide ratio is below two.

Stage 3. Automate it with vertical autoscaling, recommendation mode first. Exit when new workloads are right-sized without a human estimate.

Stage 4. Separate node pools by scheduling policy so that batch packs and stateless services spread. Exit when batch node utilization rises without resilience falling.

Stage 5. Move development environments, internal tooling, and long-tail services to scale-to-zero. Exit when idle cost falls and the cold start is acceptable to the users of those services.

Stage 6. Add deadlines to batch workloads so that a carbon-aware scheduler has something to work with, then shift flexible work into low-intensity windows. Exit when the scheduler places work by forecast rather than by cron.

Stage 7. Classify load as critical, interruptible, or deferrable, which is the prerequisite for responding to a grid signal at facility scale.

Stage 8. Storage and telemetry. Measure which log streams and dashboards are actually queried, set retention by value, attach lifecycle rules at write time, and test the deletion path.

Failure modes: reporting a total rather than a rate, so growth looks like a regression and vice versa. Omitting embodied emissions, which for lightly used hardware is the larger share. Archiving millions of small objects into a tier with a minimum billable size, which costs more than keeping them. And a deletion path that has never been exercised, first tested under a regulatory deadline.

Decision point: stage 2. Right-sizing is consistently the largest immediate reduction and it is a measurement problem rather than an engineering one. A programme that starts with algorithmic optimization spends months to achieve less than one week of right-sizing achieves.

The one rule under all six

Every stage above has an observable exit condition rather than a date. That is deliberate, and it is the single practice that separates a programme that finishes from one that is reported as on track for two years.

Write the condition down before the stage starts. Measure it. And when a stage cannot reach its condition, treat that as information about the plan rather than as a reason to move to the next stage anyway.

Appendix C of Emerging Systems Engineering, revision 1.0. Attribution and sole author credit: Ayoub Zulfiqar.