ota-reference-design/docs/06-atomic-updates.md

06. Atomic Updates: Preparation, Activation, and Interrupted Operations

Chapter 05 described where software, device data, and recovery state live. Now we need to decide in which order to change them.

The main question is simple: if power stops between two operations, what will the device do when it starts again?

We will use the same A/B system and independent service examples. They show possible designs, rather than rules for every platform.

1. What Does Atomic Mean?

An atomic operation has one visible result: the old state or the new state. A reader must not see a partly applied result.

An OTA update includes many operations. Downloading, writing software, changing boot state, and checking the new system do not become one atomic operation just because the device has two slots.

It helps to separate three questions:

Question	Mechanism in this A/B example
Can the installer prepare B without changing running system A?	Write only to the inactive slot.
Can boot control select a complete system after an interrupted state change?	Use a defined state storage and recovery method.
Does B work on this device?	Trial boot, local checks, and confirmation.

Activation means allowing the new version to start. Confirmation means accepting it after checks. They are different events. B can be complete and safe to try, yet still fail at startup.

The aim is to make each visible switch safe and to define recovery between switches.

2. Prepare B While A Remains Available

Assume A is confirmed, B is inactive, and no earlier trial is pending. The installer has exclusive control of B. Shared boot code and persistent application data are outside its write target.

Before writing B, check the update policy, target hardware, available space, and trusted update information. Then:

1. Save the installation intent, including the release and target slot.
2. Write the new software to B. Keep A as the normal boot choice.
3. Complete the platform's required storage writes and check their results.
4. Verify the installed contents against the expected release.
5. Save that this release is prepared in B.
6. Ask boot control to save a trial request for B.
7. Save the local result of that request for later reporting.

The expected release information must come from a trusted source. Comparing two untrusted hashes does not establish trust. Signature policy will be covered with security.

A check of the downloaded file alone does not prove that writing B succeeded. Define how the installed contents are checked. If the installer decompresses or changes the artifact, the check must match the resulting format.

The platform must also define when writes have reached persistent storage. A read from a cache can match the expected bytes while required storage work is still pending.

If preparation fails, keep B unavailable for trial. Preserve A and enough state to retry or stop safely. Do not replace a slot that is running, selected for a pending trial, or still required for recovery.

This ordering is described in ADR-018: Prepare and Verify Before Activation.

3. The Activation Boundary

In this example, boot control owns the trial target, attempt count, and confirmation. The OTA manager uses its interface; it does not maintain a second boot selector in a progress file.

Before boot control accepts B, the installed release and the state needed to try it must already be saved. A successful request means that boot control has saved the request according to its storage contract. It does not mean that B is confirmed.

If the request returns an error or its result is lost during a restart, the OTA manager must read boot state before retrying. An error does not always prove that nothing changed.

Preparation and activation sequence — PlantUML source

There is a gap between steps 6 and 7: boot control may have saved the request while the OTA manager still says “prepared.” After a restart, boot control can start B. The OTA manager must accept that saved boot decision and repair its progress record. It must not overwrite B or reset the trial attempt count.

Two separate records do not form one transaction. Safe ordering and recovery rules handle the states left between them.

4. Save State With an Explicit Storage Contract

Durable state is state that remains available after the failures covered by the design. A successful ordinary file write is not enough to establish this property.

For a Linux file, one possible replacement procedure is:

1. Create a temporary file in the same directory as the destination.
2. Write the complete new record and check for errors.
3. Use fsync() on the temporary file and check the result.
4. Rename it over the destination.
5. Use fsync() on the containing directory and check the result.

The Linux fsync manual explains that synchronizing a file does not by itself make its directory entry durable. This procedure needs a filesystem and storage stack that support the required operations. It is not a general recipe for raw flash or bootloader state.

After an interrupted operation, a reader needs a valid old record, a valid new record, or a defined recovery action. Records can include a format version, release identity, generation number, and a checksum. A checksum can help detect damage; it does not prove authenticity or make writes atomic.

Another option is redundant records. Keep the previous valid record while saving a new one, validate both after restart, and apply a defined selection rule. The storage design must explain how interrupted writes affect each copy. Two copies are not sufficient if one erase operation can destroy both.

When no valid boot record remains, use the platform's separate recovery policy. Do not interpret missing or unreadable state as confirmation. Do not invent a normal slot from an OTA progress file that boot control cannot trust.

5. Continue After a Restart

Before any new write, read the records owned by boot control and inspect the target. Check the release identity, rather than only a flag such as ready = true.

State found after restart	Action in this example
A selected; B partly written	Keep A; restart preparation of B from the beginning.
A selected; B contains the expected complete release; progress is old	Verify B again, then save prepared state.
A selected; B prepared; no trial requested	Keep A until policy allows activation.
B trial requested; progress still says prepared	Keep the boot request; repair progress without writing B.
Trial ended or B was confirmed while reporting was interrupted	Read the saved boot outcome; do not create a fresh trial.
Boot state is unreadable	Stop ordinary installation and use the platform recovery policy.

An operation is idempotent if repeating it has the same intended effect as doing it once. Reading state and verifying the same release can usually be repeated. Rewriting all of B can be repeated here only while B is inactive and has no pending trial.

Repeating a trial request is different. If it restores the attempt count, a failed system could receive endless attempts. Read the existing trial and its result before deciding what to do.

Package scripts, database migrations, and external actions also need their own repeat rules. An “installation in progress” flag cannot tell whether such an action completed.

Installation states and restart decisions — PlantUML source

ADR-019: Recover Interrupted Updates From Durable State defines the ownership and recovery rules used here. ADR-017 defines where the records must live.

6. An Independent Service Update

A service package may replace files in the running filesystem. Stopping the service does not make those replacements one atomic operation.

If installation stops halfway, some files or scripts may have changed while others have not. Use the package manager's documented repair procedure before assuming that the previous package can be installed safely. Retain its dependencies, required inputs, and compatible data as described in ADR-015.

For an application that supports it, another design is to prepare a complete version in a separate directory and switch a reference to it. This gives a smaller activation boundary. It still needs rules for saving that reference, stopping and starting the process, and keeping data compatible. A running process may still hold old files open after the reference changes.

Neither design makes database changes or external actions part of the software switch. Define their recovery separately.

7. A C++ Model of Interrupted Installation

The C++17 example models one A-to-B installation. It keeps simulated persistent storage separate from an installer object. Recreating the installer removes its temporary verification result while keeping saved bytes and records.

It covers interruptions after intent, partial writing, complete writing, verification, prepared state, and trial request. It also models three possible results of an interrupted boot-state write: the old record, the new record, or no valid record.

Important checks include:

• B cannot be activated before verification and saved prepared state.
• A remains unchanged during every installation scenario.
• A saved trial request is preserved when progress is old.
• A restart requires fresh verification before a new activation.
• Invalid boot state requires separate recovery.

The model uses memory, expected byte strings, and explicit reset points. It does not implement a durable file store, cryptographic verification, or a real bootloader. It does not simulate interruption inside a storage write. The three boot-record outcomes are supplied assumptions, not evidence that a storage method produces them.

The existing trial boot example continues with attempts, health checks, confirmation, and return to A. The examples are separate programs.

8. Check the Design Against Failures

Failure	Required result
Power stops during writing B	A remains available; B is not selected for trial.
Installed contents fail verification	Stop activation; retain A.
Saving prepared state fails	Do not request activation until saved state is checked.
Saving the trial request is interrupted	Boot control reads a valid record or enters defined recovery.
Trial request is saved but progress is old	Read boot state; do not rewrite B or refresh attempts.
State storage fills up or returns a write error	Stop forward progress and inspect state before retrying.
A package script stops after changing shared data	Follow its defined recovery procedure; software downgrade alone may be insufficient.

Model tests check ordering. Platform tests must also interrupt real writes, resets, and recovery on the chosen storage. Test failures during recovery too. A completed write in a C++ model does not prove that hardware will retain it.

9. What Comes Next?

Preparation keeps incomplete software out of the boot path. Activation gives a complete version a chance to run. Confirmation still depends on checks after startup.

Next, we will look more closely at rollback: when to return to a previous version, what recovery needs, and when that previous version is no longer a usable choice.