From 98edac44538b42635feac2a5c9f153dc22439c41 Mon Sep 17 00:00:00 2001 From: deeaitch Date: Mon, 21 Sep 2026 08:55:21 -0400 Subject: [PATCH] Update 02 --- docs/02-update_strategies.md | 868 ++++++----------------------------- 1 file changed, 143 insertions(+), 725 deletions(-) diff --git a/docs/02-update_strategies.md b/docs/02-update_strategies.md index fb766e7..c41d6f3 100644 --- a/docs/02-update_strategies.md +++ b/docs/02-update_strategies.md @@ -1,864 +1,282 @@ # 02. Choosing an OTA Update Strategy: Full Images, Packages, or Something Else? -In the previous chapter, we defined an OTA update as the controlled delivery and application of a new version of software or data to a remote device. +The previous chapter introduced the central problem of OTA: replacing software while preserving a path to recovery. -The next natural question is: - -> How should such an update actually be performed? - -At first glance, the answer seems simple. There are full-image updates, packages, binary deltas, containers, atomic filesystem trees, and other well-known approaches. All that remains is to compare them and choose the best one. - -But this framing starts too late. - -Before choosing a technology, we need to answer a more fundamental question: +The next question is: > **What exactly are we trying to update?** -The entire system? The operating system? A single application? A set of files? Configuration? Certificates? Maps? A machine-learning model? +The entire system? One application? Configuration? Maps? A machine-learning model? -The answer determines almost everything else: the update unit, storage requirements, verification method, activation mechanism, rollback capability, and acceptable downtime. +These targets have different lifecycles. Understanding those differences is more useful than starting with a list of tools. -In addition, some commonly compared approaches describe different aspects of the process. A full image or a package defines **what the update unit is**. An A/B layout defines **where the new version is prepared and how the system switches between versions**. A binary delta defines **how the amount of transferred data is reduced**. A cryptographic signature defines **how the origin of the update is verified**. - -These are not necessarily mutually exclusive choices. - -For example, a device may use: - -- full system images as the update unit; -- an A/B layout for safe activation; -- binary deltas to reduce network traffic; -- cryptographic signatures to verify authenticity; -- a separate mechanism for configuration updates. - -Therefore, choosing an OTA strategy is not selecting one item from a short list. It is the design of several related mechanisms. +This chapter is an introductory guide to the available approaches. The linked Architecture Decision Records (ADRs) explore the conditions under which each choice makes sense. Their examples provide starting points for further study, rather than a complete implementation recipe or a single architecture that every device should follow. --- ## 1. What Exactly Is Being Updated? -OTA is often associated exclusively with device firmware. For a small microcontroller, this may be a sufficiently accurate description: new firmware may indeed replace almost all executable content on the device. - -An embedded Linux system is usually more complex. - -It may include: +An embedded Linux device contains several layers: ```text -Bootloader -Kernel -Device tree -Root filesystem -System libraries -System services -Applications -Configuration -Persistent data -Certificates -Content +Bootloader and hardware support +Operating system and system libraries +Applications and services +Configuration and device identity +Persistent data and content ``` -These components have different lifecycles. +They do not necessarily change together. The operating system may change rarely, an application every month, and configuration whenever operational requirements change. -The kernel may be updated rarely. A user application may be updated every month. Configuration may change several times a day. A certificate may be replaced only before it expires. Maps or machine-learning models may have their own independent release cycle. +### 1.1. The Complete Software System -The first step is therefore to identify the possible update targets. - -### 1.1. The Complete System - -In this case, an update is treated as a transition from one complete device version to another: +An update can represent a transition between complete software releases: ```text -System version N - | - v -System version N + 1 +System version N -> System version N + 1 ``` -The new version may include the kernel, root filesystem, system libraries, services, and applications. +The kernel, libraries, services, and applications are built and tested together. The device receives a predefined software version rather than assembling a release from independently changing components. -The defining property of this approach is that the system is delivered and tested as a whole. The device does not assemble the new version itself from many independently changing components. It receives a predefined state. +Here, “complete” refers to the software included in that release. Device-specific settings and user data still need their own treatment. -This improves reproducibility: +### 1.2. The System Platform -> The system version installed on the device should match the version built and tested by the build system. +The operating system can also be maintained separately from applications. -This is the level at which full-image updates, A/B layouts, recovery partitions, and atomic switching between versions are usually considered. +Think of it as a platform providing the kernel, libraries, drivers, and core services that applications use. A platform update changes that foundation without necessarily releasing new versions of every application. -### 1.2. The Operating System - -Sometimes the system layer must be updated without replacing the entire contents of the device. - -Update targets may include: - -- the kernel; -- system libraries; -- system services; -- drivers and modules; -- individual distribution packages. - -This approach often relies on a package manager and the existing infrastructure of a Linux distribution. - -In this case, the device does not receive a completely prepared new state. It transforms the current system into a new one by installing, removing, or replacing individual components. +The platform may be delivered as an image or a versioned filesystem tree. Distribution packages are another possibility. The target being updated does not, by itself, determine the delivery format. ### 1.3. Applications -In many products, the base platform changes rarely while application software has a much shorter release cycle. +A device may contain a communication service, a web interface, diagnostics, and other applications with different release schedules. -For example: +Updating one independently can avoid a complete system release for a small change. But independence must exist in the software design: the application needs stable interfaces and a clearly supported platform. -```text -Embedded Linux -├── Device manager -├── Communication service -├── Web interface -├── User interface -└── Diagnostics -``` - -An update may affect only one service or application. - -This avoids replacing the entire system for a small application-level change. However, it introduces a new question: is the new application version compatible with the installed libraries, configuration, data schema, and other services? - -The smaller the update unit, the smaller the change, but the larger the number of version combinations that may need to be supported. +Choosing to release an application independently comes before choosing whether to deliver it as a package, a bundle, or a container. [ADR-007](../adr/ADR-007-independent-application-updates.md) explores that boundary. ### 1.4. Data and Content -OTA can update more than executable code. +OTA can also deliver configuration, calibration data, certificates, maps, filtering rules, or machine-learning models. -Examples include: +These have compatibility requirements too. A model may need a particular runtime, and a configuration file may only be understood by certain application versions. -- configuration; -- calibration data; -- certificates and key material; -- maps; -- voice packages; -- dictionaries; -- machine-learning models; -- filtering rules; -- static user-interface resources; -- parameter databases or diagnostic-code databases. - -Such data often cannot reasonably be included in a full system image every time it changes. It may require its own versioning, verification, compatibility, and rollback rules. - -For example, a new machine-learning model may require a specific runtime version. A new configuration may be incompatible with an older application. A new certificate may require coordinated changes on both the client and server sides. - -Therefore, even a data update remains an architectural problem rather than a simple file transfer. - -### 1.5. Multiple Levels at the Same Time - -A real product usually has more than one update unit. - -For example: - -```text -Operating system -> full system image -Applications -> packages or containers -Maps and ML models -> separate artifacts -Configuration -> small atomic documents -Certificates -> dedicated rotation mechanism -``` - -The operating system may be updated several times per year, applications monthly, data weekly, and configuration whenever necessary. - -The question is therefore not: - -> How is the device updated? - -It is: - -> **Which parts of the device are updated, how independently, and how often?** +A small artifact can still have a large effect on the device. Its size does not determine the cost of failure. --- -## 2. What Decisions Make Up an OTA Strategy? +## 2. Different Mechanisms Answer Different Questions -The term *update strategy* is often used too broadly. As a result, full images, A/B layouts, deltas, containers, and signatures appear in the same list even though they solve different problems. +Full images, A/B layouts, containers, deltas, and signatures are often presented as competing strategies. They describe different parts of an update. -It is useful to separate an OTA design into several independent questions. +An **artifact** is the versioned item delivered to a device, such as an image or a package. **Activation** makes its new version operational. **Rollback** returns to a previous version when the new one cannot be used successfully. -### 2.1. Update Unit +| Question | Examples | +|---|---| +| What is the update unit? | A system image, application package, container image, filesystem tree, or data set | +| How is it transferred? | In full, compressed, or as a delta | +| Where is it prepared? | An inactive partition, a separate directory, or the running filesystem | +| How is it activated? | A service restart, a reboot, or a switch to a prepared version | +| How is it verified? | Integrity checks and verification of an authorized source | +| How is a failure handled? | A previous version, a recovery environment, or manual service | -What is the installable artifact? +An **A/B layout** keeps two system slots so a new version can be prepared while the current one remains available. A **binary delta** describes changes needed to reconstruct a new artifact from an existing one. Neither defines which software belongs in a release. -- a full image; -- a filesystem; -- a package; -- a container; -- a data set. +For example, an update can combine: -### 2.2. Delivery Method +- a full system image as the release unit; +- an A/B layout for preparation and activation; +- delta delivery to reduce network traffic; +- signatures to establish authenticity. -How is the artifact transferred to the device? - -- in full; -- compressed; -- as a binary delta; -- in blocks; -- as a stream; -- through a local gateway or intermediate cache. - -### 2.3. Installation Method - -How is the new version prepared? - -- written to an inactive partition; -- applied directly to the running filesystem; -- assembled as a new filesystem tree; -- unpacked into a separate directory; -- imported as a new container image. - -### 2.4. Activation Method - -When does the new version become active? - -- immediately after a file is replaced; -- after a service restart; -- after a device reboot; -- after an atomic symlink or partition switch; -- after confirmation by an external system. - -### 2.5. Recovery Method - -What happens if the update fails? - -- return to the previous partition; -- boot into a recovery system; -- roll back a transaction; -- reinstall the previous package; -- return to the previous filesystem tree; -- require manual service. - -These decisions are related, but they are not the same. - -For example, a full-image update may or may not use an A/B layout. A package update may be transferred in full or as a delta. A container may be only the application delivery mechanism, while the base operating system is updated with a separate system image. +These mechanisms complement one another. Selecting an image format alone does not finish the design of activation or recovery. --- -## 3. Full-Image Updates +## 3. Full-System Image Updates -With a full-image update, the device receives a prebuilt image of the system or one of its major partitions. - -The process usually looks like this: +With this approach, the software included in a system release is distributed as a prebuilt image. ```text -Build system - | - | produces tested image - v -Update server - | - | delivers image - v -Device - | - | writes image - v -New system version +Build and test a system release + | + v + Deliver image + | + v + Install and activate it ``` -Instead of changing individual components sequentially, the device receives a complete version. +The central benefit is a known software state: the image installed on the device is the one validated before release. Dependencies within that image are settled during the build rather than resolved on the device. -### 3.1. Primary Advantage +Even a small change produces a new image release, and the resulting system still needs regression testing. Large images and hardware-specific variants can increase delivery and maintenance costs. -The main advantage is a reproducible state. +An image can also represent only the system platform in a design where applications are updated separately. In that case, its version identifies the platform, not every independently maintained component on the device. -If the image was built and tested as a whole, the device should end up in exactly that state after a successful installation. - -With a correct implementation, this reduces the number of possible version combinations: - -```text -Version A -Version B -Version C -``` - -instead of: - -```text -Kernel A + library B + service C + application D -Kernel A + library C + service C + application D -Kernel B + library C + service D + application D -... -``` - -The more independently updated components a system has, the faster the number of possible states grows. - -### 3.2. Advantages - -- predictable and reproducible system state; -- convenient system-level testing; -- no dependency resolution on the device; -- natural compatibility with an A/B layout; -- relatively clear rollback mechanism; -- the system partition can remain immutable during normal operation; -- easier identification of the exact installed component set. - -### 3.3. Disadvantages - -- large artifact size; -- increased storage requirements; -- long downloads over slow connections; -- transfer of unchanged data; -- user data and configuration require separate handling; -- a small application change may require releasing a new system image. - -### 3.4. Typical Uses - -This approach is especially natural for appliance-like devices where the entire system is treated as one product: - -- industrial controllers; -- network devices; -- automotive control units; -- medical devices; -- terminals; -- specialized embedded Linux systems. - -It is particularly attractive where reliability and reproducibility matter more than minimizing update size. +[ADR-004: Full-System Image Updates](../adr/ADR-004-full-system-image-updates.md) examines the case where the software stack is maintained as one product, with reproducibility and system-level rollback taking priority over independent component releases. --- ## 4. Package-Based Updates -With a package-based approach, the system is updated by installing or replacing individual packages. +A package groups the files and metadata needed to install a software component. It usually identifies a version and may include dependency information and installation scripts. -These may use standard Linux mechanisms such as: +Packages have many uses in Linux. The example developed in [ADR-005: Package-Based Updates](../adr/ADR-005-package-based-updates.md) focuses on independent application components running on a stable system platform: -- `deb`; -- `rpm`; -- `opkg`; +```text +System platform version A +├── Communication package +├── User-interface package +└── Diagnostics package +``` -or a custom package format created for a specific product. +The platform has its own lifecycle. Application packages are built and validated for that platform, with a corresponding package repository for each platform version. -A package usually contains: +In this model, compatibility is established through architecture and the release process. The device is not expected to discover a suitable application architecture by resolving arbitrary dependency combinations during installation. -- files; -- metadata; -- a version; -- dependencies; -- installation instructions; -- sometimes pre-installation and post-installation scripts. +A fix to diagnostics can therefore be released without replacing the whole system. The cost is maintaining component independence and the builds and repositories needed for supported platforms. -### 4.1. Primary Advantage +The package manager handles package delivery and installation. The OTA manager controls update policy, such as when installation is allowed and which operational conditions must be met. Retaining previous component versions supports downgrade; activation and the handling of application data remain part of the surrounding update design. -Packages allow only the changed components to be updated. - -If one service has been fixed, there is no need to transfer and replace the entire filesystem. - -This is especially convenient when the device is already based on a maintained Linux distribution and its components have relatively independent lifecycles. - -### 4.2. Advantages - -- smaller update size; -- reuse of an existing ecosystem; -- independent component release cycles; -- only required changes need to be installed; -- a familiar model for Linux engineers; -- convenient dependency management when the repository is tightly controlled. - -### 4.3. Disadvantages - -- a large number of possible system states; -- dependence on correct dependency resolution; -- installation scripts may fail after partial execution; -- system-wide atomicity is more difficult; -- rolling back a package does not always restore the previous state; -- data and configuration migrations may be irreversible; -- the result may depend on the initial state of the device. - -The last point is especially important. - -If two devices begin the update from different states, the same sequence of package operations does not necessarily produce exactly the same final state. - -### 4.4. A Package Manager Does Not Exclude OTA - -Using a package manager does not make a mechanism “not real OTA.” - -OTA describes the remote delivery and controlled application of an update. A package manager can absolutely be part of such a system. - -The critical questions are not whether the implementation uses `apt`, `rpm`, `opkg`, or a custom installer, but: - -- who controls the repository; -- how package authenticity is verified; -- which version transitions are allowed; -- what happens if power is lost; -- how partial installation is detected; -- whether a working state can be recovered; -- how every supported update path is tested. - -A package manager alone solves only part of the OTA problem. +The ADR explains this division of responsibility and the conditions that make the approach useful. --- ## 5. Container-Based Updates -A container image may be used as the delivery and execution unit for an application. - -In this model, the base operating system provides a container runtime, while application components are delivered separately. +A container image delivers an application together with its required user-space environment, such as libraries and runtime components. The host supplies the kernel and a container runtime that starts and manages the container. ```text -Host operating system -├── Container A -├── Container B -└── Container C +Host platform +└── Container runtime + └── Application with its user-space dependencies ``` -Updating an application may consist of downloading a new image and switching to it. +[ADR-006: Container-Based Updates](../adr/ADR-006-container-based-updates.md) considers an independently maintained subsystem, especially one supplied by another organization. Packaging its environment with the application reduces dependence on the customer's user-space libraries. -### 5.1. Advantages +This can simplify delivery across compatible platforms. It also adds an environment to maintain and may make the integrator more dependent on the supplier for diagnostics, fixes, and information about the delivered software. -- the application is delivered with a significant part of its dependencies; -- strong component isolation; -- convenient versioning; -- relatively simple return to a previous image; -- reuse of existing build and registry infrastructure; -- applications can have independent release cycles. +Containers do not create stable interfaces or correct tightly coupled components. Nor should container restart be confused with a general guarantee of stronger fault tolerance: containers still depend on the host kernel. -### 5.2. Disadvantages - -- the container runtime becomes part of the trusted platform; -- storage and memory requirements increase; -- another operational layer is introduced; -- application state still requires separate management; -- container updates do not update the kernel, drivers, or base operating system; -- desktop or cloud practices cannot automatically be transferred to a constrained embedded device. - -Containers do not replace OTA architecture. They may become one layer of it. - -For example: - -```text -Base OS -> A/B system images -Applications -> containers -Configuration -> signed documents -``` +In the scenario discussed by the ADR, independent packages are worth considering first when one organization controls the software and components can use the shared platform. Container delivery needs a clear reason for bringing a separate user-space environment. --- -## 6. Atomic Filesystem Trees and Versioned System States +## 6. Versioned Filesystem Trees -Between full images and traditional packages are approaches in which the system is represented as a versioned filesystem tree. +A filesystem tree is the set of files and directories that makes up a software version. Instead of shipping a complete block image or modifying the running system package by package, an updater can prepare a new tree and select it for use. -A new version is assembled separately, after which the device atomically switches to it. Unchanged objects may be reused, so a fully independent image does not always need to be transferred or stored. +```text +Current tree A remains available + | + v + Prepare new tree B + | + v + Activate prepared tree B +``` -Concepts in this class include: +Some systems reuse unchanged content between versions. For example, OSTree provides versioned filesystem deployments and atomic upgrades: the managed software is switched as a prepared version rather than exposed as a partially installed set of files. Its [overview](https://ostreedev.github.io/ostree/introduction/) is a starting point for exploring this model. -- content-addressed storage; -- immutable filesystem trees; -- snapshot-based deployments; -- OSTree-like models. +This still requires a way to build releases, manage retained versions, and integrate activation with the platform. Data outside the versioned software tree needs separate handling. -The central idea is: - -> The update is neither an in-place set of file changes nor necessarily a complete block image. It is a complete versioned filesystem state. - -### 6.1. Advantages - -- atomic switching between versions; -- reproducible system state; -- reuse of unchanged content; -- convenient rollback to a previous tree; -- fewer changes to the running system; -- a useful compromise between images and packages. - -### 6.2. Disadvantages - -- more complex storage model; -- additional build-tooling requirements; -- garbage collection and multi-version management; -- state outside the managed tree requires separate handling; -- integration with the bootloader and early boot may be non-trivial; -- the approach may be excessive for a simple device. - -This mechanism can be attractive, but its benefits appear only when the entire system lifecycle is genuinely designed around immutable, versioned states. +[ADR-009: Versioned Filesystem Trees](../adr/ADR-009-versioned-filesystem-trees.md) explores when maintaining complete software states with content reuse justifies that additional machinery. --- -## 7. Binary Delta Updates +## 7. Binary Delta Delivery -A binary delta contains the difference between a known old version and a new version rather than the complete new artifact. - -Conceptually: +A binary delta describes how to reconstruct a new artifact using a particular old artifact: ```text Old artifact + Delta = New artifact ``` -This may significantly reduce the amount of transferred data, especially when only a small portion of the artifact has changed. +For example, a device may already hold most of the content of the next system image. Sending a delta can avoid transferring all of that content again. -However, a delta is not a separate answer to the question of what is being updated. +The release unit is still the image. A delta changes how it reaches the device, not what the release contains or how its correctness is established. -It may be applied to: +The saving has a cost. The updater needs the expected source content, resources to reconstruct the result, and a supported path when that source is unavailable. Different starting versions may require different deltas. -- a system image; -- a partition; -- a package; -- a container layer; -- an individual file; -- a model or data set. - -It is therefore more accurate to treat a binary delta as a **delivery optimization**, not as an update unit. - -### 7.1. Advantages - -- lower network traffic; -- faster download over slow connections; -- reduced mobile or satellite data cost; -- the ability to update large artifacts through a constrained channel. - -### 7.2. Disadvantages - -- the delta depends on a specific source version; -- multiple update paths may need to be stored or generated; -- corruption of the source artifact may make application impossible; -- the device requires resources to reconstruct the new version; -- both server-side and device-side complexity increase; -- the delta must be verified as strictly as the full artifact; -- savings may be small when changes are poorly localized. - -For example, updating from version `1.0` to `1.1` may require one delta, while updating from `0.8` to `1.1` requires another. - -With many supported versions, update paths multiply: - -```text -0.8 -> 1.1 -0.9 -> 1.1 -1.0 -> 1.1 -1.0 -> 1.0.1 -> 1.1 -``` - -The more possible paths there are, the harder they are to create, test, and maintain. - -Reduced network traffic is therefore purchased with additional system complexity. +[ADR-008: Binary Delta Delivery](../adr/ADR-008-binary-delta-delivery.md) considers this optimization when bandwidth or download time is important enough to justify the added work. --- ## 8. Updating Individual Files and Data -The smallest update unit is an individual file or a small group of files. +Some updates are best understood as replacing a versioned data artifact rather than installing software. -This is a natural approach for: +For example, a device may download a new map independently of the navigation application. The application must understand its format, and an interrupted download should not leave it using an incomplete map. -- configuration; -- certificates; -- rules; -- calibration data; -- static content; -- models; -- maps; -- user-interface resources. +Useful questions include: -At first glance, such an update seems simple: download a file and replace the old one. +- Which application versions can use this data? +- How is the received artifact verified? +- When does the application start using it? +- What remains available if replacement fails? -A reliable implementation still needs to answer: +The same questions apply to smaller files, but their answers differ. Configuration may need coordinated activation, while certificates may require a dedicated rotation process rather than an ordinary return to an older file. -- how the version is checked; -- how authenticity is verified; -- how integrity is verified; -- how compatibility is verified; -- how a partially written file is avoided; -- how replacement is made atomic; -- what happens if power is lost; -- how the previous version is preserved; -- who confirms successful activation. - -Even a small file can render the device unusable if it contains critical configuration. - -Artifact size does not determine the cost of failure. +The important distinction is between the delivered data and the software that consumes it. Their release schedules can differ, but their compatibility still matters. --- ## 9. Hybrid Strategies -In most complex products, the most reasonable solution is a combination of several mechanisms. +A hybrid strategy assigns different update mechanisms to parts of a device with different needs. For example: ```text -Bootloader and base OS - -> signed full images - -> A/B installation - -> reboot-based activation - -Applications - -> packages or containers - -> independent release cycle - -Maps and ML models - -> separate artifacts - -> optional delta delivery - -Configuration - -> small signed documents - -> atomic replacement - -Certificates - -> dedicated rotation protocol +System platform -> system image +Applications -> independent packages +Maps -> separate data artifacts +Configuration -> versioned configuration updates +Certificates -> dedicated rotation mechanism ``` -This allows different properties to be applied to different components. +This example does not require every product to use all of these channels. Each additional channel needs a reason, such as a different release schedule or maintenance owner. -For the base system, the priority may be reliability and reproducibility. For maps, it may be minimal network traffic. For applications, rapid release. For certificates, strict coordination and limited validity periods. +The combination also needs coordination. If a new map requires a newer application, separate downloads do not make the two versions compatible automatically. -### 9.1. Advantages - -- each data type receives an appropriate mechanism; -- the entire system does not need to be updated for a small change; -- reliability, bandwidth, and release speed can be optimized independently; -- the design reflects the real lifecycle of each component. - -### 9.2. Disadvantages - -- multiple mechanisms must be designed, tested, and maintained; -- compatibility modelling becomes more complex; -- identifying the complete product version becomes more difficult; -- dependencies may exist between separate update channels; -- rolling back one component may require rolling back others; -- observability and diagnostics become more complex. - -A hybrid strategy must not become an accidental collection of unrelated mechanisms. - -Each independent update channel should exist for a concrete reason: - -- a different lifecycle; -- a different transfer cost; -- a different cost of failure; -- different activation requirements; -- a need for organizational independence. - -Without such a reason, an additional mechanism only increases system complexity. +[ADR-010: Hybrid Update Strategies](../adr/ADR-010-hybrid-update-strategies.md) explores how to choose useful boundaries without turning the device into an unrelated collection of update mechanisms. --- -## 10. Why There Is No Best Strategy +## 10. Which Questions Lead to a Useful Choice? -The question: +There is no universal ranking of these approaches. A useful starting point is to describe the constraints in ordinary language: -> Which OTA strategy is best? +| Question | Why it matters | +|---|---| +| Which parts change together? | Components released together may belong in one update unit. | +| Which parts need independent releases? | Separate lifecycles may justify application or data updates. | +| Who maintains each part? | An external supplier may need to deliver its own environment. | +| What happens if installation fails? | Recovery requirements influence preparation and activation. | +| How much storage and memory are available? | Keeping old and new versions or reconstructing artifacts consumes resources. | +| How expensive or slow is the connection? | Delivery optimizations may justify their maintenance cost. | +| Can the device reboot or restart a service? | Acceptable downtime influences activation. | +| How long must updates remain supportable? | Old versions, hardware variants, and tooling all create maintenance work. | -has no universal answer. - -A full image may be an excellent choice for an industrial controller and too expensive for a device using satellite connectivity. - -Packages may fit naturally into a Linux gateway and create an unacceptable number of states in a safety-critical system. - -Containers may simplify independent application releases while adding a pointless operational layer to a small device. - -A binary delta may dramatically reduce transfer cost while introducing more complexity than the rest of the updater. - -The correct question is: - -> **What constraints does our system have, and which update properties matter most?** - -### 10.1. Reliability - -What happens if power is lost in the middle of installation? - -Must the device always return to the previous version? - -Is manual recovery acceptable? - -The more expensive physical access is, the more valuable atomic installation and automatic rollback become. - -### 10.2. Storage Capacity - -Is there enough space for two complete system images? - -Can the device store the downloaded artifact while also constructing the new version? - -How many previous versions must be retained? - -Limited storage may exclude some approaches or require a more complex streaming installation process. - -### 10.3. Connectivity Cost and Quality - -Does the device use Ethernet, Wi-Fi, mobile, or satellite connectivity? - -Is the connection stable or frequently interrupted? - -Is traffic expensive? - -Can downloads safely resume? - -Over an expensive link, the complexity of delta updates may be justified. On a cheap local network, a full image may be simpler and cheaper to operate. - -### 10.4. Update Frequency - -Is the system updated once per year or several times per day? - -Rare system updates and frequent data updates naturally require different mechanisms. - -The more frequently a component changes, the more valuable an independent release cycle becomes. - -### 10.5. Component Independence - -Can the components really be updated independently? - -Are their interfaces stable? - -Is compatibility between versions maintained? - -Independent delivery does not imply independent operation. If application version `5` requires library version `3`, the two must still be coordinated. - -### 10.6. Cost of Failure - -What happens if an update is unusable? - -Does the device temporarily lose a minor feature? Stop a production line? Make a vehicle unavailable? Require a technician visit? - -The same technical failure may have completely different consequences in different products. - -### 10.7. Downtime Requirements - -Can the device be rebooted? - -How long may version activation take? - -Is restarting one service acceptable? - -Some systems may update overnight with a full reboot. Others must preserve their primary function almost continuously. - -### 10.8. Support Lifetime - -How many years will the device remain in service? - -Must the system support updates from very old versions? - -Will a team still be available to maintain a complex mechanism ten years from now? - -The OTA architecture must not only be technically possible today. It must remain maintainable throughout the entire product lifecycle. - -### 10.9. Team Capabilities - -A complex solution does not become a good solution merely because it is technically elegant. - -The team must be able to: - -- build artifacts; -- test update paths; -- operate server infrastructure; -- investigate failed updates; -- manage cryptographic keys; -- maintain compatibility; -- recover devices. - -If potential traffic savings require a mechanism the team cannot operate reliably, the optimization may increase the total cost of the product. +For a beginner, answering these questions is more valuable than choosing an OTA framework immediately. They give a vocabulary for understanding why different products use different designs. --- -## 11. Strategy Is a Consequence of Constraints +## 11. Where to Explore Further -OTA selection should not begin with a technology name. +The ADRs develop the choices introduced here. They are conditional educational examples: each describes a situation, a decision, and the consequences someone would accept in that situation. Read the relevant ADR for the detailed reasoning behind this chapter's shorter explanation. -It should begin with a model of the product: +| Question to explore | Further reading | +|---|---| +| When should the software stack be released as one image? | [ADR-004: Full-System Image Updates](../adr/ADR-004-full-system-image-updates.md) | +| When do independent packages fit a stable platform? | [ADR-005: Package-Based Updates](../adr/ADR-005-package-based-updates.md) | +| When does a subsystem need its own delivered environment? | [ADR-006: Container-Based Updates](../adr/ADR-006-container-based-updates.md) | +| When should an application have a separate release lifecycle? | [ADR-007: Independent Application Updates](../adr/ADR-007-independent-application-updates.md) | +| When is reduced transfer size worth delta processing? | [ADR-008: Binary Delta Delivery](../adr/ADR-008-binary-delta-delivery.md) | +| When should software be managed as versioned filesystem trees? | [ADR-009: Versioned Filesystem Trees](../adr/ADR-009-versioned-filesystem-trees.md) | +| When should several update mechanisms be combined? | [ADR-010: Hybrid Update Strategies](../adr/ADR-010-hybrid-update-strategies.md) | -```text -What is updated? - | - v -How often is it updated? - | - v -How expensive is failure? - | - v -What resources are available? - | - v -What recovery guarantees are required? - | - v -Which mechanism satisfies these constraints? -``` +These questions are related rather than mutually exclusive. An independently released application can use packages, and a system image can use delta delivery within a hybrid design. -Therefore: - -> **An OTA strategy is a consequence of system constraints, not a choice of the newest or most popular technology.** - -The engineering process should look approximately like this: - -1. Identify every independently updateable component. -2. Describe the lifecycle of each component. -3. Determine the cost of a failed update. -4. Record storage, memory, power, and connectivity constraints. -5. Define atomicity, activation, and rollback requirements. -6. Select an update unit for each component. -7. Only then choose specific technologies and tools. - -The order matters. - -If the process starts with a tool, the architecture will be forced to fit the tool's capabilities. If it starts with constraints, the tool can be selected for the actual problem. - ---- - -## 12. Preliminary Comparison - -The following table is not yet a complete decision matrix. It only summarizes the primary properties of the approaches discussed in this chapter. - -| Approach | Primary Unit | Main Strength | Main Cost | -|---|---|---|---| -| Full image | System or partition | Reproducibility and predictable rollback | Artifact size and storage requirements | -| Packages | System component | Independent component updates | Many possible states and difficult rollback | -| Application | Individual service or bundle | Fast independent releases | Platform compatibility | -| Container | Application with dependencies | Isolation and controlled delivery | Additional runtime and operational complexity | -| Filesystem tree | Complete filesystem state | Atomicity with content reuse | More complex storage model | -| Binary delta | Difference between artifact versions | Reduced network traffic | Dependence on the source version | -| Individual files | Configuration or data | Small and targeted changes | Atomicity and compatibility must be designed explicitly | -| Hybrid model | Multiple update units | Optimization for different lifecycles | Overall system complexity | - -There is intentionally no *best option* column. - -The same approach may be strong or weak depending on context. - ---- - -## 13. What Comes Next - -This chapter describes the space of possible solutions, but it does not declare a single winner. - -The next step will be a series of Architecture Decision Records in which each option is evaluated in the context of specific constraints. - -Each ADR should answer not: - -> What is this technology? - -but: - -> **Under which conditions would I choose it, and which consequences of that decision would I be prepared to accept?** - -The following topics will be considered separately: - -- when to choose full-image updates; -- when to choose package-based updates; -- when to update individual applications; -- when to use containers; -- when binary deltas are justified; -- when versioned filesystem trees are appropriate; -- how to design a hybrid strategy. - -The results will then be summarized in a decision matrix. - -The matrix should not select a solution automatically. Its purpose is to show which constraints push the architecture toward a particular approach and where trade-offs begin to conflict. - ---- - -## Conclusion - -An OTA update cannot be reduced to a choice between full images and packages. - -A real system must separately define: - -- what is updated; -- how independently it is updated; -- how it is delivered; -- where it is prepared; -- how it is activated; -- how it is verified; -- how it is rolled back; -- what happens when something fails. - -Full images, packages, containers, filesystem trees, and individual files define different units of system management. A/B layouts, deltas, signatures, integrity checks, and recovery mechanisms add the properties required around those units. - -A good OTA design therefore does not begin with a technology. - -It begins with constraints, the product lifecycle, and an honest answer to the question: - -> **What device state must we guarantee after every possible update outcome?** +Start with what changes and why it needs to change independently. Then explore the mechanisms that fit those boundaries, and the activation and recovery behavior they require.