init: specify dependencies between init entries

[JordanYates]

Introduce a mechanism to specify dependencies between struct init_entry instances, which includes both SYS_INIT macros and struct device instances. This allows code to specify that devices depend on a certain SYS_INIT instance, or vice-versa.

Removes the old, undocumented, unused mechanism of injecting dependencies to devices via devicetree ordinals.

Being able to specify dependencies between all init entries is progress towards #22545 and #34518

[gmarull]

lgtm, a few minor nitpicks. thanks for the effort!

[gregshue]

describing hardware dependent subsystems.

In multi-function printers, many subsystems are hardware independent, fit together with a tree of dependencies, and some of which are multi-instance (e.g., formatters, GUIs). Solving the initialization order issue for these will cover the needs for hardware-dependent subsystems.

It seems like the definition of "devicetree" is corrupting any notion that it is a viable option for describing hardware dependent subsystems.

Use devicetree to describe software structures is a violation of the design principle of Decompose to Single Responsibility (https://en.wikipedia.org/wiki/SOLID). A different mechanism needs to be created.

[carlescufi]

API meeting:

• @henrikbrixandersen We probably need to solve the whole level problem first instead of patching it. A full solution is required, that supports:
  • Dependencies between software and devices
  • Multi-instance for devices and software
• @andyross mentions this might be fixing the problem at the wrong level: this should probably be on top of SYS_INIT
• @bjarki-trackunit mentions that we should use a tree-like structure (not necessarily DT) that is parsed and then injects the necessary data into the build image
• @teburd also proposes a similar concept where a tree is parsed by a Python script and then the information is used at build time

[gregshue]

@bjarki-trackunit mentions that we should use a tree-like structure (not necessarily DT) that is parsed

The tree-like structure needs to be generated from the dependencies of each component implementation. The component implementation depends on used APIs being provided (not on other components or subsystems). When trying to apply this to a composable system like Zephyr each component describes what APIs it provides and what APIs it depends upon. Explicitly configuring this in an executable-specific tree description is impractical to create and worse to sustain.

All that needs to be provided at the executable configuration level is the instance-to-instance mapping for the components with multiple instances. It may be useful to also indicate any ordering preferences between independent components in support of an agent that runs independent initializations in parallel.

[bjarki-andreasen]

describing hardware dependent subsystems.

In multi-function printers, many subsystems are hardware independent, fit together with a tree of dependencies, and some of which are multi-instance (e.g., formatters, GUIs). Solving the initialization order issue for these will cover the needs for hardware-dependent subsystems.

It seems like the definition of "devicetree" is corrupting any notion that it is a viable option for describing hardware dependent subsystems.

Use devicetree to describe software structures is a violation of the design principle of Decompose to Single Responsibility (https://en.wikipedia.org/wiki/SOLID). A different mechanism needs to be created.

Okay, zephyr has subsystems for shell and console which are hardware dependent. To get a reference to the specific device instance to use for the subsystem, the devicetree is used. The devicetree has a predefined node named chosen for this. chosen is not part of the DT schema, and is not describing hardware, it is a binding between a submodule and a device instance. The same is the case for partitions, partitions are not hardware, they are software constructs, which are then referenced by another chosen node, zephyr,flash, entirely software subsystem pointing to entirely software construct, in the devicetree.

    chosen {
        zephyr,shell = &uart0;
        zephyr,flash = &flash0;
        zephyr,code-partition = &slot1_partition;
    };

This puts an inherent constraint on the subsystems, only one subsystem instance possible pr chosen node. It is not a far stretch to rewrite the chosen node to a more honest graph, which allows for multiple instances of subsystems, as you demanded.

    shell {
        compatible = "zephyr,shell";
        uart = &uart0;
    };
    flash {
        compatible = " zephyr,flash";
        flash = &flash0;
    };
    code {
        compatible = "zephyr,code-partitionl";
        code = &slot1_partition;
    };

It has already been decided that this was the way to link hardware and software in a system with a static kernel and hardware; a tree like structure which shares hardware and software constructs which shares their dependencies.

I don't think introducing SOLID into this discussion entirely appropriate, given it is intended for object oriented software. I prefer KIS(Superman). Why introduce multiple mechanisms spread across the system to solve an issue that is naturally described with a tree like structure, in which all of the systems must be able to cross reference each other . If you really insist on SOLID, the single responsibility of the tree is to graph the subsystems and their dependencies (and yes, devices are just subsystems tied to hardware, they are otherwise functionally identical).

[gregshue]

Okay, zephyr has subsystems for shell and console which are hardware dependent.

I disagree. They depend on a driver API, but the API could be implemented through a purely software construct. The lowest layer UART driver is "hardware" dependent (where "hardware refers to an execution agent interfacing through the memory map and processing in parallel to the executable). The vast majority of Zephyr code is intentionally hardware independent. Just look at how much code we are testing independent of hardware or hardware simulation.

I don't think introducing SOLID into this discussion entirely appropriate, given it is intended for object oriented software.

To be composable, scalable, and extensible the Zephyr architecture/ecosystem needs to and largely does follow object-oriented concepts (from https://en.wikipedia.org/wiki/Object-oriented_design#Object-oriented_concepts):

• Object/Class: The components themselves are responsible for configuration/allocation and hiding of their own OS constructs including threads, mutexes, and queues. The Zephyr API Design guidelines support multi-instance of the API provider and multi-instance of the API caller.
• Information hiding: This is largely achieved by the avoidance of globally scoped data.
• Inheritance: Though not provided yet, the ability to wrap/extend a driver implementation has already been requested.
• Interface: By putting subsystem/driver interface definitions in a separate tree from the implementation and not including the implementation source tree in the general include paths Zephyr is already promoting (enforcing?) programming to an interface. Also, the test suites are testing to interfaces.
• Polymorphism: There are two types of polymorphism. One is holding any/all of the possible subtypes (not generally available). The other is providing alternate interfaces to an object. Any component providing an interface in addition to SYS_INIT is providing alternate interfaces. (BTW, the Composite reuse principle states that "classes should achieve polymorphic behavior and code reuse by their composition rather than inheritance from a base or parent class" (https://en.wikipedia.org/wiki/Composition_over_inheritance)

A similar pattern of composability and encapsulation occurs when looking at integration with multiple downstream repositories.

The centralized tree-like structure is not composable or extensible. It doesn't meet the platform needs.

devices are just subsystems tied to hardware

I disagree. Devices are hardware. Drivers are runtime-bound interfaces. Subsystems are configurable building blocks of logic + OS constructs + ... that implement and/or use one or more interfaces.

It has already been decided that this was the way to link hardware and software in a system with a static kernel and hardware;

All the design decisions will need to be revisited when Zephyr shifts over to requirements-driven design (needed for safety-critical and secure solutions). I am simply pointing out that the current policy does not meet the scalability, composability, and extensibility inherent in the Zephyr Project mission statement.

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.