RFC: API Change: Flash: New API to access flash information

[de-nordic]

Introduction

Introducing API change that allows to define "virtual" flash devices over real flash devices.

Currently some of flash API calls, like the flash_api_pages_layout, often propagate pointers to structures directly defined at driver level; this makes it hard to setup "virtual" device that could lie about the information returned by such API call as it may require run-time allocation of resources as the "real-device" can never be assumed, which is not possible.
Aim of this RFC is to discuss changes to API that would prevent direct access to layout and this way make it easier to generate "virtual" devices.
Such virtual devices would allow defining devices that could serve as replacement for flash map API or direct encryption devices, with no need to special API layer for them, as basically any function or utility could use flash API directly, without knowing whether it is real device or "virtual".

Problem description

The flash_api_pages_layout , when enabled with CONFIG_FLASH_PAGE_LAYOUT, gives direct access to array that defines layout of flash, the user code can iterate over for gathering information; additionally there are flash_get_page_info_by_offs, and flash_get_page_info_by_idx that utilize this access.
With such direct access virtual device is not able to provide own definition of layout.

Proposed change

Proposed change is to replace the optional flash_api_pages_layout, with set of following API calls:
flash_get_page_info, flash_get_page_count, flash_get_size that would allow to quick obtain the same information but does not give direct access to information from driver.
The missing functionality of flash_get_page_info_by_offs and flash_get_page_info_by_idx can be easily added with use of the above function, even in form of utility library for flash access

Proposed change (Detailed)

Proposed addition to flash API calls:

int flash_get_page_info(const struct device *dev, off_t offset, struct flash_page_info *fpi);
ssize_t flash_get_page_count(const struct device *dev);
ssize_t flash_get_size(const struct device *dev);

note that the last two calls would be very quick as they will be given literal value during compilation.
Additional additions to struct flash_parameters would be max_page_size, which is maximum page size on the device (or any in case of uniform devices), and flags which will hold flags; currently there will be only one flag defined FDF_NON_UNIFORM_LAYOUT that means that the device has pages of different size.

With such definitions, in contrary to flash_api_pages_layout, any "virtual" driver that would be able to query the information from underlying driver, process it if needed, and return to caller.

Dependencies

Anything that uses flash_api_pages_layout will not be able to use "virtual" devices.
All flash drivers will require some patching.

Concerns and Unresolved Questions

Initialization order impact, power management (regarding the "virtual devices", device readiness).

Alternatives

There is no such attempt to flatten the API layers, where flash access is considered.

Related PRs

Proposed flash API change: #36484
Demo of flash partition device (formerly flash area device) that is possible due to change: #36487

[de-nordic]

FYI @nvlsianpu , @jfischer-no , @mbolivar-nordic , @galak , @nandojve .

[Laczen]

Do we really need virtual flash devices ? It seems to be like we would better look at the linux solution for memory technology devices (mtd) and adapt zephyr to the approach of linux. This would result in the flash api being deprecated completely and replaced with an mtd api.

Probably it is also not needed to create a virtual device as a real device, only the "master" devices needs to be a device. Other things like partitions can be (const) structs that are using this master device. A simple approach providing all the required functionality is provided in #34494. This PR does not need any API changes or whatever.

[nvlsianpu]

I was looking into virtual flash device - looks like it does the job of partition management well, it reduces amount API we have.
Can be similar idea be implemented with MTD API?, so we will be able to switch to it smoothly.
I think both ideas doesn't collide each other.

[Laczen]

@nvlsianpu, the main difference between virtual flash devices and mtd API is that virtual flash devices create a device for each partition, while mtd only uses (or creates if we would replace the flash device with a mtd device) for the "master" device. This allows support for repartitioning without any changes. This also results in the compiler removing any unused partitions from the image for the mtd API.

The mtd API could (and should) provide support for regions (like in linux) so that page support is provided from the start. It should also be extended to provide a "mtd-compatible" so that master devices can be registered (this would remove the flash API completely). Also it might be a good idea to provide some way of "linking" the different partitions (using a list) so that for each partition it is possible to find the parent and child partitions.

[carlescufi]

API meeting (6th July): no objections.

[de-nordic]

Update to API implementation:

Because adding the special API calls increases size of flash device structure significantly, the independent API calls to provide back-end for the flash_get_page_info, flash_get_page_count and flash_get_size will not be provided,
instead single API call, flash_api_get_param will be provided that will be taking the opcode for the parameter
to obtain; the increase in access time, to the parameters, should be acceptable as obtaining parameters is not
happening often and the significant reduction of size of code is worth it.
The flash_api_get_parameters and flash_api_get_layout may be also moved under the single flash_api_get_param API call reducing the API switch table even further.

The

int flash_get_page_info(const struct device *dev, off_t offset, struct flash_page_info *fpi);
ssize_t flash_get_page_count(const struct device *dev);
ssize_t flash_get_size(const struct device *dev);

can be implemented as non-syscalls with such approach.

[Laczen]

@carlescufi, strange that API meeting did not detect the above.

@de-nordic, I support that there would be one function entry in an api to access and modify "configurable items". However I think you are duplicating info to avoid syscalls, this is not a good idea.