ESPIDF : Using custom board with esp_panel_board_custom_conf.h

[martinroger]

I am trying to define a custom board out of the sdkconfig because I have to use some vendor init commands and there is a bit of trickery on the IO expander.
I have my main.cpp like so :

#include <stdio.h>
#include "esp_panel_board_custom_conf.h"
#include "esp_display_panel.hpp"
#include "esp_lib_utils.h"


using namespace esp_panel::drivers;
using namespace esp_panel::board;

static const char *TAG = "main";

extern "C" void app_main()
{
    Board *board = new Board();
    assert(board);

    ESP_LOGI(TAG,"Initializing board");
    ESP_UTILS_CHECK_FALSE_EXIT(board->init(),"Board init failed");
    ESP_UTILS_CHECK_FALSE_EXIT(board->begin(),"Board begin failed");
    auto lcd = board->getLCD();
    auto expander = board->getIO_Expander();
    expander->getBase()->printStatus();
    lcd->colorBarTest();

}

and my esp_panel_board_custom_conf.h file has #define ESP_PANEL_BOARD_DEFAULT_USE_CUSTOM (1) correctly set.

However, after compile-flash-monitor, I get an error that the default board configuration is not detected :

I (664) main_task: Calling app_main()
I (664) main: Initializing board
E (674) Panel: [esp_panel_board.cpp:0055](init):
No default board configuration detected. There are three ways to provide a default configuration: 
        1. Use the `esp_panel_board_supported_conf.h` file to enable a supported board. 
        2. Use the `esp_panel_board_custom_conf.h` file to define a custom board.
        3. Use menuconfig to enable a supported board or define a custom board.
E (704) Utils: [main.cpp:0018](app_main): Board init failed
I (714) main_task: Returned from app_main()

Where is my mistake ?

[t0menas]

Hi @martinroger, I'm not a contributer to this repo, just a user.

Do you configure your custom board through idf.py menuconfig or do you have local configuration file copies inside the project?
In term of menuconfig setup It is important that you enable Enable to skip 'esp_panel_board_*.h' under Component config -> ESP Display Panel Configurations and then set Select the default board type to Custom board

In case you have local configuration files make sure you include them to your projects CMakeLists.txt #

I (664) main_task: Calling app_main()
I (664) main: Initializing board
E (674) Panel: esp_panel_board.cpp:0055:
No default board configuration detected. There are three ways to provide a default configuration:
1. Use the esp_panel_board_supported_conf.h file to enable a supported board.
2. Use the esp_panel_board_custom_conf.h file to define a custom board.
3. Use menuconfig to enable a supported board or define a custom board.
E (704) Utils: main.cpp:0018: Board init failed
I (714) main_task: Returned from app_main()

this error you got suggests that either you did not setup idf.py menuconfig correctly or compiler doesn't see your custom board config.

Looking to hear from you

[martinroger]

Hey @t0menas , appreciate the chiming in.
So I tried a lot of different things. Ideally at this stage I would like to use an esp_panel_board_custom_conf.h file because the board I am using is not part of the supported ones and needs a couple of pre-touch macros and some specific vendor ID commands for the RGB configuration. I could technically make a fork of the library, add that board as a supported one and integrate that fork as a component, but I feel it is not the right solution.

For what it's worth, with this screen and under Platformio + Arduino framework, it works like a charm.

My CMakeLists.txt in the main folder looks something like this :

idf_component_register(SRCS "main.cpp"
                    INCLUDE_DIRS    "."
                                    "./include"
                    )

and my configuration file sits in a folder called include, next to the main.cpp file :

My main.cpp is very dumb, and for now I just want to check the IO Expander status and run a simple colorBarTest :

#include <stdio.h>
// #include "esp_panel_board_custom_conf.h"
#include "esp_display_panel.hpp"
#include "esp_lib_utils.h"



using namespace esp_panel::drivers;
using namespace esp_panel::board;

static const char *TAG = "main";

extern "C" void app_main()
{
    Board *board = new Board();
    assert(board);

    ESP_LOGI(TAG,"Initializing board");
    ESP_UTILS_CHECK_FALSE_EXIT(board->init(),"Board init failed");
    ESP_UTILS_CHECK_FALSE_EXIT(board->begin(),"Board begin failed");
    auto lcd = board->getLCD();
    auto expander = board->getIO_Expander();
    expander->getBase()->printStatus();
    lcd->colorBarTest();

}

Note that including the conf file or not changes nothing to the outcome.

In terms of sdkconfig I tried different approaches, from manually re-configuring my board via menuconfig (but it doesn't support the special macros) and I get a weird core panic during the screen initialisation, or an I2C drop... so a separate issue (I think). Given I can get the very same h file to work fine in PlatformIO + Arduino framework, you can imagine my frustration.

I also started by unchecking "Enable to skip "esp_panel_board_*.h", but when you do that the KConfig just hides any choice other than which drivers get compiled or not.

I'm fairly new to ESP-IDF (only one day of confusion and frustration) so I am thinking I am missing something very obvious that prevents the compiler from including the macros in my conf file, but really I can't seem to find what I want in the documentation.
Do you see something I missed ?
EDIT : or is my best way out of this to just create an empty component, plop my conf file in there and set the display library as a required sub-component ?

Hi @martinroger, I'm not a contributer to this repo, just a user.

Do you configure your custom board through idf.py menuconfig or do you have local configuration file copies inside the project? In term of menuconfig setup It is important that you enable Enable to skip 'esp_panel_board_*.h' under Component config -> ESP Display Panel Configurations and then set Select the default board type to Custom board

In case you have local configuration files make sure you include them to your projects CMakeLists.txt #

I (664) main_task: Calling app_main()
I (664) main: Initializing board
E (674) Panel: esp_panel_board.cpp:0055:
No default board configuration detected. There are three ways to provide a default configuration:

1. Use the esp_panel_board_supported_conf.h file to enable a supported board.
2. Use the esp_panel_board_custom_conf.h file to define a custom board.
3. Use menuconfig to enable a supported board or define a custom board.
   E (704) Utils: main.cpp:0018: Board init failed
   I (714) main_task: Returned from app_main()

this error you got suggests that either you did not setup idf.py menuconfig correctly or compiler doesn't see your custom board config.

Looking to hear from you

[t0menas]

Your observation that things tend to just work with arduino framework is absolutely correct, but as systems/projects become more and more complex inevitably one realizes the need for functionality that arduino framework lacks heavily. I'm glad you made the leap forward!
For a heads up, when you'll successfully start your board and display, initially you are likely to see much worse performance from esp-idf compared to arduino framework, because how well arduino is optimized, but that's another topic, I myself am tackling this problem right now.

Now on to your issue. How about you try to add "include/esp_panel_board_custom_conf.h" to your SRCS list in CMakeLists.txt , from my experience including dir under INCLUDE_DIRS is not enough for compiler to fetch your files. your CMakeLists.txt should look something like this

idf_component_register(
    SRCS 
        "main.cpp"
        "include/esp_panel_board_custom_conf.h"

    INCLUDE_DIRS 
        "." 
        "include"
)

also under INCLUDE_DIRS you listed "./include" which i cannot find anywhere to be the correct syntax, try the code above that I wrote

I also started by unchecking "Enable to skip "esp_panel_board_*.h", but when you do that the KConfig just hides any choice other than which drivers get compiled or not.

My bad, what I meant was you probably would like to uncheck "Enable to skip "esp_panel_drivers_conf*.h" but check "Enable to skip "esp_panel_board_*.h" because drivers will be set up by your esp_panel_board_custom_conf.h but you'll have to define your board as custom in menuconfig as this will make the library search for custom config files. That is my understanding of how this works.

or is my best way out of this to just create an empty component, plop my conf file in there and set the display library as a required sub-component ?

I don't think this is optimal solution for this case, because you'll need only a few headers, they are perfectly fine to be in your project dir as, for a lack of a better word, "definitions"

Let me hear from You :)

P.S. I just found out you can use macros (pre-begin/post-begin functions) to run your vendor id's and such things before board/expander/lcd/etc. initialization, take a look at esp_panel_board_custom_conf.h from line 573:

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////// Please utilize the following macros to execute any additional code if required /////////////////
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/**
 * @brief Pre-begin function for board initialization
 *
 * @param[in] p Pointer to the board object
 * @return true on success, false on failure
 */
/*
#define ESP_PANEL_BOARD_PRE_BEGIN_FUNCTION(p) \
    {  \
        auto board = static_cast<Board *>(p);  \
        return true;    \
    }
*/

[martinroger]

Your observation that things tend to just work with arduino framework is absolutely correct, but as systems/projects become more and more complex inevitably one realizes the need for functionality that arduino framework lacks heavily. I'm glad you made the leap forward! For a heads up, when you'll successfully start your board and display, initially you are likely to see much worse performance from esp-idf compared to arduino framework, because how well arduino is optimized, but that's another topic, I myself am tackling this problem right now.

Aye, I can relate... especially that ultimately the objective is to be able to get more performance than with the Arduino framework !

Now on to your issue. How about you try to add "include/esp_panel_board_custom_conf.h" to your SRCS list in CMakeLists.txt , from my experience including dir under INCLUDE_DIRS is not enough for compiler to fetch your files. your CMakeLists.txt should look something like this

idf_component_register(
    SRCS 
        "main.cpp"
        "include/esp_panel_board_custom_conf.h"

    INCLUDE_DIRS 
        "." 
        "include"
)

Didn't seem to change anything unfortunately.

also under INCLUDE_DIRS you listed "./include" which i cannot find anywhere to be the correct syntax, try the code above that I wrote

From my testing, both syntaxes seem to work, but you are right that yours seems more common

I also started by unchecking "Enable to skip "esp_panel_board_*.h", but when you do that the KConfig just hides any choice other than which drivers get compiled or not.

My bad, what I meant was you probably would like to uncheck "Enable to skip "esp_panel_drivers_conf*.h" but check "Enable to skip "esp_panel_board_*.h" because drivers will be set up by your esp_panel_board_custom_conf.h but you'll have to define your board as custom in menuconfig as this will make the library search for custom config files. That is my understanding of how this works.

I actually think you were right the first time ! leaving esp_panel_drivers_conf.h checked, and esp_panel_board_*.h unchecked.

or is my best way out of this to just create an empty component, plop my conf file in there and set the display library as a required sub-component ?

I don't think this is optimal solution for this case, because you'll need only a few headers, they are perfectly fine to be in your project dir as, for a lack of a better word, "definitions"

Let me hear from You :)

P.S. I just found out you can use macros (pre-begin/post-begin functions) to run your vendor id's and such things before board/expander/lcd/etc. initialization, take a look at esp_panel_board_custom_conf.h from line 573:

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////// Please utilize the following macros to execute any additional code if required /////////////////
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/**
 * @brief Pre-begin function for board initialization
 *
 * @param[in] p Pointer to the board object
 * @return true on success, false on failure
 */
/*
#define ESP_PANEL_BOARD_PRE_BEGIN_FUNCTION(p) \
    {  \
        auto board = static_cast<Board *>(p);  \
        return true;    \
    }
*/

Yeah, that's precisely why I wanted to stick to using the header file instead of going through the menuconfig route... my conf file is filled to the brim with those handy macros.

I think I might have cracked a way of doing this, after reading this thread which recommends modifying the project CMakeLists instead, like so :

cmake_minimum_required(VERSION 3.5)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
include_directories(${CMAKE_CURRENT_LIST_DIR}/main/include)
project(right_screen)

And guess what... it works ! (and is also pointing me towards some issues in my conf file, but it's easier to resolve) :

Now waiting to see if @Lzw655 has an even better recommendation on how to do this...

[t0menas]

Brilliant! Glad to see progress! 🎊 Good luck with your project :)

[Lzw655]

Hi guys,

Sorry for the late. This problem has delved into the compilation systems of Arduino and ESP-IDF. Congratulations on becoming an advanced developer!

@t0menas is right. The method of using the configuration file like esp_panel_board_*.h is mainly applicable under the Arduino framework (referenced from LVGL):

1. For the Arduino framework, it requires the compiler to be able to include the directory of esp_panel_.h*. Since esp32-arduino version v3 and later will add the .ino directory to the include list by default, we can directly place the .h file in the project directory.
2. For the esp-idf framework, ESP32_Display_Panel, as a component in CMake compilation, mainly searches for header files within the scope of the standard library and the components it explicitly depends on. Placing .h files in the main directory (you can also think of main as a component that depends on all components in components and managed_components folders) cannot be directly used as an include path by ESP32_Display_Panel (unless you explicitly specify that ESP32_Display_Panel depends on main).

Therefore, in the esp-idf framework, it is best to use Kconfig to configure the custom development board. Then, you can save all custom configurations as a file through the command idf.py save-defconfig, similar to the file sdkconfig.ci.custom.BOARD_ESPRESSIF_ESP32_S3_LCD_EV_BOARD_V1_5. When compiling, use the command idf.py -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.ci.custom.BOARD_ESPRESSIF_ESP32_S3_LCD_EV_BOARD_V1_5" set-target esp32s3 to specify which configurations to use.

@martinroger Back to the original question, indeed, it's not possible to directly configure LCD initialization parameters in Kconfig. Therefore, I reserved the configVendorCommands() interface in the LCD driver. This allows for configuration via code before board->begin(), similar to the example here. However, this also reminds me to provide an example of how to use this interface. It should be difficult for users to discover this usage method.

[APF-MRoger]

@martinroger Back to the original question, indeed, it's not possible to directly configure LCD initialization parameters in Kconfig. Therefore, I reserved the configVendorCommands() interface in the LCD driver. This allows for configuration via code before board->begin(), similar to the example here. However, this also reminds me to provide an example of how to use this interface. It should be difficult for users to discover this usage method.

Replying from my "work" account... thanks a lot for the very useful info, definitely handy to have. I suppose that for the other PRE-x/POST-x macros of the conf file, it just has to be managed by initialising and starting the various components of the board separately rather than all at once with the board->begin() statement ?

[Lzw655]

@martinroger Back to the original question, indeed, it's not possible to directly configure LCD initialization parameters in Kconfig. Therefore, I reserved the configVendorCommands() interface in the LCD driver. This allows for configuration via code before board->begin(), similar to the example here. However, this also reminds me to provide an example of how to use this interface. It should be difficult for users to discover this usage method.

Replying from my "work" account... thanks a lot for the very useful info, definitely handy to have. I suppose that for the other PRE-x/POST-x macros of the conf file, it just has to be managed by initialising and starting the various components of the board separately rather than all at once with the board->begin() statement ?

Unfortunately, these PRE-x/POST-x macros are indeed called all at once in the board->begin() statement now. If you need, I can add PRE-x/POST-x macros for each independent driver.

[martinroger]

@martinroger Back to the original question, indeed, it's not possible to directly configure LCD initialization parameters in Kconfig. Therefore, I reserved the configVendorCommands() interface in the LCD driver. This allows for configuration via code before board->begin(), similar to the example here. However, this also reminds me to provide an example of how to use this interface. It should be difficult for users to discover this usage method.

Replying from my "work" account... thanks a lot for the very useful info, definitely handy to have. I suppose that for the other PRE-x/POST-x macros of the conf file, it just has to be managed by initialising and starting the various components of the board separately rather than all at once with the board->begin() statement ?

Unfortunately, these PRE-x/POST-x macros are indeed called all at once in the board->begin() statement now. If you need, I can add PRE-x/POST-x macros for each independent driver.

It's probably not a necessity at this stage, I guess if someone really wants to use a custom board in ESP-IDF, either they follow the workaround I found and use the conf file, or they manually "prime" the components of the board in the correct order in their code. People requiring specific init sequences hints at them knowing what's up, so they are already probably capable of doing option 1 or writing option 2.

Possibly as a future extension it could be good to be able to attach callback functions to those Pre/Post points in the board->begin(), possibly specifying the name of the callbacks from a KConfig ? That would allow maintaining the use of menuconfig and allow users to define their own init functions for special cases... What are your thoughts?

And I would potentially encourage adding a note for the ESP-IDF users that want to stick to using a custom conf file in the documentation, if you want I can try and make an amendment when I prepare the PR for the additional Waveshare board I have been playing with.
Personally I like the cleanliness of it and that I can track versioning changes of the conf file when tweaking it, separately from the massive mess that the sdkconfig file can become (but it is a personal preference!).

[Lzw655]

Possibly as a future extension it could be good to be able to attach callback functions to those Pre/Post points in the board->begin(), possibly specifying the name of the callbacks from a KConfig ? That would allow maintaining the use of menuconfig and allow users to define their own init functions for special cases... What are your thoughts?

In my understanding, the configuration of the Kconfig string attribute can only be used in the form of a string, and it seems that it cannot be combined into a function name through macros. In fact, currently, the board->configCallback() interface can be used to configure attaching callback functions to those Pre/Post points. I wonder if this can meet your needs?

And I would potentially encourage adding a note for the ESP-IDF users that want to stick to using a custom conf file in the documentation, if you want I can try and make an amendment when I prepare the PR for the additional Waveshare board I have been playing with. Personally I like the cleanliness of it and that I can track versioning changes of the conf file when tweaking it, separately from the massive mess that the sdkconfig file can become (but it is a personal preference!).

That would be great! Because I am quite familiar with this process, I may not notice the hidden problems in it. Thank you very much for your contribution! 👍