Blinky fails to build (on Mac) after addition of linker script preprocessing feature

[Patater]

With the latest mbed OS master, I find I'm no longer able to build. The reason appears to be an autogenerated linker script, ./BUILD/K64F/GCC_ARM/.link_script.ld, is not being autogenerated and is therefore missing.

I did a git bisect on mbed-os and find the issue first arises with the addition of the following commit (af4d848).

commit af4d848144d3e62a429528149dc33fd8d45ec9da
Author: Jimmy Brisson <theotherjimmy@gmail.com>
Date:   Wed Feb 1 15:59:27 2017 -0600

    Use the C Pre-Processor on GCC Linker scirpts

    This allows us to define parts of the linker script outside of the
    linker script itself. In particular, we are interested in restricting
    ROM to a subsection.

Here's the output from mbed compile -m K64F -t GCC_ARM -v -c.

Compile [100.0%]: test_env.cpp
[DEBUG] Compile: /Users/username/bin/arm-none-eabi-g++ -std=gnu++98 -fno-rtti -Wvla -c -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fmessage-length=0 -fno-exceptions -fno-builtin -ffunction-sections -fdata-sections -funsigned-char -MMD -fno-delete-null-pointer-checks -fomit-frame-pointer -Os -mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=softfp -DFEATURE_LWIP=1 -D__MBED__=1 -DDEVICE_I2CSLAVE=1 -D__FPU_PRESENT=1 -DTARGET_Freescale -DDEVICE_PORTINOUT=1 -DTARGET_RTOS_M4_M7 -DDEVICE_LOWPOWERTIMER=1 -DDEVICE_RTC=1 -DTOOLCHAIN_object -DDEVICE_SERIAL_ASYNCH=1 -D__CMSIS_RTOS -DFSL_RTOS_MBED -DDEVICE_STORAGE=1 -DTARGET_KPSDK_MCUS -DTOOLCHAIN_GCC -DTARGET_CORTEX_M -DTARGET_KSDK2_MCUS -DTARGET_LIKE_CORTEX_M4 -DDEVICE_ANALOGOUT=1 -DTARGET_M4 -DTARGET_UVISOR_UNSUPPORTED -DTARGET_K64F -DDEVICE_SPI_ASYNCH=1 -DDEVICE_PWMOUT=1 -DDEVICE_INTERRUPTIN=1 -DDEVICE_I2C=1 -DDEVICE_PORTOUT=1 -D__CORTEX_M4 -DDEVICE_STDIO_MESSAGES=1 -DCPU_MK64FN1M0VMD12 -DTARGET_LIKE_MBED -DTARGET_FF_ARDUINO -DTARGET_KPSDK_CODE -DTARGET_RELEASE -DDEVICE_SERIAL_FC=1 -DFEATURE_STORAGE=1 -DMBED_BUILD_TIMESTAMP=1487264619.12 -DDEVICE_TRNG=1 -D__MBED_CMSIS_RTOS_CM -DDEVICE_SLEEP=1 -DTOOLCHAIN_GCC_ARM -DTARGET_FRDM -DTARGET_MCUXpresso_MCUS -DDEVICE_SPI=1 -DDEVICE_ERROR_RED=1 -DDEVICE_SPISLAVE=1 -DDEVICE_ANALOGIN=1 -DDEVICE_SERIAL=1 -DDEVICE_PORTIN=1 -DTARGET_MCU_K64F -DARM_MATH_CM4 @./BUILD/K64F/GCC_ARM/.includes_c497183e2336a93dbed35a2b95c3919a.txt -include ./BUILD/K64F/GCC_ARM/mbed_config.h -MD -MF ./BUILD/K64F/GCC_ARM/./test_env.d -o ./BUILD/K64F/GCC_ARM/./test_env.o /Users/username/Projects/mbed-os-example-blinky/mbed-os/features/unsupported/tests/mbed/env/test_env.cpp
[DEBUG] Return: 0
Link: mbed-os-example-blinky
[DEBUG] Preproc: /Users/username/bin/arm-none-eabi-cpp -E -P ./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/device/TOOLCHAIN_GCC_ARM/MK64FN1M0xxx12.ld -Wl,--gc-sections -Wl,--wrap,main -Wl,--wrap,_malloc_r -Wl,--wrap,_free_r -Wl,--wrap,_realloc_r -Wl,--wrap,_calloc_r -Wl,--wrap,exit -Wl,--wrap,atexit -mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=softfp -o ./BUILD/K64F/GCC_ARM/.link_script.ld
[OS ERROR] Command: /Users/username/bin/arm-none-eabi-cpp -E -P ./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/device/TOOLCHAIN_GCC_ARM/MK64FN1M0xxx12.ld -Wl,--gc-sections -Wl,--wrap,main -Wl,--wrap,_malloc_r -Wl,--wrap,_free_r -Wl,--wrap,_realloc_r -Wl,--wrap,_calloc_r -Wl,--wrap,exit -Wl,--wrap,atexit -mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=softfp -o ./BUILD/K64F/GCC_ARM/.link_script.ld
Traceback (most recent call last):
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/make.py", line 293, in <module>
    toolchain))
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/build_api.py", line 456, in build_project
    res, _ = toolchain.link_program(resources, build_path, name)
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/toolchains/__init__.py", line 1015, in link_program
    self.link(elf, r.objects, r.libraries, r.lib_dirs, r.linker_script)
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/hooks.py", line 34, in wrapper
    res = function(t_self, *args, **kwargs)
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/toolchains/gcc.py", line 223, in link
    self.default_cmd(cmd)
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/toolchains/__init__.py", line 1032, in default_cmd
    _stdout, _stderr, _rc = run_cmd(command, work_dir=getcwd(), chroot=self.CHROOT)
  File "/Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/utils.py", line 99, in run_cmd
    stderr=STDOUT if redirect else PIPE, cwd=work_dir)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
[mbed] ERROR: "python" returned error code 1.
[mbed] ERROR: Command "python -u /Users/username/Projects/mbed-os-example-blinky/mbed-os/tools/make.py -t GCC_ARM -m K64F --source . --build ./BUILD/K64F/GCC_ARM -c -v" in "/Users/username/Projects/mbed-os-example-blinky"
---
(mbed) username@e107376 ~/P/mbed-os-example-blinky>

Description

• Type: Bug
• Priority: Blocker

---

Bug

Target
K64F

Toolchain:
GCC_ARM

Toolchain version:
On Mac OS X, arm-none-eabi-gcc 5-2016-q3-update

mbed-cli version:
1.0.0

mbed-os sha:
21b91c7 Merge pull request #3753 from AnotherButler/patch-2

Expected behavior
Build should work

Actual behavior
Build fails due to linker error

Steps to reproduce

1. mbed import mbed-os-example-blinky
2. cd mbed-os-example-blinky/mbed-os
3. git checkout origin/master
4. cd ..
5. mbed compile -m K64F -t GCC_ARM -v -c
6. Observe a linker error

@theotherjimmy

[theotherjimmy]

Does it build for you on windows/linux?

[theotherjimmy]

There's no linker script in that command line. Can't read.

[theotherjimmy]

Actually, it looks like you don't have arm-none-eabi-cpp

[Patater]

I do have arm-none-eabi-cpp. How is the Python script trying to discover it? Perhaps in a way not compatible with how the defacto-official Mac package manager, Homebrew, installs it via symlink?

username@mac ~> which arm-none-eabi-cpp
/usr/local/bin/arm-none-eabi-cpp
username@mac ~> ls -l /usr/local/bin/arm-none-eabi-cpp
lrwxr-xr-x  1 username  admin  66 19 Jan 17:37 /usr/local/bin/arm-none-eabi-cpp -> ../Cellar/arm-none-eabi-gcc/5-2016-q3-update/bin/arm-none-eabi-cpp

[Patater]

It looks like the Python script is incorrectly assuming that where ever arm-none-eabi-gcc is located, arm-none-eabi-cpp is also located in that same folder. This is an assumption that, usually true, is not an assumption that needs to be made.

The Python script should just use the executable name and have the OS resolve the path in the same way as if a user typed the command at a command prompt. Alternatively, use distutils.spawn.find_executable or shutil.which, if having the absolute path known by Python is a requirement for some reason.

[theotherjimmy]

Yep that's correct. Let's just have subprocess find it whenever it's found in the path.

[theotherjimmy]

#3895 was merged. closing.