Dynamically linked libraries

[paulyoung]

tl; dr

It seems possible to use dynamically linked libraries with the simulator but I can't figure out how to modify the search paths.

Hopefully this is doable on device as well or this is all moot.

---

I'm depending on a crate that provides bindings for a library, and it expects the library to be dynamically linked.

At first I was getting Undefined symbols for architecture with crank build. I've addressed that with the following build.rs:

use std::{env, path::Path};

fn main() {
    let cargo_manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();

   // This approach doesn't work because the library is named `foo_libqux.dylib`
   /*
    println!(
        "cargo:rustc-link-search=all={}",
        Path::new(&cargo_manifest_dir).join("lib").display()
    );

    println!("cargo:rustc-link-lib=dylib=retro");
   */

    println!(
        "cargo:rustc-link-arg={}",
        Path::new(&cargo_manifest_dir).join("lib/foo_libqux.dylib").display()
    );
}

However, crank build --run results in this error at runtime:

dlopen(/Users/py/projects/paulyoung/MyProject/main/target/My Project.pdx/pdex.dylib, 0x0006): Library not loaded: foo_libqux.dylib
  Referenced from: /Users/py/projects/paulyoung/MyProject/main/target/My Project.pdx/pdex.dylib
  Reason: tried: '/nix/store/g8b81w9kpb82gm7702ca692mp5flm7pf-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/foo_libqux.dylib' (no such file), '/nix/store/g8b81w9kpb82gm7702ca692mp5flm7pf-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/foo_libqux.dylib' (no such file), '/usr/lib/swift/foo_libqux.dylib' (no such file), '/Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib' (no such file), '/Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib' (no such file), '/usr/lib/swift/foo_libqux.dylib' (no such file), '/Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib' (no such file), '/Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib' (no such file), 'foo_libqux.dylib' (no such file), '/usr/local/lib/foo_libqux.dylib' (no such file), '/usr/lib/foo_libqux.dylib' (no such file), '//foo_libqux.dylib' (no such file), '/usr/local/lib/foo_libqux.dylib' (no such file), '/usr/lib/foo_libqux.dylib' (no such file)

(expand for a bulleted list of search paths)

• /nix/store/g8b81w9kpb82gm7702ca692mp5flm7pf-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/foo_libqux.dylib
• /nix/store/g8b81w9kpb82gm7702ca692mp5flm7pf-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/foo_libqux.dylib
• /usr/lib/swift/foo_libqux.dylib
• /Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib
• /Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib
• /usr/lib/swift/foo_libqux.dylib
• /Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib
• /Users/py/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/MacOS/../Frameworks/foo_libqux.dylib
• foo_libqux.dylib
• /usr/local/lib/foo_libqux.dylib
• /usr/lib/foo_libqux.dylib
• //foo_libqux.dylib
• /usr/local/lib/foo_libqux.dylib
• /usr/lib/foo_libqux.dylib

I've tried setting LD_LIBRARY_PATH=./lib, DYLD_LIBRARY_PATH=./lib, and DYLD_FALLBACK_LIBRARY_PATH=./lib when invoking crank but it doesn't make a difference to the search paths. I tried the same thing with open -a "Playdate Simulator" ./target/My\ Project.pdx but that didn't change anything either.

I also tried adding those environment variables via cmd.env in run_pdc and run_simulator in main.rs but still no change.

This obviously isn't a solution, but I copied the library and made it available at ~/Developer/PlaydateSDK/bin/Playdate Simulator.app/Contents/Frameworks/foo_libqux.dylib and things appear to work.

It at least seems to demonstrate that this should work for the simulator as long as the search paths are set correctly. I don't have a device yet and I'm not sure where to begin on making this work there but hopefully there's nothing preventing this.

If I could get some insight or advice on how to go about this properly I'd be very grateful. Happy to make a PR to propose changes or document what I've found as a result.

[rtsuk]

I can't imagine that dynamic libraries are supported on the device, but the developer forums would be the place to ask.

Why do you need a dylib?

[paulyoung]

Why do you need a dylib?

I'm experimenting with some ideas and one of my dependencies involves a dylib. I think technically a static library can be used instead, but there are prebuilt .dylib files for all of the supported architectures.

It would be significantly more effort to figure out how to build it as a static library. I was hoping to quickly validate my idea first and avoid that for now.

If dynamic libraries aren't supported on device then I could probably move forward with the simulator for the time being and figure out building a static lib if it looks promising.

[paulyoung]

I can't imagine that dynamic libraries are supported on the device, but the developer forums would be the place to ask.

Thanks. I asked here: https://devforum.play.date/t/is-it-possible-to-load-dynamic-libraries-on-device/10637

[paulyoung]

The issue with not being able to set library paths might be explained by System Integrity Protection.

[boozook]

It could be really useful as "hot-reload" feature for develop/debug in sim only. I'll think about it after revB & OS2.0 support.

[paulyoung]

For a different project I've been able to statically link a library by using cc in my build.rs but it only works in the simulator.

When I try to build for device I get an undefined reference error from ld.

I haven't looked into what changes might need to be made to crank to support this yet.

@boozook, since you commented here I'm wondering if this is supported in your project at https://github.com/boozook/playdate

Development over there seems a bit more active so I've been considering trying it out and porting any contributions I've made to crank and crankstart that might be missing.

[boozook]

Theoretically, it’s just to compile another library for playdate-target and statically shash with your game. That's should be possible using cc and/or cmake crates in your build script.
But you also need correct flags for clang or gcc (I mean gnu-arm-gcc, you know). There's crate that can help with it, but currently only with gcc (I'll fix it in this year as I hope).
I suppose here it is, but --entry eventHandlerShim should be removed for other lib that isn’t final product.
That’s about static linking that possible for device-target build.
But for the sim we can build anything that can be run on host machine and that contains correct symbols such as entry-point, ideally using SDK-provided allocator.

When I try to build for device I get an undefined reference error from ld.

I’m sure that’s because there’s no symbols built for needed arch.

Development over there seems a bit more active so I've been considering trying it out and porting any contributions I've made to crank and crankstart that might be missing.

That’s could be really awesome and helpful, thanks! ❤️‍🔥 Yes, I love that project and don’t want to stop or any kind of step back.
Just one problem there is API complexity with optimization reasons. I know how to make it more friendlier, but that’s time. Probably one or two months 🤷🏻‍♂️. I already have some drafts for it 🤔.

---

P.s.: By the way, I recently fixed a lot of problems with cargo-playdate and pdtool related to device detection and now going to a kinda stable release. You can already try the beta that's on crates io already.

[paulyoung]

For what it’s worth, I’m using cc in build.rs to compile from source and setting the CC environment variable to the path to arm-none-eabi-gcc