Pkg app & pre-commit hook

[gdalle]

Hi @aviatesk,
Would you be open to implementing a pre-commit hook inside this repo, so that people might run JET-based linting on their package without adding it to their test dependencies? See e.g. this Discourse thread for why it would make sense (compat issues mostly). Happy to contribute it myself.
For reference, BestieTemplate runs JuliaFormatter + ExplicitImports inside pre-commit as part of their linting step, and I think JET would be a good additional candidate.

[aviatesk]

Reading the Discourse thread, it seems the issue is a general environment isolation problem between JuliaFormatter/JET.
I believe that for a complete solution to this problem, analysis tools like JET, JETLS and maybe JuliaFormatter too should bundle their dependencies themselves, in a way that I did for JETLS (you can take a look at https://github.com/aviatesk/JETLS.jl/blob/master/scripts/vendor-deps.jl or https://github.com/aviatesk/JETLS.jl/tree/release to understand this idea if interested).
I think it would be worth trying the same thing with JET, and that would solve the Discourse issue.

Moving back to the original OP suggestion, I also think the Pkg.jl executable app itself is also very useful (and indeed using executable apps does provide environment isolation, since it's allowed to run in an isolated Julia process), and I think we better provide that as well.
We first need to start by preparing a CLI for JET.
We should note that options like target_modules are basically implemented assuming that users will call JET from a Julia session, not from CLI executable, so it might be difficult to support them from the CLI. I think we need to distinguish what is and isn't supported via CLI.

[gdalle]

Sorry, yes I should have been clearer. Here's what I have in mind:

• Conceptually, JET.test_package is more of a linting step than an actual test, and so it would make sense to run it separately, like formatting. A pre-commit hook seems like a good format for that.
• Practically, this would also remove JET from the test dependencies of many packages, which helps with handling compat bounds (especially given that the latest JET doesn't support Julia 1.10).
• Running all linting and formatting steps together would still cause the environment clash between JET and JuliaFormatter, which strongly encourages the development of a Pkg app.

[aviatesk]

I see, I understood that you are only proposing a CLI, but I just took this opportunity to elaborate my idea of environment isolation through dependency bundling as an idea to fundamentally solve the problems you are facing.

Well, I think that can be done in a separate issue.

Having a CLI would be beneficial for process-level env isolation and also making it easier to use from pre-commit hooks as well. I am in favor of that.

[gdalle]

Im not sure I understand what this manual vendoring brings, compared to apps?

[aviatesk]

Manual dependency bundling (w/ on-disk UUID rewrite) means that, in principle, you can use JET without it interfering with the user's environment. This means that even if JuliaFormatter uses JuliaSyntax 0.4, you'll be able to load JET without any issues.

[aviatesk]

https://publish.obsidian.md/jetls/work/JETLS/JETLS+environment+isolation#On-disk+UUID+rewriting+approach+(vendoring) might explain this idea better, just fyi.

[gdalle]

Oh I see. I agree it is best discussed as a separate issue though.

Moving to CLI design, my initial instinct would be something like

jet-test mod path

where mod is the name of the package to test and the name of the target module, while path points to the folder where mod is defined (at "$path/Project.toml"). The main function would then create a temporary environment, dev JET and mod, then run JET.test_package. How does that sound?
Ideally one would like to specify multiple target modules but I've never written a CLI before and it isn't clear to me how one can pass arguments as a list or tuple.

[aviatesk]

I'm not a fan of having too many executable names floating around, so, I think it'd be better to just name the executable jet and let users specify analysis entry points other than test_package by providing options to the executable. Of course, I assume test_package is all you need for your use case, so it's probably fine to implement just that initially. But, I also think there might be times when you just want to see the analysis results, so the executable should probably be written in a way that it can be extended to support report_xxx functions too.

a temporary environment, dev JET and mod

I think it would be good to support a --project option for the jet executable. So, the process would be: first load JET, which should just be done by invoking jet executable, then activate the environment based on the --project argument, and finally load mod if needed. (Note: If you support things other than [report|test]_package, you won't need to pre-load mod. We wouldn't need mod argument at all for such cases though)
The implementation here might be helpful: https://github.com/aviatesk/TestRunner.jl/blob/236998a16d88a1d4aad8a591548676ae31802315/src/app.jl#L404-L410