Allow clang-tidy to load more checkers from an external library/plugin

[llvmbot]

Bugzilla Link	32739
Version	unspecified
OS	Linux
Reporter	LLVM Bugzilla Contributor
CC	@bbannier,@chgans,@justincady,@JonasToth,@froydnj,@jbytheway,@stephenhines,@steveire

Extended Description

The goal is to be able to write checkers out of tree, which are specific to to some project or customer, then just load them with: clang-tidy -plugin mychecks.so ...

Or similar

[llvmbot]

Clang-tidy has a certain level of support for statically-linked out-of-tree checks. With minimal or no changes to the code checked out from upstream that is. We've been using this with an internal build system since the project start, but proper CMake support for this may still need some work (any volunteers? ;).

As for plugins, they would offer only a very limited benefit, since:

1. one would still have to have a large part of llvm, clang and clang-tidy code to build a plugin;
2. llvm and clang don't provide a stable C++ API, so the plugin would have to be build against exactly the same version of sources the binary clang-tidy was built from, with the same toolchain, same C++ standard library and same or compatible compiler options.

For the same set of reasons clang static analyzer plugins are neither documented nor recommended by the CSA maintainers (though they are "supported" and there's an example).

[llvmbot]

How's that any different from clang supporting plugins ?

clang supports plugins and it's working very well for Linux distros so far.

1. Plugins are built against clang and llvm shared libraries.

2. Packagers rebuild the plugins each time a new LLVM version is out.

It's not perfect, and sometimes the author needs to add ifdefs to make it build, but what's the alternative ?

How will clang-tidy scale with N different vendor checks, is it just going to grow indefinitely. I already see android and fuchsia checks that I don't care about.

Is the world supposed to contribute N plugins to the tree, or will I have N statically linked clang-tidy on my system ?

[llvmbot]

Just chiming in here since my colleague Sergio pointed me to this bug report.

I haven't work on clang-tidy lately, but during additional checker development in the past, the time required to finally link clang-tidy have caused very slow development cycles. In the end we basically commented every non-interesting subdir containing checkers in CMake in order to speed up these cycles, to be able to work on our in-house checker.

In the best case, then I'd only need to build/link a clang-tidy plugin, not do a huge link of clang-tidy each time I'm changing a checker...

I think a plugin framework for clang-tidy would make a lot of sense, also wrt the other reasons Sergio mentioned.

[JonasToth]

I agree with the need for plugins.

[llvmbot]

I'd also like to see clang-tidy plugin support. I think we'd see a large ecosystem of plugins grow to support various framework-specific and project-specific checks. Even with the limitations Alexander describes (which for end users on a Linux system with package management are basically a non-issue)

[chgans]

Any update on this?

As a user of clang-tidy and clazy, i would really like to have a unify front-end to run checks in an automated way.

[jbytheway]

I also felt the need for plugin support. I decided to go ahead and implement it. I think it's worth talking about that here to help provide more evidence for the usefulness of this feature.

I have a project which releases patched clang-tidy versions with plugin support at https://github.com/jbytheway/clang-tidy-plugin-support.

Currently it only offers LLVM 8 and the implementation relies on compiling the executable with -rdynamic so it doesn't work on Windows. That suggests that adding plugin support in a more portable fashion would require factoring out much of the clang-tidy binary to a library for plugins to link against.

I set this up to support project-specific checks for https://github.com/CleverRaven/Cataclysm-DDA, where it has been very successful in automatically enforcing project style and performing code migration.

It's not easy for a typical dev to set this up to run locally, but it's also not necessary for them to. The primary use case is running in continuous integration systems, where being tied to the specific LLVM version for ABI reasons is no real hardship. At some point we will jump ahead to a newer LLVM and rewrite the custom checks, but that only has to happen every few years; it's not a serious concern.

Having to compile a custom clang-tidy version for each CI test run, rather than just compiling the plugin, would be a waste of time, and we already have problems with the time taken by our CI jobs. As mentioned above, even if the compilation were all cached, the link time is prohibitive. It feels like plugins are a good solution here.

Hopefully this offers some more evidence of the utility. I realise nothing will happen until someone actually tries submitting patches to clang. I might try that at some point, but probably not soon. In the mean time anyone with more initiative is welcome to take inspiration from the patch in my project linked above.

[llvmbot]

The way clang supports plugins on Windows is by clang.exe itself exporting the symbols and the plugins linking against that.

By passing -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=ON when building clang, it will generate a clang.lib with the symbols from clang.exe

Maybe clang-tidy could have a similar approach

[steveire]

From reading the mailing list on topics like this, it seems that part of the problem is that clang has too many symbols to export on Windows.

An alternative to making clang-tidy work with plugins is to instead create bindings to make it possible to use another (scripting) language. I made a proof of concept to create AST matchers with javascript so it is possible: https://steveire.wordpress.com/2019/04/30/the-future-of-ast-matching-refactoring-tools-eurollvm-and-accu/

[glandium]

Wasn't this fixed by 84f137a?

[PiotrZSL]

Looks like it was.