Add a "require plugin" option

[asottile]

In GitLab by @mschwager on Jan 7, 2019, 07:34

Hi all,

I'd like to request an additional option to flake8 for requiring certain plugin(s) are installed and selected during a run. For example,

$ flake8 --require=XYZ --select=XYZ /path/to/lint

I could also envision the new option implies a --select. Naming like --require-plugins also makes sense to me.

This would be very helpful for reminding developers to have a set list of flake8 plugins installed on their machine. For example, a common use case may be that a repository uses a Makefile for various common operations including linting:

lint:
    flake8 --select=XYZ /path/to/lint

.PHONY: lint

Both the CI/CD system and developers use the Makefile to run these common operations, i.e. make lint. However, if the CI/CD system has the required plugins, but the developer's machine does not, then the local dev machine will silently ignore the selected rules:

$ flake8 --select=DOESNTEXIST /path/to/lint
$ echo $?
0

I'm envisioning a flag that would loudly complain if a required plugin is not installed:

lint:
    flake8 --require=ABC,XYZ /path/to/lint

.PHONY: lint

Keying off the plugin's prefix, similar to --select and --ignore, seems to be the most consistent solution in my opinion.

Another option could be for flake8 to complain loudly if a plugin in a --select flag doesn't exist. However, a new flag makes more sense to me for ensuring backwards compatibility and because this seems like a net-new feature.

Thoughts?

[asottile]

In GitLab by @asottile on Jan 7, 2019, 08:09

Another option could be for flake8 to complain loudly if a plugin in a --select flag doesn't exist

this actually seems the best to me, ensuring backward compatibility here seems less important for something that's clearly a user error

[oliver-sanders]

To chip in a similar use case, it would be good if we could configure the plugins that flake8 uses.

Because plugins are used if they are installed we could do with a way of turning plugins off.

For example if you install two projects into the same environment, one which uses flake8-bugbear and another that uses flake-simplify, you would end up running both plugins against both projects which might raise many more errors than intended.

Perhaps we could have a --plugin option for specifying plugins to run. If the --plugin option is used only specified plugins are loaded and an error would be raised if the plugin is not installed (as with the proposed --require above).

Pure CLI example:

$ flake8 --plugin flake8-simplify projectA/src projectA/test
$ flake8 --plugin flake8-bugbear projectB

Project config example:

# projectA/tox.ini
[flake8]
paths =
    ./src
    ./test
plugins =
    flake8-simplify

# projectB/tox.ini
[flake8]
plugins =
    flake8-bugbear

[sigmavirus24]

@oliver-sanders that sounds very different to this issue and should be handled by using --select to choose what gets reported

[Dreamsorcerer]

I think the --plugins option that @oliver-sanders suggests is exactly what I end up looking for very frequently. If the option is given, then flake8 should only load the plugins in that list, and error if a named plugin is not installed.

This fixes 2 problems, the first is running plugins which shouldn't be used for a project, but happen to be installed. I know you guys like to think that everyone should just solve this with pipenv, but I think there are still problems with that (like different configs in different branches), so explicit is better than implicit.
The second problem is people running flake8 without having installed new plugins that are now required and therefore end up missing some checks.

It also potentially provides performance improvements by not loading unneeded plugins etc. It seems like a much simpler and more explicit approach, that avoids the complexities of --select that might make the feature difficult to implement.

If the --plugins option is not provided, then it should continue to load all installed plugins by default.

[sigmavirus24]

I know you guys like to think that everyone should just solve this with pipenv

Sam, I've been tolerant of a lot of your behaviour but making random shit up is crossing a line. I've never recommended using pipenv as a way around any specific kind of Flake8 behaviour.

This nonsense aside

If the option is given, then flake8 should only load the plugins in that list, and error if a named plugin is not installed.

I think regardless that this could lead to a great deal of confusion on behalf of users.

For one thing, many editors don't integrate well with "Flake8 exited before reporting any errors" so most users would go from a state of "different errors" to "no flake8 feedback whatsoever" unless we still report errors and then the editor will report them and users won't notice that a plugin is missing.

Further, projects seem to already manage this by using some kind of automation (nox, tox, etc.) that installs the required dependencies in a virtualenv and ensures that Flake8 is part of the testing process when one runs tox. So then users might install Flake8 'globally' (I use quotes here because pipx, pip install --user, etc. all exist and give that user a global feel) and get errors about needing to install a plugin for 1 project and go to working on other things and now they must update that project's configuration to avoid a different project's preferences. So now, Flake8 is helping with one part but creating significant amounts of work for another.

[Dreamsorcerer]

I know you guys like to think that everyone should just solve this with pipenv

Sam, I've been tolerant of a lot of your behaviour but making random shit up is crossing a line. I've never recommended using pipenv as a way around any specific kind of Flake8 behaviour.

Maybe I'm wrong then. Didn't mean to be offensive, but I'm sure I recall a conversation about this a couple of years ago, and the issue was closed with pipenv/virtualenv being the expected approach. Although you are mentioning virtualenv again, which is probably what I meant, rather than pipenv.

For one thing, many editors don't integrate well with "Flake8 exited before reporting any errors" so most users would go from a state of "different errors" to "no flake8 feedback whatsoever"

Hmm, would editors not highlight configuration errors? I mean, if I run flake8 --foobar it fails with an error, I'd expect an editor to highlight that somehow, in case anything was misconfigured. Even if not currently supported, it seems like something that should be supported by editors.

Further, projects seem to already manage this by using some kind of automation (nox, tox, etc.) that installs the required dependencies in a virtualenv and ensures that Flake8 is part of the testing process when one runs tox. So then users might install Flake8 'globally' (I use quotes here because pipx, pip install --user, etc. all exist and give that user a global feel) and get errors about needing to install a plugin for 1 project and go to working on other things and now they must update that project's configuration to avoid a different project's preferences. So now, Flake8 is helping with one part but creating significant amounts of work for another.

I see this approach for CI everywhere, but not necessarily for local dev. If it's not installing the dependencies on every run, doesn't this still have the same issues? e.g. If I have a plugin used in one git branch, but not the other, aren't you using the same environment for both branches?

[sigmavirus24]

I mean, if I run flake8 --foobar it fails with an error, I'd expect an editor to highlight that somehow, in case anything was misconfigured. Even if not currently supported, it seems like something that should be supported by editors.

I've seen those kinds of errors get buried by editors.

If it's not installing the dependencies on every run, doesn't this still have the same issues? e.g. If I have a plugin used in one git branch, but not the other, aren't you using the same environment for both branches?

Tox doesn't re-install every time and nox can have that disabled. That said, Tox is smart enough to recognize changes to what it cares about and recreate at that point. So no, this issue isn't present in those cases. And yes, I'd argue tox is used heavily in local development of private and public projects.

I've seen a fair amount of private/public libraries/multi-python-version tools. The best standard practice for those is nox or tox.

For single-version applications, etc., I've never seen someone not use a virtual environment (either on the host machine or on a docker image). Even projects that largely share config, like OpenStack have separate environments for Flake8 because each project has it's own slightly different config.

It's not so much that I recommend using it in a virtualenv as much as every tool that has plugins like this is best used in a virtualenv - pytest, flake8, etc.

Also, my point isn't about branch-to-branch differences. It's project-to-project. To try to clarify:

• I'm working on projectA which just added NewPlugin. I install that so everything on my system can access it.
• I switch to projectB but that has never specified a plugin list, now I get the violations from NewPlugin there unless I explicitly list what I want in that new project which I had no need for because it only ever used "vanilla" flake8.
• I switch to projectC and NewPlugin actually has violation codes that conflict with SomeOtherPlugin and now I can't distinguish the two because ProjectC also doesn't list what plugins it wants, etc.

It creates a lot more work, and will generate PRs from the people who tend to spam Python repositories with PRs to start using new features.

[Dreamsorcerer]

For single-version applications, etc., I've never seen someone not use a virtual environment (either on the host machine or on a docker image).

Even if that's true, you can still easily encounter edge cases, like switching between branches on the same project.

Even projects that largely share config, like OpenStack have separate environments for Flake8 because each project has it's own slightly different config.

Well, they are forced to because of the implicit loading of plugins, it would dangerous to do it any other way with the current design of flake8.

It creates a lot more work, and will generate PRs from the people who tend to spam Python repositories with PRs to start using new features.

Perhaps, but we're talking about a 1 line change which can be accepted or rejected as the maintainers wish.

Currently, the approach is forcing this way of working on to other people. As another example of a static analysis tool, mypy only loads plugins which are explicitly listed and will work as a globally installed application across different projects with different configs. Even pytest, which you mentioned, allows explicitly listing plugins to load them.
I think that only the implicit loading of plugins is stopping flake8 to be used in as flexible a manner as mypy.

[sigmavirus24]

Perhaps, but we're talking about a 1 line change which can be accepted or rejected as the maintainers wish.

Except that they'll have to reject it once a week or once a month. Some people just don't stop. I hate providing tools that others use to beat maintainers down into the ground with. I've been on the receiving end, nothing makes me hate a tool more