Currently for many LS we ignore the version if the binary can be found (we only download if we don't find).
We should consider putting the version into the path where we store the binaries. Also consider migration of some sort.
Idea for migration:
For each LS we store the initially used version and we add a postfix to the path only for versions differing for that. Then no migration is necessary (assuming the users didn't override the version in the past - for a few LS it was possible)