[bug] `tools.build:jobs` is ignored for MSBUILD with CMake & default is wrong

[stevenwdv]

Describe the bug

tools.build:jobs is ignored on Windows with CMake toolchain and MSBUILD (Visual Studio ...) CMake generator.

It seems cmake.py only adds parallelism flags per build tool, and doesn't know how to add them for MSBUILD. However, there is tools.microsoft.msbuild:max_cpu_count which seems to accomplish the same thing but for MSBUILD.

conan/conan/tools/cmake/cmake.py

Lines 16 to 23 in 4744d8e

	# Arguments related to parallel
	njobs = build_jobs(conanfile)
	if njobs and ("Makefiles" in generator or "Ninja" in generator) and "NMake" not in generator:
	args.append("-j{}".format(njobs))
	maxcpucount = conanfile.conf.get("tools.microsoft.msbuild:max_cpu_count", check_type=int)
	if maxcpucount is not None and "Visual Studio" in generator:
	args.append(f"/m:{maxcpucount}" if maxcpucount > 0 else "/m")

1. Why is this tools.microsoft.msbuild:max_cpu_count separate from tools.build:jobs?
2. Also, the docs on tools.build:jobs say "default: max CPUs", but this apparently doesn't apply here, as MSBUILD by default uses no paralellism.

Since CMake 3.12 it's possible to just pass --parallel (-j) to cmake --build itself, instead of to the build tool (after --). You could consider adopting this approach instead, as it should be more portable.

How to reproduce it

For example:

On Windows with Visual Studio:

$ conan install --requires 'abseil/20250814.0' --build='abseil/*' -c:a 'tools.build:jobs=20'
[...]
abseil/20250814.0: Running CMake.build()
abseil/20250814.0: RUN: cmake --build "C:\Users\pep\.conan2\p\b\absei97ce99baaf616\b\build" --config Release
[...]

On Linux with Ninja: (expected behavior)

$ conan install --requires 'abseil/20250814.0' --build='abseil/*' -c:a 'tools.build:jobs=20'
[...]
abseil/20250814.0: Running CMake.build()
abseil/20250814.0: RUN: cmake --build "/home/swdv/.conan2/p/b/absei5aae6cef00263/b/build/Release" -- -j20
[...]

[memsharded]

Hi @stevenwdv

Thanks for the feedback.

tools.build:jobs

This tools.build:jobs shouldn't be ignored by the generator. Can you please inspect the generated conan_toolchain.cmake and share it here? It should contain:

string(APPEND CONAN_CXX_FLAGS " /MP{{ parallel }}")
string(APPEND CONAN_C_FLAGS " /MP{{ parallel }}")

This sets the compilation parallelism, see for example https://www.kitware.com//cmake-building-with-all-your-cores/

The other conf tools.microsoft.msbuild:max_cpu_count set a different level of parallelism, at the target (project) level, which is a different type of parallelism. Conan maintain two configurations, because using the maximum number of cores for both is not necessarily a recommended configuration, as it can produce a counter effect because of exhaustion of resources.

[stevenwdv]

@memsharded Thanks for your reaction!

Ahh ok I didn't know that apparently MSBUILD can spawn one compiler for multiple files, which then spawns copies of itself according to /MP. I can confirm that /MP20 is being passed in conan_toolchain.cmake, and that the compiler is indeed called with this flag. However, this clashes with what I see happening in the process tree, because I almost never see multiple cl.exe processes:

Compare this with Linux + Ninja:

A world of difference!

Do you know what causes this?

I guess means cl.exe is called for just a single file each time. Maybe there are a lot of targets with just a single file? The page you linked says:

With no extra options the IDE will perform target level parallelism during the build.

But does this also hold for MSVC when used outside of the IDE? Should I be passing tools.microsoft.msbuild:max_cpu_count as well? Or is there some other issue going on?

[memsharded]

I think that might be an artifact of the task manager/explorer.
I am doing:

conan install --requires="opencv/[*]" --build=opencv/*

and using Windows 11 default built-in task manager, and I can see bursts of multiple cl.exe instances being spawned, and quickly closed, sometimes 2, sometimes 3, sometimes 8 parallel cl.exe

So it might be a thing of the refresh rate and how Windows manages processes and subprocess or something like that.

But does this also hold for MSVC when used outside of the IDE? Should I be passing tools.microsoft.msbuild:max_cpu_count as well? Or is there some other issue going on?

It depends on the structure of the project. If the project has just 1 library or executable with thousands of files, then tools.microsoft.msbuild:max_cpu_count will do nothing. If it has many different targets, each target with 1 or 2 files then tools.microsoft.msbuild:max_cpu_count might make a difference, and tools.build:jobs will probably do nothing or almost nothing.

[stevenwdv]

@memsharded I guess that may be it. Maybe we should combine tools.build:jobs with a small tools.microsoft.msbuild:max_cpu_count in our build then.

In any case, I did find another bug related to this, maybe this should be a separate ticket, if you agree it's a bug. It's actually a bug the other way around:
When working with CMake presets from Conan, it apparently sets execution.jobs in the buildPresets entry to the number of CPUs (if not overridden by tools.build:jobs). However, CMake then passes this to MSBUILD as -maxCpuCount (aka /m). This means that when we build our own project using Conan's preset, it spawns many MSBUILD processes, on top of the /MP<cpu count> that Conan also passes to cl.exe.
I don't think this is what is supposed to happen. I guess Conan and CMake have different opinions on where parallelism should happen, or at least cmake --build doesn't want to (or cannot) alter compiler parameters and just wants to pass something to the build tool.

[memsharded]

This was added back in #15422 for Conan 2.1

According to https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html

An optional integer. Equivalent to passing --parallel or -j on the command line.

And the -j concurrency is generally understood as a parallel compiler invocation, at least in all other systems? It is a bit unexpected that CMake decides to use that -j/--parallel to set the Visual max_cpu_count instead.

But it seems that it is what it is doing: https://stackoverflow.com/questions/10688549/how-do-i-configure-portable-parallel-builds-in-cmake

cmake.exe --build --parallel
starts a root MSBuild process with the following arguments:

msbuild.exe /m: /p:CL_MPCount=1 <another-useful-arguments...>

So it could be that the presets jobs have to be changed to max_cpu_count, then use the jobs as the /p:CL_MPcount=<n_cl> n_cl count:

cmake.exe --build --parallel <n_msbuild> -- /p:CL_MPcount=<n_cl>

But I don't know if this is possible to be set via presets.

[stevenwdv]

@memsharded

then use the jobs as the /p:CL_MPcount=<n_cl> n_cl count

You mean tools.build:jobs? This shouldn't be necessary, right? Since /MP is already in the conan_toolchain.cmake.

---

Should I create a separate ticket to track this CMake preset issue? Since it's not what the original ticket is about. (You can also do that if you want.) And then we can close this one.