[bug] Concurrent python environment setup races on the shared workspace .venv (uv venv collision + uv sync member churn)

[kibumpark-carbonsix]

Describe the bug

On a cold workspace (no .venv, no prior .moon/cache state), running moon ci with enough
parallel targets makes multiple python environment actors race on the workspace .venv, and one
dies:

▮▮▮▮ uv venv .venv --no-managed-python --no-python-downloads --no-progress
▮▮▮▮ uv venv .venv --no-managed-python --no-python-downloads --no-progress
...
Creating virtual environment at: .venv
error: Failed to create virtual environment
  Caused by: A virtual environment already exists at: .venv
hint: Use the `--clear` flag or set `UV_VENV_CLEAR=1` to replace the existing virtual environment

A timestamped CI failure trace shows three overlapping writer classes, not just the two uv venv
invocations:

07:59:25.42  ▮▮▮▮ uv venv .venv                    # announced
07:59:25.83  ▮▮▮▮ uv venv .venv                    # announced
07:59:28.27  Creating virtual environment at: .venv                          # project-scoped actor, its own dir (cwd=<project>)
07:59:28.71  Creating virtual environment at: /abs/…/<workspace>/.venv       # from `uv sync` — implicit creation when missing
07:59:28.99  Creating virtual environment at: .venv
             error: … A virtual environment already exists at: .venv          # workspace-scoped `uv venv` loses

So the colliding pair in our trace is the workspace-scoped SetupEnvironment's uv venv against
an InstallDependencies' uv sync (workspace-scoped, or a project-scoped one whose uv sync
resolves upward to the workspace env), which implicitly creates the venv when it is missing.
The python toolchain already skips emitting uv venv when the directory exists
(moonrepo/plugins#142), but on a cold start every actor samples "missing" before any of them
creates it — a TOCTOU window the existence check cannot close.

The two environment-setup identities, from .moon/cache/hashes/*.json:

root=''       projectId=None      toolchainId=unstable_python   # workspace-scoped
root='tools'  projectId='tools'   toolchainId=unstable_python   # project-scoped

The per-command lock in crates/actions/src/plugins/commands.rs
(create_lock("{prefix}-{cmd.get_cache_key()}")) does not serialize these: setup-environment and
install-dependencies prefixes (and differing commands) take different locks. It is a
command-identity lock, not a resource lock on the resolved venv path.

This looks like the unresolved half of #2524 / #2525. Those were reported as an indefinite hang
(two actions contending for one per-toolchain lock) and were closed by making the locks more
unique — which turned the deadlock into this race. Your own comment on #2524 describes the missing
half:

the setup environment action either a) should have different hashes across projects, or b) share
the same setup environment so it only runs once. The fact that it doesn't seem to be doing either
of those feels like a bug.

(a) landed; (b) did not, and (b) — or serialization on the resolved venv path — is what a shared
.venv needs.

Still present in the latest release

Checked against v2.4.6 source: crates/actions/src/actions/setup_environment.rs, the plugin
command lock, and the action-graph builder are unchanged since v2.4.3, and the bundled plugin
versions are identical (python_toolchain 0.2.0, python_uv_toolchain 0.1.3), so upgrading does
not change the behavior.

Steps to reproduce

1. A uv workspace whose root pyproject.toml owns the .venv, with unstable_python +
   unstable_uv in .moon/toolchains.yml.
2. A second project that acquires the python toolchain (declared on a task, or detected from a
   python3 … / uv … task command).
3. Remove .venv and .moon/cache/hashes so all environment actors start cold.
4. Run enough parallel targets to overlap the actors, e.g. moon ci on a change selecting many
   projects. With 8 resolved targets we saw one uv venv; with 20, overlapping writers and the
   failure. Collision is timing-dependent — locally the actors often serialize by chance.

Second face — the same race class survives venv creation: member uninstall under running tasks

After we contained the uv venv collision (below), the same underlying gap resurfaced through
uv sync scope divergence. A project-scoped InstallDependencies runs a default-scope
uv sync (root project only), while tasks run uv run --all-packages; the desired member sets
differ, so the action's sync uninstalls the installed workspace members — and uv's
project-environment lock serializes syncs against each other, not against already-running
processes. Timestamped CI evidence:

08:47:56  <member-app>:test starts (pytest, ~30s)
08:48:04  project-scoped InstallDependencies `uv sync` → "Uninstalled 38 packages in 30ms"
08:48:26  test ends: 122 passed, 5 errors — importlib.metadata lookups of a workspace member
          (pkg_version("<member-app>")) fail with PackageNotFoundError mid-run

We contained this too (aligning syncArgs to --all-packages), but both faces share one root
cause, which is the actual report here.

Expected behavior

Environment actors that resolve to the same environment should either be deduplicated into one
action or serialized on that resource — and mutations of a shared environment should not run
concurrently with tasks that use it. Per-toolchain/per-command hashing and locking are
insufficient, because the mutable resource is the resolved environment (venv path + installed
member set), not the toolchain or command identity. uv sync's implicit venv creation and its
member-set reconciliation both participate in the race even when no second uv venv is emitted.

Workaround we ship (containment, not a fix)

Two parts. First, serialize the cold provisioning before the parallel run:

- run: MOON_CONCURRENCY=1 moon run <cheap workspace-member python task>
- run: moon ci --include-relations --downstream deep

The warm-up creates the venv serially; every later actor then samples "exists" and emits no
uv venv (the plugins#142 path). Second, align the sync scope so InstallDependencies stops
fighting the tasks over the member set:

unstable_uv:
  syncArgs: ["--all-packages", "--no-managed-python", "--no-python-downloads", "--no-progress"]

(Note for other readers: syncArgs replaces the plugin's fallback args, so the fallback flags
must be restated.) Both are verified in our CI. We rejected --clear / UV_VENV_CLEAR=1 (the
hint uv prints — two concurrent clears race destructively) and venvArgs: ["--allow-existing"]
(permits the overlay instead of serializing the writers).

Environment

• moon 2.4.3 (behavior also confirmed unchanged in v2.4.6 source)
• unstable_python 3.13, unstable_uv 0.11.25
• Linux (CI: Ubuntu x86_64 on RunsOn; also reproduced locally on Linux)

Additional context

Analysis and this report are AI-assisted; the evidence above (action hashes, timestamped CI logs,
target counts, source references) is from real runs in our repository and from reading the moon
v2.4.3/v2.4.6 and moonrepo/plugins sources.

[milesj]

Give v2.5 a shot.

[kibumpark-carbonsix]

Tried v2.5.0 — still reproduces, and the action graph is unchanged for this case.

What v2.5.0 changed here. SetupEnvironment now also depends on SetupToolchain for the
package manager and on SyncWorkspace. Same target set, same repo, moon action-graph --dot:

-    3 [ label="SetupProto(0.58.2)"]
+    3 [ label="SetupProto(0.60.2)"]
     4 [ label="SetupToolchain(unstable_python:~3.13)"]
     5 [ label="SetupToolchain(unstable_uv:0.11.25)"]
     6 [ label="SetupEnvironment(unstable_python)"]
     7 [ label="InstallDependencies(unstable_python)"]
    21 [ label="SetupEnvironment(unstable_python, tools)"]
    22 [ label="InstallDependencies(unstable_python, tools)"]
...
+    6 -> 5
+    6 -> 0
+   21 -> 5
+   21 -> 0

Those edges are added to both chain heads, so 6 and 21 remain peers that become ready together.
The two chains still have no edge between them, and install_dependencies_root /
setup_environment_root do not collapse them — their guard is "the located dependencies root is the
workspace root", and this case does not take that path.

Why not, precisely. The repo is a uv workspace with
members = ["apps/py/*", "libs/py/*"]. tools/ is a moon project (language: bash) whose tasks
shell out to python3 tools/<script>, and it is not a member of that uv workspace. The locator
does find a dependency root, but in_dependencies_workspace is false, so the action root is
replaced with project.source:

// If not in the dependencies workspace (if there is one),
// or is a stand-alone project with its own lockfile,
// we must extract the project ID and source (root)
let (project_id, root) = if in_workspace {
    (None, rel_root)
} else if let Some(project) = project {
    (Some(project.id.clone()), project.source.clone())

That comment states the assumption that breaks here: a project outside the dependencies workspace is
treated as stand-alone with its own lockfile. tools/ has neither a lockfile nor a
pyproject.toml — it simply is not a listed member. So moon scopes the provisioning action to
tools, while the uv sync that action runs resolves upward and mutates the workspace .venv.
The environment moon accounts for and the environment uv mutates are not the same one.

That the workspace .venv is created by the tools chain is forced by the graph, not inferred
from the log: 7 -> 6 and 22 -> 21, so the workspace InstallDependencies cannot start until the
workspace SetupEnvironment finishes — and in the failing run that node is the one that fails,
so it never completed and node 7 never ran. The only install action that could have created
<workspace>/.venv before that point is node 22, scoped to tools.

moon's own hash manifests on v2.5.0 record both actors emitting a byte-identical command that
differs only in working directory, which is what makes them resolve to different .venv paths:

actionNode: {"root": "",      "toolchainId": "unstable_python"}
  uv ["venv", ".venv", "--no-managed-python", "--no-python-downloads", "--no-progress"]
  cwd: /…/<workspace>/

actionNode: {"root": "tools", "projectId": "tools", "toolchainId": "unstable_python"}
  uv ["venv", ".venv", "--no-managed-python", "--no-python-downloads", "--no-progress"]
  cwd: /…/<workspace>/tools

The workspace-scoped InstallDependencies manifest carries
"members": ["apps/py/*", "libs/py/*"]; the tools one carries no members at all — moon models
tools as a standalone environment throughout, while uv does not.

A question we could not answer from outside: do these two commands produce the same
cmd.get_cache_key(), and therefore the same {prefix}-{key} lock? The args are byte-identical and
only cwd differs. If cwd is not part of the key, the two actors serialize on one lock even though
they target different paths — and the plugin's "does .venv exist?" decision is made when the
command is emitted, not when it runs after the lock wait. We could not test that from here, so we
are not asserting it; you can answer it immediately.

Reproduction on v2.5.0: 250 cold runs (.venv, tools/.venv, .moon/cache removed before each
run, no warm-up), 1 failure — same order as 2.4.3 (1 in 110 runs on the same machine; a 24-core idle
box rarely interleaves them, our CI runners do so far more often). The failing run:

uv venv .venv --no-managed-python --no-python-downloads --no-progress
Creating virtual environment at: .venv
uv sync --all-packages --no-managed-python --no-python-downloads --no-progress
Creating virtual environment at: /…/<workspace>/.venv          # workspace env, from a sync
  … the tools tasks run and pass …
uv venv .venv --no-managed-python --no-python-downloads --no-progress
Creating virtual environment at: .venv
error: Failed to create virtual environment
  Caused by: A virtual environment already exists at: .venv

Byte-for-byte the failure from the original report.

A deterministic case, so this does not need a scheduler race. The fixture is small: one uv
workspace with a single Python member project, plus one moon project that is not a workspace
member and whose task invokes python3. moon action-graph --dot then shows two
SetupEnvironment/InstallDependencies pairs with no edge between them, both of which reach the
same .venv. The invariant we would suggest asserting: actions that can mutate the same resolved
environment must be canonicalized into one action, or share a barrier that every consumer of that
environment waits on. Keying on the toolchain, the command, or moon's computed root is not enough,
because the resource is the environment uv resolves at execution time.

Note the second symptom in the original report is the same defect on the consumer side, not a
separate bug: an InstallDependencies uv sync reconciles the member set while a task that uses
that environment is already running. A consumer today waits only for its own project's install node,
not for every action able to mutate the environment it runs in.

Environment: moon 2.5.0, unstable_python 3.13, unstable_uv 0.11.25, Linux x86_64.

Our warm-up workaround still contains the creation race on 2.5.0, so we are not blocked. It is
containment, not a fix — it does not prevent a later uv sync from running alongside a task that
uses the environment; it only relies on those syncs being no-ops once the environment is warm.

We are happy to implement this and open a PR, if you'll tell us which direction you'd accept —
the two we see lead to different changes:

1. Canonicalize environment actions by the environment the toolchain actually resolves, so a
   non-member project that resolves upward shares the workspace's SetupEnvironment /
   InstallDependencies nodes instead of getting its own pair. This is the (b) you described on
   Per-OFD flock self-deadlock in CacheEngine::create_lock — process hangs indefinitely when ≥2 projects share a managed toolchain #2524, extended to cover projects outside the dependencies workspace.
2. Barrier: keep the separate actions, but have every action that can mutate a given environment,
   and every task that consumes it, order against a single node for that environment.

(1) is the smaller graph change but needs the toolchain to report the environment it will actually
touch, not just the root moon computed. (2) leaves the current identities alone but touches task
scheduling more broadly. We'd rather ask than guess, since this is your architecture.

As a smaller first step we can send a test-only PR with the fixture described above (one uv
workspace member plus one non-member project invoking python3) and an action-graph snapshot
asserting whichever invariant you pick — useful either way, and it makes any later fix verifiable.
We can also verify a patch of yours against a real workspace before you cut a release.

[milesj]

Thanks for the deep dive. Will throw fable at it.

[milesj]

@kibumpark-carbonsix Question, is tools actually a python project? Or is it just some kind of directory?

[kibumpark-carbonsix]

Warning

Superseded. The explanation in this comment is wrong: the toolchains declarations on pre-commit and audit-python are task-local and do not propagate project-wide. The "every task" observation was real but had a different cause — see the corrected account. Kept for the thread record.

---

Neither, exactly — and my earlier description of it was sloppy, sorry. Let me correct it.

tools is a moon project, but not a Python package:

# tools/moon.yml
id: "tools"
language: "bash"
layer: "tool"

No pyproject.toml, no lockfile, and it is not a member of the uv workspace
(members = ["apps/py/*", "libs/py/*"]). It holds repo-wide maintenance and CI scripts — policy
checks, doc lints, drift guards. We register it as a moon project so those scripts are visible in
the graph and runnable as tasks, not because anything there is installable.

What attaches the Python toolchain to it is not the python3 … commands I mentioned before —
it is two tasks that declare the toolchains explicitly, because they run repo-wide tooling from the
workspace's dev dependency group:

pre-commit:
  command: "uv run --locked pre-commit run --all-files --show-diff-on-failure"
  toolchains: ["unstable_python", "unstable_uv"]
audit-python:
  command: "uv run --locked pip-audit"
  toolchains: ["unstable_python", "unstable_uv"]

That resolves onto every task in the project, including ones that touch neither Python nor uv:

$ moon query tasks
adapters-selftest    toolchains=['unstable_uv','unstable_python']   cmd='python3'
pre-commit           toolchains=['unstable_uv','unstable_python']   cmd='uv'
policy-check         toolchains=['unstable_uv','unstable_python']   cmd='sh'

And a single one of them is enough to create the second environment chain:

$ moon action-graph --dot tools:policy-check
    3 [ label="SetupToolchain(unstable_python:~3.13)"]
    4 [ label="SetupToolchain(unstable_uv:0.11.25)"]
    5 [ label="SetupEnvironment(unstable_python, tools)"]
    6 [ label="InstallDependencies(unstable_python, tools)"]
    7 [ label="RunTask(tools:policy-check)"]

So the shape of it is: these tasks want the workspace environment — uv run --locked pre-commit resolves upward to the workspace .venv and the dev group installed there — while moon
scopes their provisioning to tools, because tools is not a workspace member. That mismatch is
the whole bug from our side.

Which turns into a question for you, and it may be the more useful one: what is the intended way
to say "this task runs in the dependency workspace's environment" for a project that is not itself a
member of that workspace? If declaring toolchains: ["unstable_python", "unstable_uv"] on a
non-member project is not that, we are happy to be told we configured it wrong and restructure — for
instance by making the repo-wide tooling a real workspace member. If it is the supported way, then
the provisioning scope should follow the environment the toolchain resolves rather than the
project's membership.

Either answer is useful to us. If it is our misconfiguration, note that the underlying divergence
still stands on its own: any non-member project whose uv invocation resolves upward gets an action
scoped to itself while mutating the parent environment.

[kibumpark-carbonsix]

Note

Partly superseded. The correction here is right — toolchains resolve per task — but it is incomplete, and the caching speculation at the end is withdrawn: the trigger is a tools/.venv directory, not a cached artifact. See the corrected account.

---

Correction to my previous reply — the "correction" in it was wrong. My original account was right.

Toolchains resolve per task, from the command. The explicit toolchains declarations on
pre-commit and audit-python are task-local and do not propagate project-wide:

tools:adapters-selftest   toolchains=['unstable_uv','unstable_python']   cmd='python3'
tools:pre-commit          toolchains=['unstable_uv','unstable_python']   cmd='uv'
tools:policy-check        toolchains=['system']                          cmd='sh'
tools:wiki-lint           toolchains=['system']                          cmd='sh'
tools:audit-rust          toolchains=['rust']                            cmd='cargo-deny'

moon action-graph --dot tools:policy-check therefore has no provisioning nodes —
SyncWorkspace -> SyncProject(tools) -> RunTask and nothing else — while
moon action-graph --dot tools:adapters-selftest (a plain python3 … task with no declaration of
its own) does create SetupEnvironment(unstable_python, tools) and
InstallDependencies(unstable_python, tools).

So please disregard the "every task in the project" claim and both excerpts in my previous reply.
The tools-scoped chain in the reproduction comes from the selected python3 … tasks, exactly as my
first reply described.

How I got it wrong is maybe worth a line, since it may be a bug of its own: the query and graph I
pasted came from a long-lived working copy, and a clean checkout of the same commit with the same
moon version resolves differently — system instead of unstable_python/unstable_uv for the
sh tasks. --cache off did not change the stale answer. I have not isolated which cached artifact
carries it, so I am not filing anything yet; if I can reproduce it from scratch I will open a
separate issue.

Two smaller corrections while I am here:

1. I misread the source comment I quoted. It reads "If not in the dependencies workspace (if there
   is one), or is a stand-alone project with its own lockfile" — two separate conditions routed
   to the same branch, not an assumption that anything outside the workspace is standalone. The
   behavior I described is still what we observe; my gloss on the comment was wrong.
2. I said uv run resolves upward from tools. It does not — those tasks set
   runFromWorkspaceRoot: true, so they already start at the workspace root. The upward resolution
   that matters is the one inside the tools-scoped provisioning action, whose recorded cwd is
   <workspace>/tools.

None of this changes the reproduction or the graph evidence in my first reply, which stand.

The configuration question, narrowed to the case that actually exists: for a project that is not a
member of the dependency workspace but whose tasks resolve the Python/uv toolchains from their
commands, is there a supported way to make its SetupEnvironment / InstallDependencies actions
target the dependency workspace's environment instead of a project-scoped one?

[kibumpark-carbonsix]

I isolated the discrepancy from my last reply, and I withdraw the caching speculation in it. The
cache controls when the change becomes visible, not what causes it.

An empty tools/.venv directory is enough to make moon attach the Python/uv toolchains to a
language: bash project. tools/ has no Python package or dependency manifest and no lockfile.
Resolution of the plain-sh task tools:policy-check, clearing .moon/cache before every probe:

                                        moon 2.4.3                          moon 2.5.0
no tools/.venv                          ['system']                          ['system']
empty tools/.venv (mkdir, no files)     ['unstable_uv','unstable_python']   ['unstable_uv','unstable_python']
tools/.venv created by that moon        ['unstable_uv','unstable_python']   ['unstable_uv','unstable_python']

The project's language stays bash; what changes is the resolved toolchains, and they are
inherited by every task in the project. With the directory present, moon action-graph --dot tools:policy-check contains SetupEnvironment(unstable_python, tools) and
InstallDependencies(unstable_python, tools). Without it, that same sh task resolves ['system']
and has no provisioning nodes at all.

Where the directory comes from. Not from us — moon's own tools-scoped provisioning creates it,
the uv venv .venv with cwd: <workspace>/tools from the manifest I quoted earlier. Row 3 is that
sequence with nothing pre-seeded: remove tools/.venv, clear the workspace graph so it is rebuilt
artifact-free, run one python3 … task in tools, observe moon recreate the directory, clear the
graph again, then re-query. Under both versions the project's sh tasks come back carrying the
Python toolchains.

So the provisioning action's output feeds the detection that decides which tasks get a provisioning
action. That widens the set of target selections whose action graph includes the tools-scoped chain.
I have not measured an increase in race frequency and am not claiming one — only that the
surface that can produce the collision grows.

Two qualifications so this is not read as more than it is:

• The transition is not instantaneous. Creating or removing tools/.venv does not change an
  already-cached workspace graph, and --cache off does not clear it; the resolution changes when
  the graph is next rebuilt. That lag is what sent me chasing a caching bug.
• This does not vindicate my second reply. Its stated cause — the explicit toolchains declarations
  on pre-commit and audit-python propagating project-wide — is still wrong; they are task-local.
  My third reply was right about that and about the artifact-free checkout, but its categorical
  "toolchains resolve per task, from the command" was incomplete: project-level detection can add
  inherited toolchains on top.

The question this raises: should a bare .venv directory count as evidence of project-level
Python usage for a project that declares language: bash and carries no Python manifest — when that
directory is something moon generated itself? Excluding generated environment directories from
detection, or not scoping an environment into a project that has no Python manifest, would each cut
this path.

Configured toolchains: unstable_python 3.13 and unstable_uv 0.11.25, Linux x86_64. One detail
worth flagging separately: the pyvenv.cfg in the environments created during row 3 records
uv = 0.12.3, not the configured 0.11.25 — so the provisioning command did not run the pinned uv on
this machine. Both moon versions were run from the same tracked worktree with the .prototools pin
swapped and .moon/cache cleared before each query.

[milesj]

Ok tools not being a "real" python project makes sense in how this is happening.

moon has been making the assumption that all projects would be in the dependency workspace (uv/etc), or the project itself is the workspace. But for this situation, it's neither.

Right now it creates the graph like RunTask -> InstallDeps -> SetupEnv(python) -> SetupToolchain(python). But this isn't correct since it's not in the dep workspace.

I think changing the graph to simply RunTask -> SetupToolchain(python) should fix this. That way we don't create the .venv, nor install Python dependencies, but it ensures Python has been installed.

[kibumpark-carbonsix]

That matches what we see, and the graph you described is exactly the one both kinds of tools task
get today. On 2.4.3, moon action-graph --dot:

tools:pre-commit   (uv run --locked pre-commit …, declares unstable_python + unstable_uv)
tools:codegen-check (python3 tools/docs-nav …, declares nothing)

  → identical chain for both:
    3 SetupToolchain(unstable_python:~3.13)
    5 SetupEnvironment(unstable_python, tools)
    6 InstallDependencies(unstable_python, tools)
    7 RunTask(tools:<task>)
    5 -> 3    6 -> 5    7 -> 6

Worth stating because it separates two groups that your change treats very differently.

For the python3 tools/<script> tasks your fix is simply right. There are twenty-eight of them,
none declaring the Python/uv toolchains, and their scripts import only the standard library plus
sibling modules inside tools/ — no third-party package, including under lazy/indented imports. They get a
.venv and a uv sync today that they have no use for. RunTask -> SetupToolchain(python) is what
they actually need.

That also closes the loose end from my last comment: with tools/.venv absent and .moon/cache
cleared, a python3 … task still resolves unstable_python/unstable_uv and still gets the full
chain, while the sh tasks resolve to nothing at all (SyncWorkspace -> SyncProject -> RunTask).
So the bare-.venv-directory detection I reported only ever flipped the sh tasks; the python3
tasks resolve from the command, as you'd expect.

What the change removes. Two actors we have evidence for:

• the second uv venv .venv — the tools-scoped SetupEnvironment — which is one half of the
  reported collision;
• the tools-scoped InstallDependencies, whose command is a plain uv sync. In this workspace
  shape, a plain uv sync resolving to the workspace root prunes workspace members that were
  installed only by --all-packages. That is the second symptom in the original report, and it
  reproduces deterministically with no scheduler race involved. uv 0.11.25, three-file fixture
  (non-package workspace root, members = ["libs/*"], one member demo):

$ uv sync --all-packages
 + demo==0.0.0 (from file:///…/libs/demo)
$ uv sync
Uninstalled 1 package in 0.74ms
 - demo==0.0.0 (from file:///…/libs/demo)

Both mechanisms we can point at go away. We would not claim more than that, because of the next
part.

What it leaves open: the two tasks that do consume the workspace environment. They are
uv run --locked pre-commit … and uv run --locked pip-audit, both runFromWorkspaceRoot: true,
so they execute at the workspace root against the workspace .venv and the dev dependency
group installed into it. Today RunTask has an edge to the tools-scoped InstallDependencies,
which is what ensures that group is present in a standalone tools invocation. After the change the
proposed tools chain contains no action that ensures it before RunTask; a workspace-scoped
InstallDependencies may exist elsewhere in a mixed graph, but nothing orders it against the tools
task. The command then provisions the environment itself:

$ ls .venv
ls: cannot access '.venv': No such file or directory
$ uv run --locked python -c "print('ran')"
Creating virtual environment at: .venv
ran

In a mixed invocation that also schedules a real workspace Python task, RunTask(tools:pre-commit)
would wait only on SetupToolchain, while the workspace-scoped SetupEnvironment — uv venv .venv,
without --allow-existing — has no edge ordering it against the tools task, and both can target
<workspace>/.venv when it is initially absent. We have not built your patch, so that is a schedule
the proposed graph appears to permit, not a failure we reproduced.

The question that decides what you build. For a task that belongs to a project outside the
dependency workspace but runs from the workspace root and consumes that workspace's environment, is
moon's intended contract:

(a) toolchain-only — provisioning the shared environment is the command's business, and moon does
not order it; or

(b) something the task can declare, so the graph can produce
RunTask(tools:pre-commit) -> InstallDependencies(<workspace>) -> SetupEnvironment(<workspace>)
without making tools/ a uv member?

Task-level rather than project-level matters here, since this one project holds both groups above —
twenty-eight tasks that want (a) and two that want (b).

If (a) is the intended boundary, that answers our question and we will restructure on our side. We
would not suggest inferring workspace consumption from runFromWorkspaceRoot: most root-running
tasks in tools are the stdlib-only ones and declare no Python/uv toolchain, so execution cwd alone
is not a sufficient signal.

One orthogonal data point. uv venv --allow-existing succeeds when a fully created environment
already exists and, in our sequential same-interpreter fixture, left the installed packages in
place. We did not test overlapping creation or a partially initialized .venv, so this is not
evidence that it makes concurrent setup safe — only that uv offers a non-destructive alternative
here, where its own error hint points at --clear.

Offer stands. We can send the three-file fixture as a test-only PR with an action-graph
snapshot for whichever invariant you choose; that is the deterministic check of the graph change.
Separately, once a patched build exists, we can rerun the same cold harness against the real
workspace to look for residual failures outside that fixture. Since the baseline produced only one
failure in 250 runs, a clean stress run would be corroboration, not proof.

[milesj]

Try v2.5.1.

[kibumpark-carbonsix]

Tried v2.5.1. The reported case is fixed — thank you.

Graph. Every tools task that resolves the Python/uv toolchains now gets
RunTask -> SetupToolchain(unstable_python) / SetupToolchain(unstable_uv) and nothing
environment-related; the tools-scoped SetupEnvironment / InstallDependencies pair is gone. A
mixed selection (5 python3 tools/<script> tasks + 3 workspace-member tasks) has one provisioning
pair (workspace) instead of two.

Cold stress. 250 cold runs of that mixed set (.venv, tools/.venv, .moon/cache removed
before each run, no warm-up, 24-core box): 0 failures, exactly one uv venv per run. Same removal
protocol as the v2.5.0 run that gave 1/250 (different task mix). Both mechanisms in the original
report — the second uv venv and the tools-scoped plain uv sync — are gone by construction, so
this is corroboration, not proof.

What it leaves — the schedule from my previous comment reproduced in every cold run here. For a
non-member task that consumes the workspace environment (uv run --locked … with
runFromWorkspaceRoot, e.g. our tools:pre-commit), a cold mixed run failed every time on this
machine:

tools:pre-commit + domain-kernel:lint, cold:              3/3 failed
synthetic mirror of the same task shape:                 50/50 failed
same, with UV_NO_MANAGED_PYTHON=1 in the task's env:     20/20 failed

(The last row makes the task's uv run resolve the same proto interpreter the plugin uses, so the
collision does not depend on the interpreter divergence in observation 2.)

MOON_LOG=debug timeline of one run (ms):

36.973  SetupEnvironment(unstable_python) and RunTask(tools:…) start in the same ms
36.996  spawn: uv venv .venv --no-managed-python …            (SetupEnvironment)
37.053  spawn: uv run --locked …                              (the task, from the workspace root)
        the task's uv run creates .venv first
37.139  uv venv: "A virtual environment already exists at: .venv"
37.140  SetupEnvironment recorded Failed; pipeline abort begins

Why it is newly exposed: on 2.4.x/2.5.0 the task ran only after its own tools-scoped
InstallDependencies, whose uv sync had already created the workspace .venv (the trace in the
original report), so the task never had a venv to create. On 2.5.1 nothing environment-related
precedes it, so its implicit venv creation races the workspace uv venv — and here it wins every
time. We did not run this probe on 2.4.x, so the before/after is structural, not measured.
RunTask still has no edge to the workspace SetupEnvironment / InstallDependencies — the graph
is exactly the one you described, and that is the gap.

We are not blocked: our serial warm-up (MOON_CONCURRENCY=1 moon run <member task> before the
parallel run) still contains it — 20/20 warm runs of the synthetic probe pair pass, and
SetupEnvironment emits no uv venv once the directory exists. But whenever such a non-member
task is selected in a cold pipeline that also provisions the workspace environment, the graph
exposes it to this unordered race unless the environment is warmed first.

So the question from my last comment is the one that remains: for a task in a non-member project
that runs at the workspace root and consumes that workspace's environment, is there (or should
there be) a task-level way to order it after the workspace SetupEnvironment /
InstallDependencies? Task-level, because the same project mixes many tasks that want exactly what
2.5.1 does now with a few that consume the workspace environment. If the intended answer is "that
is the command's business, not moon's", that's a fine answer too — we would then restructure on our
side.

Two smaller observations from the same debug log, in case they are useful:

1. After SetupEnvironment was recorded as Failed (37.140), its required dependent
   InstallDependencies was dispatched in the same millisecond, before the abort; it ran to
   completion (status=Passed, 37.431) and was marked aborted only afterwards. The abort did
   terminate the running task child. So a dependent ran even though its required dependency failed —
   here it was uv sync, which removed and recreated the venv, so it did mutate the environment.
2. The task's bare uv run chose a uv-managed CPython 3.13.12 (present on this dev box, preferred
   by uv for .python-version = 3.13), while the plugin's --no-managed-python commands resolved
   proto's 3.13.14 — which is why the later uv sync "Removed virtual environment" and recreated it.
   Machine/config-specific and mostly ours to configure; with UV_NO_MANAGED_PYTHON=1 the task also
   picked 3.13.14, the recreate step disappeared and the collision stayed. So the divergence
   explains the recreation, not the collision.

Environment: moon 2.5.1, unstable_python 3.13, unstable_uv 0.12.1, Linux x86_64. Offer stands
for the fixture / action-graph snapshot PR if you want the non-member-consumer case pinned down.

[milesj]

I fixed #1 in v2.5.2. #2 is a little more tricky, will look into.

for a task in a non-member project
that runs at the workspace root and consumes that workspace's environment, is there (or should
there be) a task-level way to order it after the workspace SetupEnvironment /
InstallDependencies?

This one is a bigger lift, but I'll look into it for v2.6.