Skip to content

Commit a8441bf

Browse files
authored
refactor: Change concurrency approach (#100)
## Description Before, the concurrency approach for slots / handles for individual hashes was based on a top level task pool and "slots" that were managed in a map in the main actor. There were some tricky race conditions for the case where a handle would be "revived" when it was already executing its Drop fn - which does crucial things like writing the bitfield file. Also, there was actual parallelism per hash, which is probably not beneficial at all. Now basically each hash runs effectively single threaded, meaning that we can later go from actual mutexes to more lightweight synchronisation primitives like https://crates.io/crates/atomic_refcell . Unfortunately everything must still be Send due to the fact that we run this whole thing on a multi-threaded executor 🤷 , thank you tokio. Otherwise we could just use a [RefCell](https://doc.rust-lang.org/std/cell/struct.RefCell.html). Now the concurrency is based on a task pool that will always contain at most a single task per hash. Multiple tasks that operate on the same hash are being handled concurrently, but not in parallel, using a `FuturesUnordered`. The drop case is handled in a cleaner way - when an actor becomes idle, it "gives back" its state to the owner - the manager actor. If a task is being spawned while drop runs, these tasks go into the inbox and the actor gets revived immediately afterwards. The manager also has a pool of inactive actors to prevent reallocations. All this is abstracted away by the entity_manager. The entire entity_manager module could at some point become a separate crate, but for now I have inlined it so I don't need to do another crate. ## Breaking Changes <!-- Optional, if there are any breaking changes document them, including how to migrate older code. --> ## Notes & open questions <!-- Any notes, remarks or open questions you have to make about the PR. --> ## Change checklist - [ ] Self-review. - [ ] Documentation updates following the [style guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text), if relevant. - [ ] Tests if relevant. - [ ] All breaking changes documented.
1 parent 35a8189 commit a8441bf

File tree

9 files changed

+1583
-248
lines changed

9 files changed

+1583
-248
lines changed

Cargo.lock

Lines changed: 7 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ testresult = "0.4.1"
5858
tracing-subscriber = { version = "0.3.19", features = ["fmt"] }
5959
tracing-test = "0.2.5"
6060
walkdir = "2.5.0"
61+
atomic_refcell = "0.1.13"
6162
iroh = { version = "0.91", features = ["discovery-local-network"]}
6263

6364
[features]

src/metrics.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ use iroh_metrics::{Counter, MetricsGroup};
44

55
/// Enum of metrics for the module
66
#[allow(missing_docs)]
7+
#[allow(dead_code)]
78
#[derive(Debug, Default, MetricsGroup)]
89
#[metrics(name = "iroh-blobs")]
910
pub struct Metrics {

0 commit comments

Comments
 (0)