refactor: kind of everything #421

MuntasirSZN · 2025-09-06T09:34:13Z

No description provided.

- Optimize Loc::new() to eliminate string allocations during byte-to-char conversion - Replace HashMap with IndexMap for better cache performance in models - Use SmallVec for commonly small collections (ranges, statements, declarations) - Flatten cache structure with combined keys for better memory layout - Optimize merge operations to use HashSet for O(1) lookup instead of dedup_by - Add helper functions for efficient data structure conversions - Use more efficient parallel processing patterns Co-authored-by: MuntasirSZN <[email protected]>

- Update miri_tests.rs to use constructors instead of direct struct initialization - Replace Vec::new() with appropriate SmallVec constructors in tests - Fix capacity assertions to match SmallVec minimums - Ensure all tests compile and run correctly with new optimized data structures Co-authored-by: MuntasirSZN <[email protected]>

…omprehensive configuration Co-authored-by: MuntasirSZN <[email protected]>

Co-authored-by: MuntasirSZN <[email protected]>

…timizations Co-authored-by: MuntasirSZN <[email protected]>

chore: another clippy fix Update docs/cache-configuration.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Update src/bin/core/cache.rs Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Update src/cache.rs Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> remove allow unused

Co-authored-by: MuntasirSZN <[email protected]>

…used code Co-authored-by: MuntasirSZN <[email protected]>

… checks Co-authored-by: MuntasirSZN <[email protected]>

Co-authored-by: MuntasirSZN <[email protected]>

…opying code Co-authored-by: MuntasirSZN <[email protected]>

Co-authored-by: MuntasirSZN <[email protected]>

…dhash

MuntasirSZN · 2025-12-23T15:38:36Z

Also pls set up codecov, manual code coverage is error prone

MuntasirSZN · 2025-12-23T15:46:31Z

Also codspeed to continuous benches

cordx56 · 2025-12-25T16:45:11Z

@MuntasirSZN There are a lot of changes, and you didn't provide enough description of the motivation. I felt most optimization is not needed, and it makes reading the code difficult.

MuntasirSZN · 2025-12-25T17:06:56Z

@MuntasirSZN There are a lot of changes, and you didn't provide enough description of the motivation. I felt most optimization is not needed, and it makes reading the code difficult.

I disagree. I myself cannot run rustowl, just because it just is tooooo much resource hungry. Everyone doesnt run beefy machines. Also about code, it isnt that hard, just some optimizations that make it faster, and they are known to many people (caching).

cordx56 · 2025-12-25T21:21:25Z

Yes, RustOwl needs many resources. But your change resolves the situation significantly? I didn't measured, but if you can, please describe and show the data. I feel the change sacrifices readability.

MuntasirSZN · 2025-12-26T03:49:02Z

Yes, RustOwl needs many resources. But your change resolves the situation significantly? I didn't measured, but if you can, please describe and show the data. I feel the change sacrifices readability.

Ok i will. I am busy for some days so it will take some time

MuntasirSZN · 2025-12-29T04:23:03Z

`rustowl check perf-tests/dummy-package`

Cold build (aka first run no cache)

command: hyperfine "rm -rf perf-tests/dummy-package/target/owl && ./target/debug/rustowl check ./perf-tests/dummy-package"

~/.rustowl dir exists, no download needed
No cache, full check must happen

`main` branch

Time (mean ± σ):     33.889 s ±  0.825 s    [User: 67.681 s, System: 12.357 s]
Range (min … max):   32.203 s … 35.199 s    10 runs

Metric	Value
User time (wall clock time)	67.681s
System time	12.357s

this branch

Time (mean ± σ):     25.153 s ±  2.306 s    [User: 59.941 s, System: 11.364 s]
Range (min … max):   22.177 s … 30.916 s    10 runs

Metric	Value
User time (wall clock time)	59.941s
System time	11.364s

Summary

This branch is about 25.8% faster than main in cold build scenario.

Warm build (aka second run with cache)

~/.rustowl dir exists, no download needed
Cache exists, only incremental check needed

command: hyperfine "rm -rf ./target/debug/rustowl check ./perf-tests/dummy-package"

`main` branch

Time (mean ± σ):     10.729 s ±  0.661 s    [User: 1.229 s, System: 0.692 s]
Range (min … max):   10.112 s … 11.943 s    10 runs

Metric	Value
User time (wall clock time)	1.229s
System time	0.692s

this branch

Time (mean ± σ):      1.341 s ±  0.072 s    [User: 0.878 s, System: 0.265 s]
Range (min … max):    1.227 s …  1.444 s    10 runs

Metric	Value
User time (wall clock time)	1.341s
System time	0.265s

Summary

This branch is about 87.5% faster than main in warm build scenario.

cordx56 · 2025-12-29T06:10:13Z

Okay

MuntasirSZN · 2025-12-29T06:54:07Z

@cordx56 can you do a announcement on github discussions and discord with [at]everyone to test this as much as possible? After merge of course, to minimize as much bugs as possible.

Currently its 99% complete, the xtask refactor will be another pr. Pls review completely.

cordx56

reviewing

PROMPT.md

cordx56 · 2025-12-29T06:35:35Z

src/bin/core/analyze/transform.rs

+        body.var_debug_info.len(),
+        foldhash::quality::RandomState::default(),
+    );
+    for debug in &body.var_debug_info {


Why didn't you use iterator idiom? I feel this loop idiom is too redundant.

cordx56 · 2025-12-29T06:42:06Z

src/bin/core/analyze/transform.rs

+
+    let mut result = EcoVec::with_capacity(basic_blocks.len());
+
+    for (_bb, bb_data) in basic_blocks.iter_enumerated() {


Same as above.

cordx56 · 2025-12-29T06:48:41Z

src/utils.rs

+/// `rustc` byte positions behave as if `\r` bytes do not exist in the source.
+/// `Loc` is a *logical character index* where `\r` is ignored too.
+///
+/// `Loc::new(source, byte_pos, offset)` previously scanned `source` on every


I thought the comment about previous implementation is needless.

same. please left any comments if you resolve the conversation without fixing the code.

I removed comment? @cordx56

Do you want to remove last two lines? (though rgey are just context)

src/bin/core/cache.rs

src/bin/rustowl.rs

src/bin/rustowlc.rs

src/cache.rs

cordx56 · 2025-12-29T07:54:59Z

@MuntasirSZN checks failed. please fix

cordx56

Please read my comment carefully.

src/bin/rustowl.rs

src/lib.rs

cordx56 · 2025-12-30T08:01:35Z

src/lsp/analyze.rs


 impl Analyzer {
-    pub async fn new(path: impl AsRef<Path>, rustc_threads: usize) -> Result<Self, ()> {
+    pub async fn new(path: impl AsRef<Path>, rustc_threads: usize) -> Result<Self> {


Why you remove second type argument?

There arent any errors expected?

cordx56 · 2025-12-30T08:03:38Z

src/toolchain.rs

+    }
+
+    #[test]
+    fn test_recursive_read_dir_non_existent() {


Why you resolve this comment without any comments?

src/lsp/analyze.rs

src/lsp/backend.rs

cordx56 · 2025-12-30T08:13:17Z

src/models.rs

+    }
+
+    #[test]
+    fn test_file_operations() {


Why you resolved this without editing or commenting?

cordx56 · 2025-12-30T08:14:01Z

src/shells.rs

+    }
+
+    #[test]
+    fn test_shell_error_message_patterns() {


cordx56 · 2025-12-30T08:15:51Z

src/utils.rs

+/// `rustc` byte positions behave as if `\r` bytes do not exist in the source.
+/// `Loc` is a *logical character index* where `\r` is ignored too.
+///
+/// `Loc::new(source, byte_pos, offset)` previously scanned `source` on every


same. please left any comments if you resolve the conversation without fixing the code.

cordx56 · 2025-12-30T08:24:18Z

src/lsp/analyze.rs

+    pub async fn new(path: impl AsRef<Path>, rustc_threads: usize) -> Result<Self> {
        let path = path.as_ref().to_path_buf();

+        // `cargo metadata` may invoke `rustc` for target information. It must use a real


false. setup_cargo_command does not use rustc. rustc version may differ from rustowlc version.

cordx56 · 2025-12-30T08:31:01Z

Some comments are wrong. Please check carefully before writing docs and codes.

MuntasirSZN · 2025-12-30T14:19:41Z

@cordx56 sorry before i say anything, but as a maintainer you can do some edits too instead of just asking me. I am saying this because i just changed houses and i am busy as heck. Also you have much more knowledge about this then me (expected). So pls solve any issues as you see put, in any case if you want my opinion write a review comment.

Sorry again......

- Update to use reqwest v0.13 (we now use default features as some are default) - Remove rustls (aws_lc is now used by default) - No more rustls native roots, we use rustls platform verifier (better) which uses aws_lc and others according to platform (not openssl though)

Copilot AI and others added 30 commits September 2, 2025 20:41

feat: implement robust caching system with intelligent eviction and c…

e1115fd

…omprehensive configuration Co-authored-by: MuntasirSZN <[email protected]>

style: format code according to rustfmt standards

8f1019c

Co-authored-by: MuntasirSZN <[email protected]>

chore: lint fix

898758e

fix: resolve all clippy warnings with code improvements and boxing op…

8426154

…timizations Co-authored-by: MuntasirSZN <[email protected]>

fix: remove unused smallvec macro import

8758985

Co-authored-by: MuntasirSZN <[email protected]>

fix: apply coderabbitai review fixes for cache, documentation, and un…

d83e2ff

…used code Co-authored-by: MuntasirSZN <[email protected]>

refactor: use target/rustowl/cache, bring back dead_code and unused

f1293db

fix: prevent cache memory overshoot by adding post-insertion eviction…

57d6da3

… checks Co-authored-by: MuntasirSZN <[email protected]>

chore: apply fixes

e555ccb

chore: format

0f96914

chore: fix

98ce316

chore: remove unused import

cd118b9

feat: implement error handling with eros crate

c8a3c5d

Co-authored-by: MuntasirSZN <[email protected]>

feat: migrate from tower-lsp to tower-lsp-server for better maintenance

e2ebac7

Co-authored-by: MuntasirSZN <[email protected]>

refactor: improve shells.rs to delegate to clap_complete instead of c…

c8c6f6d

…opying code Co-authored-by: MuntasirSZN <[email protected]>

docs: add comprehensive documentation to core modules

fd4c969

Co-authored-by: MuntasirSZN <[email protected]>

test: add comprehensive unit tests for utils, error, and shells modules

75fae53

Co-authored-by: MuntasirSZN <[email protected]>

perf: optimize string operations and improve error handling robustness

5ea3ff9

Co-authored-by: MuntasirSZN <[email protected]>

chore: fix deps and issues

dab8dde

chore: cleanup

47aba25

chore: cleanup again

3ac859c

chore: format

0be4ce6

chore: remove pub extern crate

f7dfd5f

chore: remove tracing again

d28d08f

chore: invasive replace all hashmaps and hashsets with indexmap + fol…

c0ae6b0

…dhash

chore: fix, use quality foldhash hasher, fix seen_ids|

717fc6a

chore: fix tests

aa5c873

MuntasirSZN removed the do-build-check Execute build check on PR label Dec 23, 2025

MuntasirSZN added 2 commits December 23, 2025 21:46

refactor!: more perf, acknowledge review comments

c50a868

Merge remote-tracking branch 'upstream/main' into feat/test-frenzy

f7bb77d

MuntasirSZN requested a review from cordx56 December 23, 2025 16:05

MuntasirSZN added the do-build-check Execute build check on PR label Dec 23, 2025

chore: done

d86a781

MuntasirSZN added do-build-check Execute build check on PR and removed do-build-check Execute build check on PR labels Dec 29, 2025

cordx56 reviewed Dec 29, 2025

View reviewed changes

MuntasirSZN added 4 commits December 29, 2025 18:54

Delete PROMPT.md

69b7a5e

Update utils.rs

2cdc554

Update cache.rs

9ac8672

Update rustowlc.rs

230d4dc

cordx56 reviewed Dec 30, 2025

View reviewed changes

MuntasirSZN added 2 commits December 31, 2025 10:21

fix: allow webpki license

cf738cf

MuntasirSZN removed the do-build-check Execute build check on PR label Dec 31, 2025

fix: acknowledge comments?

09ee9bb


		let mut result = EcoVec::with_capacity(basic_blocks.len());

		for (_bb, bb_data) in basic_blocks.iter_enumerated() {

Uh oh!

refactor: kind of everything #421

Are you sure you want to change the base?

refactor: kind of everything #421

Uh oh!

Conversation

MuntasirSZN commented Sep 6, 2025

Uh oh!

MuntasirSZN commented Dec 23, 2025

Uh oh!

MuntasirSZN commented Dec 23, 2025

Uh oh!

cordx56 commented Dec 25, 2025

Uh oh!

MuntasirSZN commented Dec 25, 2025

Uh oh!

cordx56 commented Dec 25, 2025

Uh oh!

MuntasirSZN commented Dec 26, 2025

Uh oh!

MuntasirSZN commented Dec 29, 2025

rustowl check perf-tests/dummy-package

Cold build (aka first run no cache)

main branch

this branch

Summary

Warm build (aka second run with cache)

main branch

this branch

Summary

Uh oh!

cordx56 commented Dec 29, 2025

Uh oh!

MuntasirSZN commented Dec 29, 2025

Uh oh!

cordx56 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cordx56 Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cordx56 commented Dec 29, 2025

Uh oh!

cordx56 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

`rustowl check perf-tests/dummy-package`

`main` branch

`main` branch

cordx56 Dec 29, 2025 •

edited

Loading