Most time consuming test files

[blyxyas]

Recently, I've run an experiment using SPEEDTEST of, benchmarking each test file 100 times and getting the average time, then recollecting all that data into a single file containing average of milliseconds + number of lines in that test file (+ filename), and using a Python script (had to de-rust here, I can now see why people use Python for data science hehe :3)

So, now we have a list of the top 150 most consuming lints out there.

Milliseconds	Lines	File
108	11	tests/ui/print_stderr.rs
1012	103	tests/ui/default_trait_access.rs
1209	122	tests/ui/shadow.rs
825	82	tests/ui/mut_mut.rs
930	92	tests/ui/almost_complete_range.rs
912	89	tests/ui/redundant_field_names.rs
902	88	tests/ui/min_ident_chars.rs
205	20	tests/ui/unnested_or_patterns2.rs
205	20	tests/ui/vec_resize_to_zero.rs
205	20	tests/ui/default_instead_of_iter_empty.rs
206	20	tests/ui/tabs_in_doc_comments.rs
207	20	tests/ui/ptr_offset_with_cast.rs
306	29	tests/ui/reversed_empty_ranges_fixable.rs
203	19	tests/ui/tests_outside_test_module.rs
107	10	tests/ui/double_neg.rs
312	29	tests/ui/skip_while_next.rs
410	38	tests/ui/iter_skip_next.rs
206	19	tests/ui/reversed_empty_ranges_unfixable.rs
206	19	tests/ui/string_add_assign.rs
206	19	tests/ui/zero_ptr_no_std.rs
207	19	tests/ui/floating_point_exp.rs
1018	91	tests/ui/equatable_if_let.rs
213	19	tests/ui/filter_map_next.rs
852	75	tests/ui/single_call_fn.rs
205	18	tests/ui/inline_fn_without_body.rs
205	18	tests/ui/writeln_empty_string.rs
915	80	tests/ui/manual_rem_euclid.rs
206	18	tests/ui/borrow_as_ptr.rs
288	25	tests/ui/bind_instead_of_map.rs
105	9	tests/ui/deprecated_old.rs
247	21	tests/ui/filter_map_next_fixable.rs
106	9	tests/ui/str_to_string.rs
107	9	tests/ui/builtin_type_shadow.rs
1980	166	tests/ui/format_args.rs
920	77	tests/ui/inconsistent_struct_constructor.rs
205	17	tests/ui/println_empty_string.rs
956	79	tests/ui/needless_return_with_question_mark.rs
209	17	tests/ui/floating_point_logbase.rs
911	74	tests/ui/iter_over_hash_type.rs
137	11	tests/ui/if_not_else_bittest.rs
918	73	tests/ui/borrow_deref_ref.rs
954	75	tests/ui/let_underscore_untyped.rs
217	17	tests/ui/filter_map_identity.rs
205	16	tests/ui/entry_btree.rs
205	16	tests/ui/is_digit_ascii_radix.rs
205	16	tests/ui/unchecked_duration_subtraction.rs
206	16	tests/ui/unknown_clippy_lints.rs
206	16	tests/ui/unused_rounding.rs
207	16	tests/ui/short_circuit_statement.rs
104	8	tests/ui/non_minimal_cfg2.rs
406	31	tests/ui/uninlined_format_args_panic.rs
106	8	tests/ui/exit3.rs
214	16	tests/ui/create_dir.rs
108	8	tests/ui/string_to_string.rs
405	30	tests/ui/thread_local_initializer_can_be_made_const.rs
205	15	tests/ui/int_plus_one.rs
205	15	tests/ui/len_zero_ranges.rs
205	15	tests/ui/reversed_empty_ranges_loops_unfixable.rs
206	15	tests/ui/map_collect_result_unit.rs
206	15	tests/ui/repeat_once.rs
207	15	tests/ui/flat_map_identity.rs
207	15	tests/ui/transmute_int_to_char.rs
838	59	tests/ui/single_range_in_vec_init.rs
1213	85	tests/ui/useless_attribute.rs
229	16	tests/ui/iter_next_loop.rs
960	67	tests/ui/needless_arbitrary_self_type.rs
1195	81	tests/ui/filter_map_bool_then.rs
828	56	tests/ui/manual_float_methods.rs
224	15	tests/ui/non_minimal_cfg.rs
912	61	tests/ui/ptr_cast_constness.rs
105	7	tests/ui/empty_enum_without_never_type.rs
1108	73	tests/ui/std_instead_of_core.rs
914	59	tests/ui/deref_addrof.rs
1198	76	tests/ui/literals.rs
205	13	tests/ui/floating_point_hypot.rs
206	13	tests/ui/needless_parens_on_range_literals.rs
206	13	tests/ui/suspicious_command_arg_space.rs
206	13	tests/ui/trailing_zeros.rs
206	13	tests/ui/zero_ptr.rs
209	13	tests/ui/needless_option_take.rs
561	34	tests/ui/find_map.rs
216	13	tests/ui/types.rs
1419	85	tests/ui/manual_assert.rs
875	51	tests/ui/empty_loop.rs
206	12	tests/ui/flat_map_option.rs
920	53	tests/ui/unneeded_wildcard_pattern.rs
838	48	tests/ui/arc_with_non_send_sync.rs
1258	71	tests/ui/impl.rs
911	51	tests/ui/reserve_after_initialization.rs
890	49	tests/ui/allow_attributes.rs
820	44	tests/ui/allow_attributes_without_reason.rs
205	11	tests/ui/track-diagnostics.rs
914	49	tests/ui/string_lit_chars_any.rs
909	48	tests/ui/mistyped_literal_suffix.rs
211	11	tests/ui/as_underscore.rs
826	43	tests/ui/unnecessary_fallible_conversions_unfixable.rs
511	26	tests/ui/manual_find.rs
914	45	tests/ui/manual_slice_size_calculation.rs
204	10	tests/ui/methods_fixable.rs
874	42	tests/ui/let_with_type_underscore.rs
105	5	tests/ui/missing_doc_crate_missing.rs
105	5	tests/ui/missing_inline_executable.rs
993	46	tests/ui/needless_arbitrary_self_type_unfixable.rs
1037	48	tests/ui/four_forward_slashes.rs
1263	56	tests/ui/ignored_unit_patterns.rs
204	9	tests/ui/unnecessary_self_imports.rs
205	9	tests/ui/to_digit_is_some.rs
1162	51	tests/ui/macro_use_imports_expect.rs
206	9	tests/ui/bytes_nth.rs
921	39	tests/ui/toplevel_ref_arg.rs
900	38	tests/ui/pub_with_shorthand.rs
809	34	tests/ui/unneeded_field_pattern.rs
906	38	tests/ui/pub_without_shorthand.rs
1135	47	tests/ui/patterns.rs
2344	92	tests/ui/format.rs
205	8	tests/ui/char_lit_as_u8_suggestions.rs
206	8	tests/ui/empty_enum.rs
207	8	tests/ui/to_string_in_format_args_incremental.rs
208	8	tests/ui/char_lit_as_u8.rs
864	33	tests/ui/needless_pub_self.rs
1261	48	tests/ui/macro_use_imports.rs
106	4	tests/ui/missing_doc_crate.rs
107	4	tests/ui/unknown_attribute.rs
1110	41	tests/ui/unseparated_prefix_literals.rs
824	29	tests/ui/string_add.rs
205	7	tests/ui/open_options_fixable.rs
205	7	tests/ui/path_buf_push_overwrite.rs
326	11	tests/ui/map_unit_fn.rs
208	7	tests/ui/option_map_or_err_ok.rs
812	27	tests/ui/toplevel_ref_arg_non_rustfix.rs
850	28	tests/ui/must_use_unit.rs
403	13	tests/ui/unwrap_or.rs
817	26	tests/ui/proc_macro.rs
205	6	tests/ui/four_forward_slashes_first_line.rs
911	26	tests/ui/redundant_at_rest_pattern.rs
809	23	tests/ui/as_conversions.rs
106	3	tests/ui/issue-3145.rs
951	24	tests/ui/iter_skip_zero.rs
205	5	tests/ui/obfuscated_if_else.rs
211	5	tests/ui/string_from_utf8_as_bytes.rs
887	21	tests/ui/unwrap.rs
828	17	tests/ui/option_env_unwrap.rs
210	4	tests/ui/author.rs
1010	19	tests/ui/range.rs
1137	21	tests/ui/expect.rs
813	13	tests/ui/deref_addrof_macro.rs
809	12	tests/ui/mem_replace_macro.rs
3431	35	tests/ui/large_const_arrays.rs
205	2	tests/ui/renamed_builtin_attr.rs
827	6	tests/ui/unnecessary_fallible_conversions.rs

I'll do some inspection in these lints (primarily seeing which function calls are repeated between them) and then... Fix them (?)
If you want to optimize some of these lints, you're free to do so! (we probably won't collide in the same lint hehe :3)

[y21]

I've been looking at single_call_fn specifically. It's doing another pass through the whole crate at the very end which doesn't seem necessary. We should be able to do what the visitor is doing at the same time as the lint (noticed it while working on #12183)

[PartiallyUntyped]

I'd like to look around filter_map_*s if that's okay.

[xFrednet]

@blyxyas Could you explain the table in a bit more detail? Your description says:

benchmarking each lint 100 times

but you list individual test files instead of lints. Do you mean that you ran Clippy 100x on each file and measured that? In that case, it's likely that the time doesn't just measure the tested lint but also other lints, which might be allowed or still got relatively far in their checks.

---

Maybe related sidenote: For our Metadata Collection Monster 🐉 we actually have a thing, that only registers that one lint pass and thereby suppresses all other lints:

rust-clippy/clippy_lints/src/lib.rs

Lines 595 to 600 in 8baeb26

	{
	if std::env::var("ENABLE_METADATA_COLLECTION").eq(&Ok("1".to_string())) {
	store.register_late_pass(|_| Box::new(utils::internal_lints::metadata_collector::MetadataCollector::new()));
	return;
	}
	}

Maybe such a mechanism could be used to actually measure the impact that a single lint has on Clippy's performance

[blyxyas]

Sorry for the delay, I totally forgot about this comment and was making a retrospect of the performance project when I stumbled on it again 😅

Yeah, I executed a script similar to this:

for file in tests:
    SPEEDTEST=ui cargo uitest | <text processing to get average time> > RESULTS$file

If there's a better, we have a server to get more accurate numbers!

[xFrednet]

Okay, I've update the title to reflect this a bit better :)

[blyxyas]

Closing this in favour of #14707 :)