Add ability to override block fields in debug_ calls#10405
Merged
Conversation
flcl42
force-pushed
the
block-overrides
branch
from
92ea277 to
bcd33d6
Compare
flcl42
requested review from
Demuirgos,
LukaszRozmej,
MarekM25,
benaadams and
smartprogrammer93
as code owners
Member
|
@flcl42 isn't the issue effect of EIP-7825 Transaction Gas Limit Cap? |
Contributor
Author
I doubt it's connected to that EIP, maybe the fix will uncover more, but seems like it's just block gas limits override related @avinashbo Could you test if it works OK for you or provide complete request so I can run it against a node I have locally, please? |
benaadams
approved these changes
Feb 10, 2026
LukaszRozmej
approved these changes
Feb 10, 2026
kamilchodola
added a commit
that referenced
this pull request
Feb 17, 2026
* Fixes 4 flaky test + refactors dbs (#10407) * Fix XDC flaky reward test with correct signature for transaction * Add Retry to LockFairnessTest * fix GetMemoryOwner for managed dbs + refactors * whitespace * Decrease Retain_Some_PersistedNodes threshold to resolve flakiness * fix review * tiny refactor * Parallelize Trie.Tests * Parallelize Nethermind.Blockchain.Test * Revert FilterManagerTests parall * Update Dockerfiles (#10409) Co-authored-by: rubo <rubo@users.noreply.github.com> * parallelize txpool test + fixes for parallel blockchain tests (#10418) * Fix XDC flaky reward test with correct signature for transaction * Add Retry to LockFairnessTest * fix GetMemoryOwner for managed dbs + refactors * whitespace * Decrease Retain_Some_PersistedNodes threshold to resolve flakiness * fix review * tiny refactor * Parallelize Trie.Tests * Parallelize Nethermind.Blockchain.Test * Revert FilterManagerTests parall * parallelize some TxPool tests * Try parallelize more tests * make TxPool tests more parallelizable * revert * fix * fix issues in Nethermind.Blockchain.Test * more fixes * more fixes * FilterManagerTests nonparallelizable * fix * fix flaky test * fix shutter test * Add retry to Fuzz_accounts_with_storage * retry flaky test * add [NonParallelizable] * XDC - Add custom state sync allocation strategy (#10399) add custom state sync allocation strategy for xdc * ProcessingStats Extensibility (#10420) * processing stats extensibility * improvement * build fix * fix * fix: correct Bytes.BytesComparer length comparison ordering (#10353) * fix: correct Bytes.BytesComparer length comparison ordering The BytesComparer had inverted length comparison logic: - null was considered greater than non-null (should be less) - shorter arrays with same prefix were considered greater than longer arrays (should be less) Fixed by inverting the return values for length-related comparisons: - null < non-null - [] < [x, ...] (empty < non-empty) - [prefix] < [prefix, more] (shorter with same prefix < longer) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Switched to sequence compare to * Address comment * fix: handle null y in BytesComparer when x is non-null Null arrays implicitly convert to empty ReadOnlySpan<byte> in SequenceCompareTo, losing the null vs empty distinction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * Use CodeInfo type instead of ICodeInfo (#10423) * Use CodeInfo type instead of ICodeInfo Replace the ICodeInfo abstraction with a concrete CodeInfo type and adapt related APIs and implementations. CodeInfo was extended to carry precompile info, provide Code/CodeSpan, Version, IsPrecompile/IsEmpty semantics and background analysis. EofCodeInfo now derives from CodeInfo and provides EOF-specific data and versioning. PrecompileInfo was removed and precompiles are represented as CodeInfo (wrapping CachedPrecompile when needed). API changes: ICodeInfoRepository, IOverridableCodeInfoRepository and IPrecompileProvider signatures and caches now use CodeInfo; CodeInfoFactory and CodeLruCache updated accordingly. Call sites across the VM, instruction implementations, tracers, transaction processing, repositories and tests were updated to use CodeInfo directly and to perform EOF checks using 'is EofCodeInfo' where appropriate. JumpDestinationAnalyzer.MachineCode was made accessible for CodeInfo.Code. Overall this unifies EOF and precompile handling under a single CodeInfo model and simplifies caching and execution logic. * Feedback * Update CI workflows for Taiko/Surge integration tests (#10419) * Add CI workflow for Surge integration tests and update workflow for Taiko * Update CI workflow to reference the correct surge configuration file * Update CI workflows to increase timeout for integration tests and adjust repository references * Resolve comments * Parallelizable does not work on all XDC tests (#10431) remove broken Parallelizable Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * Add JitAsm tool to be able to analyse the Jit output (#10432) * Add JitAsm tool to be able to analyse the Jit output * Spelling * Address AGENTS.md LINQ guideline feedback on JitAsm PR (#10433) * Initial plan * Strengthen AGENTS.md LINQ guideline with explicit examples Co-authored-by: benaadams <1142958+benaadams@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: benaadams <1142958+benaadams@users.noreply.github.com> * Spell * feedback * Feedback * Improve initialization --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: benaadams <1142958+benaadams@users.noreply.github.com> * Update README with performance highlights of Nethermind (#10359) * Update README with performance highlights of Nethermind Revised description to emphasize performance metrics. * Update README.md Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * Update README.md Co-authored-by: Ruben Buniatyan <rubo@users.noreply.github.com> * Update README with image sources and badges * Add 'srcset' to cspell.json dictionary * Remove cspell directives from README Removed cspell directives from README.md. --------- Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Ruben Buniatyan <rubo@users.noreply.github.com> * Auto-update fast sync settings (#10449) Co-authored-by: rubo <rubo@users.noreply.github.com> * Update OP Superchain chains (#10448) Co-authored-by: emlautarom1 <emlautarom1@users.noreply.github.com> * Fix flaky tests: timing and race condition (#10455) - PeerManagerTests: Increase After timeout from 1000ms to 3000ms in Will_not_stop_trying_on_rlpx_connection_failure to prevent false failures on loaded CI runners - SyncServerTests: Use Interlocked.Increment in Broadcast_NewBlock_on_arrival_to_sqrt_of_peers to fix race condition where concurrent SyncPeerMock background threads could lose count increments - Rename _travisDelay fields to _delay, _delayLong, _delayLonger Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Metrics and BlockStatistics extension (#10429) * Metrics and BlockStatistics extension * extensibility fixes * AGENTS shouldn't be repetitive when adding code (#10460) * more Agents changes (#10461) * more Agents changes avoid var, use testcases * Revise testing guidelines and LINQ usage recommendations Updated guidelines for adding tests and using LINQ. * CodeInfo Extension (#10467) refactoring & codeinfo changes * fix: add missing yield break after shutdown response in `JsonRpcProcessor` (#10462) * fix: add missing yield break after shutdown response * Add test * ensure pipereader is completed on shutdown early-exit * cover pipereader completion when shutdown is requested --------- Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * Fix fuzz issue (#10459) * Fix flaky network test (#10463) * Update packages (#10421) * Remove redundant allocation in Rlp.Encode<T> when input is already Rlp (#10468) * Update Rlp.cs * Add regression test for Rlp.Encode<T> with Rlp input --------- Co-authored-by: Ben {chmark} Adams <thundercat@illyriad.co.uk> * Add SeqlockCache (#10415) * Add SeqlockCache and use for Accounts and Values * Add tests * Spelling * By in * Feedback * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix * Feedback * Change to 2 way cache * 2-way skew-associative * Add prefetch for way 1 and hash-bit alternating eviction Prefetch way 1's cache line while checking way 0 to hide L2/L3 latency in the skew layout where ways are ~1MB apart. Use a hash bit to alternate eviction when both ways are live instead of always evicting way 0 — measured identical hit rates with zero extra memory. * Add Volatile.Read/Write for _hashes in BlockhashProvider Ensures cross-thread visibility of the prefetched hashes array on ARM's weak memory model. On x86 (TSO) these are no-ops. * Convert NodeStorageCache and PreBlockCaches.RlpCache to SeqlockCache * Inline Address comparison in StorageCell.Equals for SeqlockCache hot path Seal Address class and inline 20-byte comparison (Vector128 + uint) directly in StorageCell.Equals to eliminate Address.Equals calls that the JIT refuses to inline in deep chains. Adds ReferenceEquals fast-path. * Outline cold paths in SeqlockCache and fix IsPersisted delegate allocation Extract GetOrAddMiss as NoInlining to keep GetOrAdd hit path lean (2122 -> 748 bytes). Change WriteEntry to NoInlining and collapse eviction tree to single call site. Reuse cached _tryLoadRlp delegate in PreCachedTrieStore.IsPersisted. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add ability to override block fields in debug_ calls (#10405) * Align tx sending error code (#10464) * Optimize transaction processing via encoded-tx trie root and fast block re-encode (#10435) * Add encoded-tx root calc and trie decode perf Allow computing the transactions trie root directly from RLP-encoded transactions and update callers to use it. ExecutionPayload.TryGetBlock now passes the encoded Transactions to TxTrie.CalculateRoot. Implement TxTrie.CalculateRoot(ReadOnlySpan<byte[]>) and InitializeFromEncodedTransactions to populate the trie from encoded payloads and compute RootHash; use a TrackingCappedArrayPool and UpdateRootHash. Add a unit test to assert encoded and decoded transaction paths produce the same root. Also refine trie node RLP encoding parallelization: UseParallel now checks the node's non-null child count (only parallelize when >= 4 children and multiple CPU cores) to avoid parallel overhead on small branches. * Fast block re-encode * Feedback * chore: Update .NET packages (#10482) Update .NET packages * Move tx block-format wrapping logic from BlockDecoder to TxDecoder (#10476) * Move tx block-format wrapping logic from BlockDecoder to TxDecoder Extract the legacy-vs-typed tx wrapping knowledge into static helpers on TxDecoder (GetBlockFormatLength, WriteBlockFormat) so BlockDecoder no longer directly checks TxType.Legacy for encoding decisions. * Feedback * Update Dockerfiles (#10485) Co-authored-by: rubo <rubo@users.noreply.github.com> * fix: clear reference-type elements in Truncate to prevent pool memory leak (#10472) * fix: clear reference-type elements in Truncate to prevent pool memory leak * add helper --------- Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * Remove Vault and NDM remnants (#5462) (#10478) - Remove Vault from ModuleType and spaceneth configs - Remove NDM from Protocol, ProtocolParser, and tests - Remove unused NDM timeout constants from Timeouts - Update AddCapabilityMessageSerializer comment Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * Faster FastHash (#10410) * Faster FastHash * Feedback * Split FastHash into dispatcher + dedicated AES/CRC methods Split monolithic FastHash into thin dispatcher + FastHashAesX64, FastHashAesArm, and FastHashCrc. Eliminates XMM saves from CRC path, removes ARM ternaries from x64 codegen, replaces MEMCPY tail with CRC scalar processing, and simplifies 4-lane fold to 3 XOR + 1 AES. AesX64 codegen: 294 bytes (was 564), CRC: no XMM registers at all. AES/CRC ratio: 1.75x at 256B, 2x at 1024B. * Optimize JSON-RPC request parsing and processing (#10453) * Add encoded-tx root calc and trie decode perf Allow computing the transactions trie root directly from RLP-encoded transactions and update callers to use it. ExecutionPayload.TryGetBlock now passes the encoded Transactions to TxTrie.CalculateRoot. Implement TxTrie.CalculateRoot(ReadOnlySpan<byte[]>) and InitializeFromEncodedTransactions to populate the trie from encoded payloads and compute RootHash; use a TrackingCappedArrayPool and UpdateRootHash. Add a unit test to assert encoded and decoded transaction paths produce the same root. Also refine trie node RLP encoding parallelization: UseParallel now checks the node's non-null child count (only parallelize when >= 4 children and multiple CPU cores) to avoid parallel overhead on small branches. * Fast block re-encode * Feedback * Cache TokenValidationParameters as readonly field Move TokenValidationParameters construction from per-request AuthenticateCore to the constructor, eliminating repeated allocation on cache-miss auth path. * Implement manual HS256 JWT fast validator with library fallback Add zero-allocation manual JWT validation for known HS256 header formats. Uses HMACSHA256.HashData (static) and CryptographicOperations.FixedTimeEquals for signature verification. Handles iat and exp claims. Falls back to Microsoft.IdentityModel for unrecognized header formats. * Refactor ByteArrayConverter write path Replace WriteRawValue with WriteStringValue(ReadOnlySpan<byte>) on the main value-write path, eliminating manual quote handling. Raise stackalloc threshold to 256 bytes. Use "0x0"u8 literal for zero-value fast path. Delegate-based overload retained for property name writes. * Add fixed-size fast path for 32-byte hash converters Hash256Converter and ValueHash256Converter now use a dedicated 66-byte stackalloc path (0x + 64 hex chars) that skips CountLeadingNibbleZeros, ArrayPool, and the generic ByteArrayConverter write path. * Direct numeric-to-hex for Long, ULong, UInt256 converters Use BitOperations.LeadingZeroCount and nibble-extraction loop to write hex directly without byte array intermediary. UInt256 uses LZCNT on limbs. All paths use WriteStringValue instead of WriteRawValue. Zero values use "0x0"u8 literal. ZeroPaddedHex uses fixed 66-byte stackalloc path. * Replace ForcedNumberConversion AsyncLocal with ThreadStatic cache GetFinalConversion() now reads from a ThreadStatic cache instead of AsyncLocal on every call. The ThreadAwareAsyncLocal wrapper updates both the AsyncLocal and ThreadStatic when Value is set, maintaining backward compatibility with tracing/debug converters. * Pool CancellationTokenSource with TryReset BuildTimeoutCancellationToken now rents from a ConcurrentBag pool. TryReset() returns CTS to pool on completion. Debugger-attached path skips pooling. Eliminates per-request CTS allocation on hot path. * Increase StreamPipeWriter minimumBufferSize from 4KB to 16KB Reduces Grow calls during serialization of typical Engine API responses (~5-20KB). Tradeoff: slightly higher per-connection memory baseline. * Bypass BufferResponses for Engine API + call StartAsync before body Authenticated single responses (Engine API) skip RecyclableStream double-copy. Call StartAsync() before serialization to flush headers early and avoid PinnedBlockMemoryPool.Rent allocations. * Replace JsonRpcContext.Current AsyncLocal with ThreadStatic Replace AsyncLocal<JsonRpcContext?> with [ThreadStatic] field and a ThreadStaticAccessor wrapper that preserves the .Value API shape. This eliminates the ExecutionContext capture cost on every JSON-RPC request. * Use synchronous JsonSerializer.Serialize for PipeWriter path Replace JsonSerializer.SerializeAsync(PipeWriter, ...) with synchronous Utf8JsonWriter + JsonSerializer.Serialize. This eliminates the async state machine allocation since PipeWriter implements IBufferWriter<byte> and data is flushed by the caller's CompleteAsync(). * Eliminate CTS for Engine API path Skip per-request CancellationTokenSource allocation for authenticated Engine API requests. They use CancellationToken.None since they are from trusted consensus clients and must complete for consensus. Connection drops are handled by the PipeReader. Non-engine paths still use the pooled CTS from T08. * Engine API terminal middleware before routing/CORS Add an early-pipeline middleware that intercepts authenticated engine port POST requests before routing, CORS, response compression, and WebSocket middleware. Engine API requests are handled directly, writing to the response body without buffering. Non-engine requests pass through to the standard middleware pipeline. * Skip CountingPipeReader when Content-Length is known In the engine API fast lane, use ctx.Request.BodyReader directly and Content-Length for metrics, eliminating CountingPipeReader overhead. In the standard handler, skip CountingPipeReader when Content-Length is available and fall back to it only for chunked requests. * Seal hot classes and capture concrete types Seal EthereumJsonSerializer, JsonRpcProcessor, and JsonRpcService (no subclasses exist). In Startup.cs, cast DI-resolved interfaces to their concrete types and use those in middleware closures, enabling JIT devirtualization of hot-path method calls. * Micro-optimizations in request processing - Replace LINQ Select in batch deserialization with explicit loop - Use TryGetDecimal instead of GetRawText for numeric JSON-RPC id parsing - Replace ElementAtOrDefault with bounds-checked indexed access in LogRequest - Intern known engine method names via ValueEquals to avoid string allocation - Remove unused System.Linq using from JsonRpcProcessor * Startup warmup for serializer metadata Add EthereumJsonSerializer.WarmupSerializer() that pre-serializes instances to populate System.Text.Json metadata caches. Call it at startup with JsonRpcSuccessResponse and JsonRpcErrorResponse to eliminate cold-start serialization overhead on first engine request. * Move log interpolation to NoInlining local/static functions Keep string interpolation out of hot paths by moving log bodies to [MethodImpl(MethodImplOptions.NoInlining)] functions so the JIT does not inline the interpolation into callers. * STJ source generation for engine, eth, debug, and trace API types Add source-generated JsonSerializerContext instances for all major RPC type families to eliminate reflection-based metadata lookup on hot paths. - EngineApiJsonContext: 19 engine API types (payloads, forkchoice, blobs) - FacadeJsonContext: BlockForRpc, TransactionForRpc, FilterLog, SyncingResult - EthRpcJsonContext: receipts, fee history, account proofs, plus debug/trace types (GethLikeTxTrace, ParityTxTraceFromStore, ChainLevelForRpc, etc.) - JsonRpcResponseJsonContext: success/error response envelope types Infrastructure changes: - AddTypeInfoResolver() on EthereumJsonSerializer with version-tracked propagation to all existing serializer instances - Cache JsonTypeInfo on ExpectedParameter for typed deserialization - WriteJsonRpcResponse() in Startup.cs for typed response serialization - Add [JsonIgnore] to Span properties on Hash256, Bloom, Signature to prevent SYSLIB1225 source generator errors * Optimize JwtAuthentication * Optimize ByteArrayConverter * Test fix * Spelling * Optimize LongConverter * Spell * Optimize Hash256Converter * Optimize UInt256Converter * Fix tests * Spelling * Feedback * Optimize * Optimize * Spell * Also Avx512 * Formatting * Spell * Drop the weird pattern matching * Use concurrentqueue instead * Stream blobs directly Before (3 copies of hex data): 1. ByteArrayConverter → hex-encode into rented ArrayPool<byte> buffer (262KB) 2. Utf8JsonWriter.WriteRawValue → memcpy into Utf8JsonWriter's internal buffer (262KB) 3. Utf8JsonWriter flush → memcpy into PipeWriter/Kestrel send buffer (262KB) After (1 copy): 1. OutputBytesToByteHex → hex-encode directly into writer.GetSpan() (Kestrel's send buffer) So copies 2 and 3 are eliminated. Copy 1 (the binary→hex transform) remains * formatting * Spell * Tidy up * tidy up * Feeedback * Feedback * Reduce contention in trie root hashing Replace lock+List in TrackingCappedArrayPool with ConcurrentQueue for parallel paths and bare List for sequential paths. Skip parallel root hashing for small tries (<=64 items) to avoid scheduling overhead. Fix race condition where ReceiptTrie and TxTrie called UpdateRootHash without propagating canBeParallel, causing concurrent List.Add on a non-thread-safe collection. * Feedback * Run newPayload inline (#10479) * Skip response compression for Engine API requests * Optimize engine_getBlobsV2 with fused batch lookup and zero-copy proofs Replace N+1 lock acquisitions with a single fused TryGetBlobsAndProofsV1 that atomically counts and extracts blobs under one lock. Use ReadOnlyMemory<byte[]> to window into wrapper.Proofs arrays instead of copying via Slice+spread, eliminating ~0.4MB of proof allocations per request. Replace ArrayPoolList with parallel arrays, removing the pool rent/return overhead. * Split blob lookups into two phases to reduce lock hold time (#10173) Add ITxStorage.TryGetMany for batched RocksDB MultiGet and override TryGetBlobsAndProofsV1 in PersistentBlobTxDistinctSortedPool with a two-phase approach: fast in-memory + cache lookups under lock, then a single batched DB read outside the lock. This avoids holding the pool's McsLock during potentially slow I/O for up to 128 blobs per request. Co-Authored-By: Lukasz Rozmej <lukasz@nethermind.io> * Remove dead GetBlobCounts method (#10159) No longer called after getBlobsV2 switched to batched TryGetBlobsAndProofsV1. Co-Authored-By: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> * Feedback * Spelling * Add some const to JwtAuthentication to understand the code better * Consolidate duplicated JSON-RPC request processing pipelines in Startup Extract shared ProcessJsonRpcRequestCoreAsync and PushErrorResponseAsync instance methods from the engine API fast lane and standard handler, eliminating ~100 lines of duplicated processing logic. Unify status code constants, auth error handling, and add streamable response support to the standard path. * Make BlobsV2DirectResponse enumerator explicit; pool byte[64] keys in BlobTxStorage - Convert GetEnumerator to explicit interface implementation since it is only used by tests via IEnumerable<T> cast - Add ConcurrentQueue-based pool for exact-size byte[64] DB lookup keys in TryGetMany to avoid per-call allocations --------- Co-authored-by: Lukasz Rozmej <lukasz@nethermind.io> Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Fix BackgroundTaskScheduler queue overflow during block processing (#10488) * Initial plan * Fix BackgroundTaskScheduler queue overflow by removing signal wait from scheduler threads During block processing, ManualResetEventSlim blocked all scheduler threads in BelowNormalPriorityTaskScheduler, preventing StartChannel from draining expired tasks. New tasks (P2P transaction messages) continued arriving, filling the queue beyond capacity (1024) and triggering task drops. Fix: Remove the ManualResetEventSlim signal entirely. The existing cancellation token mechanism in StartChannel already handles block processing correctly — expired tasks get drained with cancelled tokens (fast return), while non-expired tasks are re-queued with a 1ms throttle until their deadline passes or block processing ends. Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> * Replace blocking ManualResetEventSlim with async TaskCompletionSource signal; add high-capacity stress test The original ManualResetEventSlim blocked scheduler threads in ProcessBackgroundTasks(), preventing StartChannel from draining expired tasks during block processing. Replace with TaskCompletionSource-based async signal awaited in StartChannel's Throttle path. Add comprehensive stress test that fills a 1024-capacity queue across multiple block processing cycles, verifying: - Tasks are dropped when queue exceeds capacity - Expired tasks drain with cancelled tokens during block processing - Queue recovers and accepts new tasks after draining - Mixed short/long-lived tasks behave correctly across cycles - Queue remains fully operational after repeated block processing Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> * Double BackgroundTaskMaxNumber default from 1024 to 2048 Co-authored-by: benaadams <1142958+benaadams@users.noreply.github.com> * Add BenchmarkDotNet benchmark for BackgroundTaskScheduler throughput Adds BackgroundTaskSchedulerBenchmarks with two scenarios: - ScheduleAndDrainDuringBlockProcessing: simulates real-world scenario with periodic block-processing pauses (5 cycles, 50ms each) - ScheduleAndDrainWithoutBlockProcessing: baseline without interruptions Parameters: capacity (1024/2048), concurrency (2), block processing duration (50ms), cycles (5). Also fixes flaky over-capacity assertion in stress test that raced with the async task draining. Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> Co-authored-by: benaadams <1142958+benaadams@users.noreply.github.com> * Update Dockerfiles (#10493) Co-authored-by: rubo <rubo@users.noreply.github.com> * fix: unsubscribe from BlocksProcessing in Dispose (#10499) Signed-off-by: Olexandr88 <radole1203@gmail.com> * Fix Kute Windows performance: disable proxy detection, fix file check (#10491) * Fix Kute Windows performance: disable proxy detection, fix file check - Disable WPAD proxy auto-detection in HttpClient by using SocketsHttpHandler with UseProxy=false. On Windows, the default HttpClient triggers a WPAD lookup (~2s timeout) on every fresh process, inflating measured NP times by ~2000ms. - Fix FileMessageProvider to use pathInfo.Exists instead of HasFlag(FileAttributes.Normal). On Windows, files have Archive attribute, not Normal, causing all file reads to fail. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update comment to remove 'WPAD' reference --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * chore: Remove unused code across multiple modules (#10440) * claude experiments * reverted changes * Remove unused code * fix: restore _logger field in TdxsClient to prevent CS9113 build error The logManager parameter is DI-injected and cannot be removed, so the field must remain to consume it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unused EVM exception classes BadInstructionException, InvalidCodeException, and TransactionCollisionException are never instantiated anywhere in the codebase. The EVM uses EvmExceptionType enums and CallResult directly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: remove unused code (Phase 1) - Delete 5 empty (0-byte) files in TxPool and Consensus - Remove 7 unused private methods: - TrieStore.IsStillNeeded() - PatriciaTree.ThrowInvalidDataException/ThrowMissingChildException/ThrowMissingPrefixException - EthereumRunner.Stop() - EthereumL1Bridge.LogReorg() - Ssz.DecodeDynamicOffset() (moved VarOffsetSize to BasicTypes.cs) - Remove 7 unused private/static fields: - TrieNode._nodeDecoder - PatriciaTree._singleByteKeys - PersistentStorageProvider._loadFromTreeStorageFunc - SmallTrieVisitContext._branchChildIndex - TdxsClient._logger - InitializeBlockchain._logManager - SyncProgressResolver._logger - Remove 6 dead NDM/LES timeout constants from Timeouts.cs - Remove 11 unused/duplicate ErrorCodes constants - Remove 3 unused L1 epoch constants from EthereumL1Bridge - Clean up unused using directives (System.IO, System) * fix: remove unused logManager parameter from TdxsClient Removes unused ILogManager parameter that was causing CI build failure. The parameter was not used anywhere in the class after previous cleanup. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * refactor: remove unused types (Phase 2) Delete 15 unreferenced type files identified through codebase-wide static analysis: Synchronization (5): - AllocationChangeEventArgs - StaticStrategy, NullStrategy, ClientTypeStrategy - StrategySelectionType enum Core (5): - CompositePublisher, SortedRealList, ConcurrentWriteBatcher - CappedArrayMemoryManager, ISpanCache Serialization/Network/RPC (5): - LogPublisher, CountingTextWriter, CountingTextReader - IDiscoveryMsgSerializersProvider, IJsonRpcResult Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unused logManager parameter from SyncProgressResolver constructor (#10452) * Initial plan * refactor: remove unused logManager parameter from SyncProgressResolver Co-authored-by: MarekM25 <9356351+MarekM25@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MarekM25 <9356351+MarekM25@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: smartprogrammer93 <smartprogrammer93@users.noreply.github.com> Co-authored-by: smartprogrammer93 <smartprogrammer@windowslive.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: MarekM25 <9356351+MarekM25@users.noreply.github.com> * fix(network): use TrySetResult in MessageQueue.Handle to prevent race condition (#10486) * fix(network): use TrySetResult in MessageQueue.Handle to prevent race condition * add tests * Fixes --------- Co-authored-by: Ben Adams <thundercat@illyriad.co.uk> * fix: fix operator precedence bug in VoteDecoder.GetContentLength (#10383) Update VoteDecoder.cs Co-authored-by: ak88 <anders@nethermind.io> * ci: add automatic PR labeling workflow (#10512) * ci: add PR auto-labeler workflow Automatically labels PRs based on: - Template checkbox selections (bugfix, feature, breaking, etc.) - Conventional commit title prefixes (fix:, feat:, perf:, etc.) - Changed file paths (evm, network, optimism, taiko, xdc, etc.) - EIP mentions in title - Dockerfile changes, chain configs, test-only PRs, removal-only PRs * docs: require PR template usage in AGENTS.md Clarify that PRs must follow the template format and tick the type-of-change checkboxes, which drive automatic labeling. * ci: use pull_request_target for PR labeler to support fork PRs * ci: add DbModule.cs path rule for database label * ci: add trie and state+storage path rules for PR labeler * ci: match slash separator in title prefix (e.g. Perf/xdc) * ci: add sync, snap sync path rules for PR labeler * Remove redundant null check and duplicate array in Program.cs (#10506) Update Program.cs Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * Use copy-on-write snapshots in MetricsController (#10501) * fix(monitoring): use copy-on-write snapshots in MetricsController to prevent concurrent modification Replace List<Action> and Dictionary iteration with volatile array snapshots to prevent InvalidOperationException when RegisterMetrics or AddMetricsUpdateAction races with the timer-driven UpdateAllMetrics. * fix(monitoring): use copy-on-write snapshots in MetricsController to prevent concurrent modification Replace List<Action> and Dictionary iteration with volatile array snapshots to prevent InvalidOperationException when RegisterMetrics or AddMetricsUpdateAction races with the timer-driven UpdateAllMetrics. * chore: remove redundant AddDatabase for BlobTransactions (#10510) Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * perf: reduce block processing overhead via journal bypass and commit elimination - Eliminate pre-execution Commit in TransactionProcessor for non-tracing, non-restore paths - Add read-cache separation for StateProvider (bypass journal for pure reads) - Add read-cache separation for PersistentStorageProvider (bypass journal for SLOAD) - Reduce block-level commits from 4 to 3 (beacon root/blockhash accumulate with tx changes) - Preserve full tracing correctness with fallback paths for Before values and storage reads Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * SyncInfo decoder for XDC p2p (#10509) syncinfo decoder * Revert "Merge branch 'master' of https://github.com/NethermindEth/nethermind into perf/block-processing-pipeline-v1" This reverts commit 1d7dede, reversing changes made to 46ae4aa. * ci: add JsonRpc path pattern to PR labeler (#10526) ci: add JsonRpc path pattern to PR labeler for automatic rpc label Adds src/Nethermind/Nethermind.JsonRpc to the pathToLabel array so PRs modifying JsonRpc files get automatically labeled with 'rpc'. Co-authored-by: smartprogrammer93 <smartprogrammer93@users.noreply.github.com> * Log index (#8464) * Adjust `OnReceiptsInserted` logging * Naming * Normalize block number formatting # Conflicts: # src/Nethermind/Nethermind.Facade/Find/LogIndexService.cs * Build fix * Code cleanup * Use ancient receipts barrier for backwards sync target & include target values in RPC * Adjust logging and waiting times * Fresh start fix * Adjust min target block calculation * Fix repeated completion logging * Temporary more logging again * Try recover receipts * Optimize processing of no-tx blocks * Fir out-of-range error * Fix receipts deletion (#9231) * separate receipt deletion from receipt tx deletion * regression test * tidy test --------- Co-authored-by: Marc Harvey-Hill <10379486+Marchhill@users.noreply.github.com> * `logIndex_blockNumbers` RPC * Logging tweaks * [WIP] Support for topic index * Remove key size dependencies * Try compress key in both DBs * Separate merge operator per column family * Fixes * Remove possible collisions as causing invalid merge order * More fixes * Constant db key postfix size & fixes * Screw this, use separate DB per topic index * Get rid of generic iterator interface * Fixes for seeking, but getting block numbers is now much slower * Code cleanup * Do not remove leading zeroes & revert seeking * Fix filter matching * Tests fixes * Move filter tests to separate class & code formatting * Support for concurrent forward/backward sync & tests * Fixes for concurrent sync & more aggresive test * Code cleanup * Fix log index syncing blocking other tasks * Refactor syncing for better concurrency * Increase receipts cache size to 1024 * Log receipts loading time * Adjust parallelism values * Track backward/forward syncing stats separately * Revert "Increase receipts cache size to 1024" This reverts commit 18a13d5. * Set DB version * Option to delay compressor start * Include index version and reset data if version is invalid * Log index compaction RPC * todo * Add missing service registration * Remove "waiting for block" trace logging * Do start indexing before DB initialization * Code cleanup * Fix double DB initialization * Fix aggressive memory usage * Stop on error * todos * Formatting * Configuration for log-index & support for disabling/resetting index via config * Updates to logIndex_status, include status, last-update-data, and last-error * Replace last-update with `DateTimeOffset` * Typo fix * Syncing service small refactoring, fixes, and basic tests * Fix repeated completion logging * Adjust default `SyncFetchBatchParallelism` * TurboPFor updates to match package version * Build fix * Configurable compression algorithm, handle if unsupported * Support for `IWriteBatch` clearing * Remove `RequireCommitWriteBatch` * Throw if stopped * Simplify `GetDbSize` * Prevent DB corruption in case of mid-sync error & tests * Remove stopping on error for now * Updated compression-algo-change tests * Formatting * Small logging fix * ~Temporary log completion time * Configurable logging details * Fix for invalid ranges concatenation * Revert "~Temporary log completion time" This reverts commit f3f631f. * Additional check to prevent using block 0 as pivot * Flag to verify eth_getLogs responses from index * Build fix * Switch to nuget package for TurboPFor * Parallelize `LogIndexStorageFilterTests` * Formatting * Try fix `LogIndexServiceTests` * TurboPFor package update * Fix immediate backward sync completion * Tests for `LogFinder` index range calculation * Min number of blocks to use index * PR cleanup * Better handle write attempt during stopping * Temporary fix for missing Nuget dependency * PR cleanup * Revert attempt to use `OneTimeTearDown` * Optimize compressor memory usage * Fail log index on background job error * Dependency fix * Take in-progress into account when waiting for compression * Use array pool for compression * Adjust background exception handling * Adjust exception handling * Move registrations * Take sync direction from aggregation * Remove first-block-added notification * Remove testing methods * Remove unused code & PR cleanup * Formatting * PR cleanup * Namespace update * PR cleanup * Builder code cleanup * Config update * Remove RPC response verification * PR cleanup * Tests fix * Disable index by default * Tests fix * Fix compaction being disabled * Increase compaction logging level * Make compression optional * Remove completed TODO * Logging updates * Return RPC response verification flag with proper checks * Build fix * PR cleanup * Naming update to match existing signatures * Partial test run fix * Logging tweak * PR cleanup * PR cleanup * Move Merge to separate interface * PR feedback * Storage stability improvements & enable `OneTimeTearDown` in integration tests * Receipts events renaming * PR feedback * Formatting * Code cleanup * Code cleanup * Try make storage disposing thread-safe * Do not throw from merge operator * Disable `OneTimeTearDown` again * Fix missing DB config * Fix disposing in case of error in ctor * Use sorted view instead of iterator * Do not publicly expose iterator * Code cleanup * Revert changes to DB config reading * PR feedback * PR feedback * PR feedback * Build fix * Cherry-pick updates * Commit all columns via single batch, store metadata in separate table * Fix saving redundant data to range bound * DB config fix * Code formatting * Tests fix * [WIP] block number enumerator (nested) * [WIP] Fixing enumerator * [WIP] Fixing enumerator * Fixes, code cleanup & use array pool for enumerator value * Code cleanup * Visitor tests cleanup * Formatting * Simplify filter/expression updates * Code cleanup * Build fix * Tests fix * Code cleanup * Remove `AscListHelper` as not needed * Inline `UnionEnumerator`, v1 * Revert "Inline `UnionEnumerator`, v1" This reverts commit 5b72543. * More merge operator tests * Code cleanup * Update TurboPFor package * Simplify `LogFinder` * Build fix attempt * Make `LogIndexFilterVisitor` enumerators internal * Remove unused parameter * Code cleanup * Formatting * Spelling * Try make cspell happy * Try make cspell happy * Try make cspell happy * Try make cspell happy * PR fixes * Test build fix * PR fixes * Move log-index features to inherited `IndexedLogFinder` * Simplify `filter_all_logs_iteratively` test * Formatting * Simplify `StartLogIndex` step * Block numbers copying optimization (little-endian only) * Test fix * Detailed in-code docs * Doc tweaks * More doc tweaks * PR cleanup * Make `GetBlockReceipts` a bit more explicit * Build fix * Spelling * minor refactors * more minor refactors * minor refactor of Compressor - add alternative lookup to avoid allocation as long as possible in TryEnqueue * simplifications in LogIndexStorage * simplify semaphores * move to stackalloc in LogIndexStorage where possible * simplify merge operators to inline array * correctly dispose ArrayPoolList in Merge tests * more simple refactors * In LogIndexBuilder replace Dictionaries iwth Direction * Simplify UnionEnumerator * Removed linq from IndexedLogFinder * Code cleanup * Simplify DBs initialization * Code cleanup * PR feedback * Code cleanup * Remove `storage` parameter from `LogIndexEnumerator` * Add complex LogIndex integration tests (#10425) * Add complex LogIndex integration tests for reorgs, concurrency, and error propagation - Add Concurrent_ReorgAndBackwardSync_Get_Test: validates concurrent reorg and backward sync using different semaphores without deadlocks or data corruption - Add Set_ReorgOutOfOrder_Get_Test: documents that descending-order reorgs are unsupported (MergeOperator only applies first Reorg operand per key by write order) - Add Set_BackgroundJobFailure_SubsequentOps_Test: verifies MergeOperator errors propagate via OnBackgroundError and permanently fail all subsequent operations - Add BackgroundFailingLogIndexStorage test subclass that injects corrupt merge operands to trigger LogIndexStateException in MergeOperator - Remove resolved TODO comments Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * minor test refactors * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix * Code cleanup * More code cleanup --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alex Bespalov <alexb5dh@gmail.com> * Simplify Compactor concurrency with Channel<T> (#10424) * Simplify Compactor concurrency with Channel<T> Replace AutoResetEvent/ManualResetEvent synchronization with a bounded Channel<TaskCompletionSource?>, reducing complexity and removing the WaitOneAsync extension dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix error propagation in Compactor catch block Use TrySetException(ex) for real exceptions so ForceAsync() callers observe the actual error. Keep TrySetCanceled() only for OperationCanceledException. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Address PR review comments - Coalesce concurrent ForceAsync() calls into a single compaction via a shared TaskCompletionSource, preserving the old behavior. - Use TrySetException instead of TrySetCanceled for non-cancellation errors so callers observe the actual exception. - Add debug log for compaction loop cancellation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Make ForceAsync exclusive * whitespace * Add Compactor.Dispose * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adjust ordering * Add comments and prepare `Compactor` for tests * Some `Compactor` tests * Spelling * Spelling dictionary update --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alex Bespalov <alexb5dh@gmail.com> * [WIP] use `PruningConfig.PruningBoundary` as default reorg depth * Use `PruningConfig.PruningBoundary` as default reorg depth * Take `PruningBoundary` before it's overwritten via `SnapServingMaxDepth` * Revert test change * Flaky test fix --------- Co-authored-by: Marc <Marchhill@users.noreply.github.com> Co-authored-by: Marc Harvey-Hill <10379486+Marchhill@users.noreply.github.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Auto-update fast sync settings (#10535) Co-authored-by: rubo <rubo@users.noreply.github.com> * Update OP Superchain chains (#10536) Co-authored-by: emlautarom1 <emlautarom1@users.noreply.github.com> * fix: fix malformed RLP encoding in L1OriginDecoder (#10525) * Update L1OriginDecoder.cs * Update L1OriginStoreTests.cs * Update SurgeGasPriceOracle for Surge Shasta (#10290) * SurgeGasPriceOracle updates for Shasta * minor fixes * more minor fixes * Resolve comments --------- Co-authored-by: Nurbakyt Madibek <jmadibekov@gmail.com> * Estimate Gas fix (#10559) * Estimate Gas fix * format * format * test fix * Update src/Nethermind/Nethermind.Blockchain.Test/TransactionProcessorTests.cs * Update src/Nethermind/Nethermind.Evm.Test/Tracing/GasEstimationTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix shared method --------- Co-authored-by: Ahmad Bitar <33181301+smartprogrammer93@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * test(evm): add extcode cache regressions for in-block code changes --------- Signed-off-by: Olexandr88 <radole1203@gmail.com> Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> Co-authored-by: core-repository-dispatch-app[bot] <173070810+core-repository-dispatch-app[bot]@users.noreply.github.com> Co-authored-by: rubo <rubo@users.noreply.github.com> Co-authored-by: Carmen Irene Cabrera Rodríguez <49727740+cicr99@users.noreply.github.com> Co-authored-by: Stavros Vlachakis <89769224+svlachakis@users.noreply.github.com> Co-authored-by: Amirul Ashraf <asdacap@gmail.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Ben {chmark} Adams <thundercat@illyriad.co.uk> Co-authored-by: Nurbakyt Madibek <jmadibekov@gmail.com> Co-authored-by: ak88 <anders@nethermind.io> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: benaadams <1142958+benaadams@users.noreply.github.com> Co-authored-by: Marek Moraczyński <marekm2504@gmail.com> Co-authored-by: emlautarom1 <emlautarom1@users.noreply.github.com> Co-authored-by: VolodymyrBg <aqdrgg19@gmail.com> Co-authored-by: splinter <futons_blare_7s@icloud.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alexey Osipov <me@flcl.me> Co-authored-by: phrwlk <phrwlk7@gmail.com> Co-authored-by: Nova ✰⋆⁺ <novaheidt@gmail.com> Co-authored-by: Lukasz Rozmej <lukasz@nethermind.io> Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: Olexandr88 <radole1203@gmail.com> Co-authored-by: smartprogrammer93 <smartprogrammer93@users.noreply.github.com> Co-authored-by: smartprogrammer93 <smartprogrammer@windowslive.com> Co-authored-by: MarekM25 <9356351+MarekM25@users.noreply.github.com> Co-authored-by: andrewshab <152420261+andrewshab3@users.noreply.github.com> Co-authored-by: 0xFloki <signal-rooms-7n@icloud.com> Co-authored-by: Forostovec <ilonaforostovec22@gmail.com> Co-authored-by: Ahmad Bitar <33181301+smartprogrammer93@users.noreply.github.com> Co-authored-by: Alex <alexb5dh@gmail.com> Co-authored-by: Marc <Marchhill@users.noreply.github.com> Co-authored-by: Marc Harvey-Hill <10379486+Marchhill@users.noreply.github.com> Co-authored-by: bobtajson <152420524+bobtajson@users.noreply.github.com> Co-authored-by: Diptanshu Kakwani <dipkakwani@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10351
Changes
Types of changes
What types of changes does your code introduce?
Testing
Requires testing
If yes, did you write tests?