-
Notifications
You must be signed in to change notification settings - Fork 697
Chore/remove affirmation map from chainscoordinator #6332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jferrant
wants to merge
24
commits into
stacks-network:develop
Choose a base branch
from
jferrant:chore/remove-affirmation-map-from-chainscoordinator
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
ba1ceac
Remove handle_affirmation_reorg, check_pox_anchor_affirmation
jferrant c695609
Remove check_chainstate_against_burnchain_affirmations
jferrant da251cf
Remove undo_stacks_block_orphaning
jferrant fac38a7
Remove affirmation_maps_active
jferrant 4c7abfc
Remove find_invalid_and_revalidated_sortitions, get_snapshots_and_aff…
jferrant 32aa1a6
Remove check for compatible_stacks_blocks in handle_new_epoch2_burnch…
jferrant 2d57bce
Remove find_highest_stacks_block_with_compatible_affirmation_map
jferrant 4f43f7f
Remove consolidate_affirmation_maps, *_get_*_affirmation_map, is_bloc…
jferrant d1fdb69
Fix failing test_inv_sync_start_reward_cycle
jferrant 36c427b
Remove AffirmationMaps from getinfo endpoint
jferrant ff7b3b4
Merge branch 'develop' of https://github.com/stacks-network/stacks-co…
jferrant 70b773a
Remove require_affirmed_anchor_blocks config options
jferrant 86d281c
Remove always_use_affirmation_maps config option
jferrant e40e0cf
Remove assume_present_anchor_blocks config option
jferrant b72a6ab
check_missing_anchor_block should stall if missing anchor block regar…
jferrant 6c36933
Fix test_pox_no_anchor_selected
jferrant ea15d55
Fix test_pox_no_anchor_selected
jferrant 33fbe18
Fix test_simple_setup
jferrant cd1b4cc
Remove affirmation_overrides config
jferrant b74cf10
Fix test_generate_markdown_with_real_fixture_data
jferrant e114345
Fix test_get_block_availability
jferrant acc9678
Merge branch 'develop' of https://github.com/stacks-network/stacks-co…
jferrant 53bf9ea
Remove TODO comment
jferrant 9cad386
Make test_nakamoto_inv_sync_across_epoch_change panic instead of timi…
jferrant File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,9 +13,8 @@ use stacks::chainstate::burn::db::sortdb::SortitionDB; | |
use stacks::chainstate::burn::{BlockSnapshot, ConsensusHash}; | ||
use stacks::chainstate::coordinator::comm::{CoordinatorChannels, CoordinatorReceivers}; | ||
use stacks::chainstate::coordinator::{ | ||
migrate_chainstate_dbs, static_get_canonical_affirmation_map, | ||
static_get_heaviest_affirmation_map, static_get_stacks_tip_affirmation_map, ChainsCoordinator, | ||
ChainsCoordinatorConfig, CoordinatorCommunication, Error as coord_error, | ||
migrate_chainstate_dbs, ChainsCoordinator, ChainsCoordinatorConfig, CoordinatorCommunication, | ||
Error as coord_error, | ||
}; | ||
use stacks::chainstate::stacks::db::{ChainStateBootData, StacksChainState}; | ||
use stacks::chainstate::stacks::miner::{signal_mining_blocked, signal_mining_ready, MinerStatus}; | ||
|
@@ -702,11 +701,6 @@ impl RunLoop { | |
let mut fee_estimator = moved_config.make_fee_estimator(); | ||
|
||
let coord_config = ChainsCoordinatorConfig { | ||
assume_present_anchor_blocks: moved_config.node.assume_present_anchor_blocks, | ||
always_use_affirmation_maps: moved_config.node.always_use_affirmation_maps, | ||
require_affirmed_anchor_blocks: moved_config | ||
.node | ||
.require_affirmed_anchor_blocks, | ||
txindex: moved_config.node.txindex, | ||
}; | ||
ChainsCoordinator::run( | ||
|
@@ -794,8 +788,6 @@ impl RunLoop { | |
fn drive_pox_reorg_stacks_block_processing( | ||
globals: &Globals, | ||
config: &Config, | ||
burnchain: &Burnchain, | ||
sortdb: &SortitionDB, | ||
last_stacks_pox_reorg_recover_time: &mut u128, | ||
) { | ||
let miner_config = config.get_miner_config(); | ||
|
@@ -812,92 +804,10 @@ impl RunLoop { | |
return; | ||
} | ||
|
||
// compare stacks and heaviest AMs | ||
let burnchain_db = burnchain | ||
.open_burnchain_db(false) | ||
.expect("FATAL: failed to open burnchain DB"); | ||
|
||
let sn = SortitionDB::get_canonical_burn_chain_tip(sortdb.conn()) | ||
.expect("FATAL: could not read sortition DB"); | ||
|
||
let indexer = make_bitcoin_indexer(config, Some(globals.should_keep_running.clone())); | ||
|
||
let heaviest_affirmation_map = match static_get_heaviest_affirmation_map( | ||
burnchain, | ||
&indexer, | ||
&burnchain_db, | ||
sortdb, | ||
&sn.sortition_id, | ||
) { | ||
Ok(am) => am, | ||
Err(e) => { | ||
warn!("Failed to find heaviest affirmation map: {e:?}"); | ||
return; | ||
} | ||
}; | ||
|
||
let highest_sn = SortitionDB::get_highest_known_burn_chain_tip(sortdb.conn()) | ||
.expect("FATAL: could not read sortition DB"); | ||
|
||
let canonical_burnchain_tip = burnchain_db | ||
.get_canonical_chain_tip() | ||
.expect("FATAL: could not read burnchain DB"); | ||
|
||
let sortition_tip_affirmation_map = | ||
match SortitionDB::find_sortition_tip_affirmation_map(sortdb, &sn.sortition_id) { | ||
Ok(am) => am, | ||
Err(e) => { | ||
warn!("Failed to find sortition affirmation map: {e:?}"); | ||
return; | ||
} | ||
}; | ||
|
||
let stacks_tip_affirmation_map = static_get_stacks_tip_affirmation_map( | ||
&burnchain_db, | ||
sortdb, | ||
&sn.sortition_id, | ||
&sn.canonical_stacks_tip_consensus_hash, | ||
&sn.canonical_stacks_tip_hash, | ||
) | ||
.expect("FATAL: could not query stacks DB"); | ||
|
||
if stacks_tip_affirmation_map.len() < heaviest_affirmation_map.len() | ||
|| stacks_tip_affirmation_map | ||
.find_divergence(&heaviest_affirmation_map) | ||
.is_some() | ||
{ | ||
// the sortition affirmation map might also be inconsistent, so we'll need to fix that | ||
// (i.e. the underlying sortitions) before we can fix the stacks fork | ||
if sortition_tip_affirmation_map.len() < heaviest_affirmation_map.len() | ||
|| sortition_tip_affirmation_map | ||
.find_divergence(&heaviest_affirmation_map) | ||
.is_some() | ||
{ | ||
debug!("Drive burn block processing: possible PoX reorg (sortition tip: {sortition_tip_affirmation_map}, heaviest: {heaviest_affirmation_map})"); | ||
globals.coord().announce_new_burn_block(); | ||
} else if highest_sn.block_height == sn.block_height | ||
&& sn.block_height == canonical_burnchain_tip.block_height | ||
{ | ||
// need to force an affirmation reorg because there will be no more burn block | ||
// announcements. | ||
debug!("Drive burn block processing: possible PoX reorg (sortition tip: {sortition_tip_affirmation_map}, heaviest: {heaviest_affirmation_map}, burn height {})", sn.block_height); | ||
globals.coord().announce_new_burn_block(); | ||
} | ||
|
||
debug!( | ||
"Drive stacks block processing: possible PoX reorg (stacks tip: {stacks_tip_affirmation_map}, heaviest: {heaviest_affirmation_map})" | ||
); | ||
globals.coord().announce_new_stacks_block(); | ||
} else { | ||
debug!( | ||
"Drive stacks block processing: no need (stacks tip: {stacks_tip_affirmation_map}, heaviest: {heaviest_affirmation_map})" | ||
); | ||
|
||
// announce a new stacks block to force the chains coordinator | ||
// to wake up anyways. this isn't free, so we have to make sure | ||
// the chain-liveness thread doesn't wake up too often | ||
globals.coord().announce_new_stacks_block(); | ||
} | ||
// announce a new stacks block to force the chains coordinator | ||
// to wake up anyways. this isn't free, so we have to make sure | ||
// the chain-liveness thread doesn't wake up too often | ||
globals.coord().announce_new_stacks_block(); | ||
|
||
*last_stacks_pox_reorg_recover_time = get_epoch_time_secs().into(); | ||
} | ||
|
@@ -913,7 +823,6 @@ impl RunLoop { | |
config: &Config, | ||
burnchain: &Burnchain, | ||
sortdb: &SortitionDB, | ||
chain_state_db: &StacksChainState, | ||
last_burn_pox_reorg_recover_time: &mut u128, | ||
last_announce_time: &mut u128, | ||
) { | ||
|
@@ -953,82 +862,6 @@ impl RunLoop { | |
return; | ||
} | ||
|
||
// NOTE: this could be lower than the highest_sn | ||
let sn = SortitionDB::get_canonical_burn_chain_tip(sortdb.conn()) | ||
.expect("FATAL: could not read sortition DB"); | ||
|
||
let sortition_tip_affirmation_map = | ||
match SortitionDB::find_sortition_tip_affirmation_map(sortdb, &sn.sortition_id) { | ||
Ok(am) => am, | ||
Err(e) => { | ||
warn!("Failed to find sortition affirmation map: {e:?}"); | ||
return; | ||
} | ||
}; | ||
|
||
let indexer = make_bitcoin_indexer(config, Some(globals.should_keep_running.clone())); | ||
|
||
let heaviest_affirmation_map = match static_get_heaviest_affirmation_map( | ||
burnchain, | ||
&indexer, | ||
&burnchain_db, | ||
sortdb, | ||
&sn.sortition_id, | ||
) { | ||
Ok(am) => am, | ||
Err(e) => { | ||
warn!("Failed to find heaviest affirmation map: {e:?}"); | ||
return; | ||
} | ||
}; | ||
|
||
let canonical_affirmation_map = match static_get_canonical_affirmation_map( | ||
burnchain, | ||
&indexer, | ||
&burnchain_db, | ||
sortdb, | ||
chain_state_db, | ||
&sn.sortition_id, | ||
) { | ||
Ok(am) => am, | ||
Err(e) => { | ||
warn!("Failed to find canonical affirmation map: {e:?}"); | ||
return; | ||
} | ||
}; | ||
|
||
if sortition_tip_affirmation_map.len() < heaviest_affirmation_map.len() | ||
|| sortition_tip_affirmation_map | ||
.find_divergence(&heaviest_affirmation_map) | ||
.is_some() | ||
|| sn.block_height < highest_sn.block_height | ||
{ | ||
debug!("Drive burn block processing: possible PoX reorg (sortition tip: {sortition_tip_affirmation_map}, heaviest: {heaviest_affirmation_map}, {} <? {})", sn.block_height, highest_sn.block_height); | ||
globals.coord().announce_new_burn_block(); | ||
globals.coord().announce_new_stacks_block(); | ||
*last_announce_time = get_epoch_time_secs().into(); | ||
} else if sortition_tip_affirmation_map.len() >= heaviest_affirmation_map.len() | ||
&& sortition_tip_affirmation_map.len() <= canonical_affirmation_map.len() | ||
{ | ||
if let Some(divergence_rc) = | ||
canonical_affirmation_map.find_divergence(&sortition_tip_affirmation_map) | ||
{ | ||
if divergence_rc + 1 >= (heaviest_affirmation_map.len() as u64) { | ||
// we have unaffirmed PoX anchor blocks that are not yet processed in the sortition history | ||
debug!("Drive burnchain processing: possible PoX reorg from unprocessed anchor block(s) (sortition tip: {sortition_tip_affirmation_map}, heaviest: {heaviest_affirmation_map}, canonical: {canonical_affirmation_map})"); | ||
globals.coord().announce_new_burn_block(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should I still be announcing burn block and stacks block? Feel like something is missing here... |
||
globals.coord().announce_new_stacks_block(); | ||
*last_announce_time = get_epoch_time_secs().into(); | ||
} | ||
} | ||
} else { | ||
debug!( | ||
"Drive burn block processing: no need (sortition tip: {sortition_tip_affirmation_map}, heaviest: {heaviest_affirmation_map}, {} </ {})", | ||
sn.block_height, | ||
highest_sn.block_height | ||
); | ||
} | ||
|
||
*last_burn_pox_reorg_recover_time = get_epoch_time_secs().into(); | ||
|
||
// unconditionally bump every 5 minutes, just in case. | ||
|
@@ -1051,7 +884,6 @@ impl RunLoop { | |
config: Config, | ||
burnchain: Burnchain, | ||
sortdb: SortitionDB, | ||
chain_state_db: StacksChainState, | ||
) { | ||
let mut last_burn_pox_reorg_recover_time = 0; | ||
let mut last_stacks_pox_reorg_recover_time = 0; | ||
|
@@ -1066,15 +898,12 @@ impl RunLoop { | |
&config, | ||
&burnchain, | ||
&sortdb, | ||
&chain_state_db, | ||
&mut last_burn_pox_reorg_recover_time, | ||
&mut last_burn_announce_time, | ||
); | ||
Self::drive_pox_reorg_stacks_block_processing( | ||
&globals, | ||
&config, | ||
&burnchain, | ||
&sortdb, | ||
&mut last_stacks_pox_reorg_recover_time, | ||
); | ||
|
||
|
@@ -1092,20 +921,10 @@ impl RunLoop { | |
.open_sortition_db(true) | ||
.expect("FATAL: could not open sortition DB"); | ||
|
||
let (chain_state_db, _) = StacksChainState::open( | ||
config.is_mainnet(), | ||
config.burnchain.chain_id, | ||
&config.get_chainstate_path_str(), | ||
Some(config.node.get_marf_opts()), | ||
) | ||
.unwrap(); | ||
|
||
thread::Builder::new() | ||
.name(format!("chain-liveness-{}", config.node.rpc_bind)) | ||
.stack_size(BLOCK_PROCESSOR_STACK_SIZE) | ||
.spawn(move || { | ||
Self::drive_chain_liveness(globals, config, burnchain, sortdb, chain_state_db) | ||
}) | ||
.spawn(move || Self::drive_chain_liveness(globals, config, burnchain, sortdb)) | ||
.expect("FATAL: failed to spawn chain liveness thread") | ||
} | ||
|
||
|
@@ -1209,19 +1028,9 @@ impl RunLoop { | |
let liveness_thread = self.spawn_chain_liveness_thread(globals.clone()); | ||
|
||
// Wait for all pending sortitions to process | ||
let mut burnchain_db = burnchain_config | ||
let burnchain_db = burnchain_config | ||
.open_burnchain_db(true) | ||
.expect("FATAL: failed to open burnchain DB"); | ||
if !self.config.burnchain.affirmation_overrides.is_empty() { | ||
let tx = burnchain_db | ||
.tx_begin() | ||
.expect("FATAL: failed to begin burnchain DB tx"); | ||
for (reward_cycle, affirmation) in self.config.burnchain.affirmation_overrides.iter() { | ||
tx.set_override_affirmation_map(*reward_cycle, affirmation.clone()).unwrap_or_else(|_| panic!("FATAL: failed to set affirmation override ({affirmation}) for reward cycle {reward_cycle}")); | ||
} | ||
tx.commit() | ||
.expect("FATAL: failed to commit burnchain DB tx"); | ||
} | ||
let burnchain_db_tip = burnchain_db | ||
.get_canonical_chain_tip() | ||
.expect("FATAL: failed to query burnchain DB"); | ||
|
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like I might have a bug here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Announcing new burn blocks and stacks blocks is a no-op if there are no blocks present, but we can't just busy-loop doing this because they incur I/O. I think that replacing the "drive chain liveness" code with a loop that just calls
announce_new_burn_block()
andannounce_new_stacks_block()
every so often should be enough