Skip to content

[DNM/WIP] PSS: multistage provider download experiment #37350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 39 commits into
base: main
Choose a base branch
from
Draft

[DNM/WIP] PSS: multistage provider download experiment #37350

wants to merge 39 commits into from

Conversation

SarahFrench
Copy link
Member

WIP

Target Release

1.14.x

Rollback Plan

  • If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

CHANGELOG entry

  • This change is user-facing and I added a changelog entry.
  • This change is not user-facing.

@SarahFrench SarahFrench added the no-changelog-needed Add this to your PR if the change does not require a changelog entry label Jul 18, 2025
Base automatically changed from pss/multistage-provider-download-prep to main July 30, 2025 18:11
@SarahFrench SarahFrench force-pushed the pss/multistage-provider-download-experiment branch from 35e5fd6 to e4e4587 Compare July 31, 2025 10:00
@SarahFrench
Add forked version of run logic that's only used if experiments are…
e9245e7 
… enabled
@SarahFrench
Add getProvidersFromConfig method, initially as an exact copy of getP…
bdd7816 
…roviders
@SarahFrench
Make getProvidersFromConfig not use state to get providers
9dc5cd2
@SarahFrench
Reorder actions in experimental init - load in full config before con…
ef1acfd 
…figuring the backend.
@SarahFrench
Add appendLockedDependencies method to Meta to allow multi-phase …
6045248 
…saving to the dep locks file
@SarahFrench
Update test
9fd914c
@SarahFrench
Update experimental init to use new getProvidersFromConfig method
18a51e3
@SarahFrench
Remove old usage of getProviders
7143565 
At this point we're unable to download the providers only described by state
@SarahFrench
Remove misleading comment
a51e1ad 
This isn't accurate after previous reordering!
@SarahFrench
Add new getProvidersFromState method that only accepts state informat…
ea3ec60 
…ion as input for getting providers. Use in experimental init and append values to existing deps lock file
@SarahFrench
Update messages sent to view about provider download phases
446a748
@SarahFrench
Change init to save updates to the deps lock file only once
0cf33c6
@SarahFrench
Make Terraform output report that a lock file _will_ be made after pr…
3ad0c30 
…oviders are determined from config
@SarahFrench
Remove use of ProviderDownloadOutcomes
ade36e8
@SarahFrench
Move repeated code into separate method
d7a4fd9
@SarahFrench
Change provider download approach: determine if locks changed at poin…
1328f98 
…t of attempting to update the lockfile, keep record of incomplete providers inside init command struct
@SarahFrench
Refactor mergeLockedDependencies and update test
740d810
@SarahFrench
Add comments to provider download methods
f2e719b
@SarahFrench
Fix issue where incorrect message ouput to view when downloading prov…
ebb3d27 
…iders
@SarahFrench
Add WIP tests for 2 phase provider download
6c9a127
@SarahFrench
Update mergeLockedDependencies method to be more generic
2126e55
@SarahFrench
Update getProvidersFromState method to receive in-progress config l…
e8b03af 
…ocks and merge those with any locks on file. This allows re-use of providers downloaded by `getProvidersFromConfig` in the same init command
@SarahFrench
Fix config for TestInit_stateStoreBlockIsExperimental
ad0db8e
@SarahFrench
Improve testing of mergeLockedDependencies; state locks are always mi…
ecc2313 
…ssing version constraints
@SarahFrench
Refactor tests, add test case showing providers aren't downloaded twice
e56ecdf
@SarahFrench
Add test case to cover use of the -upgrade flag
0ea0c6f
@SarahFrench
Change the message shown when a provider is reused during the second …
4e65895 
…provider download step.

When downloading providers described only in the state then the provider may already be downloaded from a previous init (i.e. is recorded in the deps lock file) or downloaded during step 1 of provider download. The message here needs to cover both potential scenarios.
@SarahFrench
Update mergeLockedDependencies comment
e089961
@SarahFrench SarahFrench force-pushed the pss/multistage-provider-download-experiment branch from 23f73b1 to e089961 Compare August 6, 2025 17:06
@SarahFrench
Copy link
Member Author

SarahFrench commented Aug 8, 2025 •
edited
Loading

TestInit_stateStoreBlockIsExperimental is passing when run in isolation, but is failing when run in parallel to other tests. Looking into why that's the case...

Edit: it's due to the modification of the package level terraformBlockSchema variable. 🤦🏻 - this is fine for gating the feature for real users, but means test can interfere with each other.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
no-changelog-needed Add this to your PR if the change does not require a changelog entry
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@SarahFrench