Skip to content

Add new subcommand to update migrations that contain breaking changes #855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jun 13, 2025

Conversation

kvch
Copy link
Contributor

@kvch kvch commented May 22, 2025

This PR adds a new subcommand called update. It updates migrations that contain breaking changes in the selected folder.

Usage:
  pgroll update <directory> [flags]

Examples:
update ./migrations

Flags:
  -h, --help    help for update
  -j, --json    Output migration file in JSON format instead of YAML

Dev notes

The new migrations.FileUpdater follows the chain of responsibility design pattern. In the FileUpdater you can register new updater functions (migrations.updaterFn) for any pgroll migration. The file updater iterates over all operations in a migration and run the registered updater functions in order.

The ordering of the updater functions is important. Please always add a new updater function to the end of the list.

Closes #770

@github-actions github-actions bot temporarily deployed to Docs Preview May 22, 2025 17:50 Inactive
@kvch kvch force-pushed the feature-update-command-i branch from 8ceb579 to dbe2d33 Compare May 22, 2025 17:51
@github-actions github-actions bot temporarily deployed to Docs Preview May 22, 2025 17:52 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview May 22, 2025 17:54 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview May 22, 2025 17:55 Inactive
@kvch kvch force-pushed the feature-update-command-i branch from 20340d9 to 97926a0 Compare May 22, 2025 17:56
@github-actions github-actions bot temporarily deployed to Docs Preview May 22, 2025 17:56 Inactive
@andrew-farries andrew-farries self-requested a review June 2, 2025 11:15
Copy link
Collaborator

@andrew-farries andrew-farries left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good; left some comments 👍

cmd/update.go Outdated
}

updateCmd.Flags().BoolVarP(&useJSON, "json", "j", false, "Output migration file in JSON format instead of YAML")
updateCmd.Flags().BoolVarP(&local, "local", "l", false, "Update all local migration files")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this flag. IMO the update command should awlays operate on all migrations in the given directory.

I'm not sure what the use-case is for only updating the local migations that have yet to be applied.

In any case, source control allows users to only commit the updates they want.

I'd remove the flag and make the command always operate on all migrations.

Copy link
Contributor Author

@kvch kvch Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --local flag is running on all local files, not just unapplied ones. By default pgroll updates only unapplied migrations. I think the flag is valuable when you pull all migrations from the database and run the command with --local. Or when you do not specify the flag you just update the local, unapplied files because we do not want to rewrite the history in the database. WDYT?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still in favour of removing the flag and having the command always run on all files in the local directory.

The use case for this command AIUI is:

  1. A user upgrades pgroll from version x to version y.
  2. Version y makes a breaking change to the YAML/JSON format for a particular operation.
  3. The user has already applied some migrations in the old format to their target database.

Now as a user I want to get all migrations into the new format so that I have a set of migration files for my project that I can use with the version y of pgroll.

  1. The user runs pgroll update migrations/ to rewrite the migration files.
  2. The user commits those changes to source control.

Now I can bootstrap other DBs from scratch with the migration files in migrations/.

I don't see the use case for only updating the migration files that have yet to be applied to the database?

we do not want to rewrite the history in the database

This command only ever touches local files, not migrations in the database though right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command only ever touches local files, not migrations in the database though right?

Yes.

@github-actions github-actions bot temporarily deployed to Docs Preview June 3, 2025 14:18 Inactive
@kvch kvch force-pushed the feature-update-command-i branch from f3a5cb1 to 4c686c1 Compare June 4, 2025 14:10
@github-actions github-actions bot temporarily deployed to Docs Preview June 4, 2025 14:11 Inactive
@kvch kvch force-pushed the feature-update-command-i branch from 4c686c1 to 8466301 Compare June 4, 2025 14:26
@github-actions github-actions bot temporarily deployed to Docs Preview June 4, 2025 14:27 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 4, 2025 14:28 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 5, 2025 08:16 Inactive
@kvch kvch requested a review from andrew-farries June 10, 2025 15:37
Copy link
Collaborator

@andrew-farries andrew-farries left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good; left a few comments.

cmd/update.go Outdated
}

updateCmd.Flags().BoolVarP(&useJSON, "json", "j", false, "Output migration file in JSON format instead of YAML")
updateCmd.Flags().BoolVarP(&local, "local", "l", false, "Update all local migration files")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still in favour of removing the flag and having the command always run on all files in the local directory.

The use case for this command AIUI is:

  1. A user upgrades pgroll from version x to version y.
  2. Version y makes a breaking change to the YAML/JSON format for a particular operation.
  3. The user has already applied some migrations in the old format to their target database.

Now as a user I want to get all migrations into the new format so that I have a set of migration files for my project that I can use with the version y of pgroll.

  1. The user runs pgroll update migrations/ to rewrite the migration files.
  2. The user commits those changes to source control.

Now I can bootstrap other DBs from scratch with the migration files in migrations/.

I don't see the use case for only updating the migration files that have yet to be applied to the database?

we do not want to rewrite the history in the database

This command only ever touches local files, not migrations in the database though right?

Co-authored-by: Andrew Farries <[email protected]>
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 11:27 Inactive
Co-authored-by: Andrew Farries <[email protected]>
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 11:27 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 11:28 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 11:55 Inactive
@kvch
Copy link
Contributor Author

kvch commented Jun 12, 2025

Updated the PR. I will update the guide in my other PR after this gets merged.

@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 12:01 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 12:06 Inactive
@kvch kvch requested a review from andrew-farries June 12, 2025 14:04
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 14:04 Inactive
Co-authored-by: Andrew Farries <[email protected]>
@github-actions github-actions bot temporarily deployed to Docs Preview June 13, 2025 10:48 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 13, 2025 11:27 Inactive
@kvch kvch requested a review from andrew-farries June 13, 2025 13:15
@kvch kvch merged commit c3fddb3 into xataio:main Jun 13, 2025
29 of 30 checks passed
kvch added a commit that referenced this pull request Jun 30, 2025
This PR adds more details to `update` command.

Requires #855

---------

Co-authored-by: Andrew Farries <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide a smoother path for pgroll version upgrades
2 participants