Skip to content

Backup schema keys in incremental backups.#5147

Merged
martinmr merged 4 commits intomasterfrom
martinmr/backup-schema
Apr 9, 2020
Merged

Backup schema keys in incremental backups.#5147
martinmr merged 4 commits intomasterfrom
martinmr/backup-schema

Conversation

@martinmr
Copy link
Copy Markdown
Contributor

@martinmr martinmr commented Apr 9, 2020

Description.

The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.

Also ensured that the Version of the schema keys is always one. AFAIK this is not
an issue but I think it's better to explicitly set the versions of these keys to 1 during
backups and restores.

GitHub Issue or Jira number.

https://dgraph.atlassian.net/browse/DGRAPH-1224

Affected releases.

20.03.1

Changelog tags.

fixed

Please indicate if this is a breaking change.

No

Please indicate if this is an enterprise feature.

Yes

Please indicate if documentation needs to be updated.

No

Please indicate if end to end testing is needed.

I added unit tests to ensure the schema is correct but it'd be good to manually test this before.


This change is Reviewable

@martinmr martinmr requested a review from manishrjain as a code owner April 9, 2020 01:14
@martinmr martinmr requested a review from a team April 9, 2020 01:14
Comment thread worker/restore.go Outdated
Comment thread worker/restore.go Outdated
Copy link
Copy Markdown
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @martinmr)


worker/backup_processor.go, line 285 at r2 (raw file):

			Value:     valCopy,
			UserMeta:  []byte{item.UserMeta()},
			Version:   1,

item.Version()


worker/restore.go, line 193 at r2 (raw file):

				// version value of one.
				kv.Key = restoreKey
				kv.Version = 1

Use the same version.

@parasssh
Copy link
Copy Markdown
Contributor

parasssh commented Apr 9, 2020

We should back port this 20.03 as well as 1.2

Copy link
Copy Markdown
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 5 of 7 files reviewed, 4 unresolved discussions (waiting on @golangcibot and @manishrjain)


worker/backup_processor.go, line 285 at r2 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

item.Version()

Done.


worker/restore.go, line 89 at r1 (raw file):

Previously, golangcibot (Bot from GolangCI) wrote…

Error return value of db.DropPrefix is not checked (from errcheck)

Done.


worker/restore.go, line 90 at r1 (raw file):

Previously, golangcibot (Bot from GolangCI) wrote…

Error return value of db.DropPrefix is not checked (from errcheck)

Done.


worker/restore.go, line 193 at r2 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

Use the same version.

Done.

@martinmr martinmr merged commit f8beadd into master Apr 9, 2020
@martinmr martinmr deleted the martinmr/backup-schema branch April 9, 2020 23:41
martinmr added a commit that referenced this pull request Apr 10, 2020
The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.
martinmr added a commit that referenced this pull request Apr 10, 2020
The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.
martinmr added a commit that referenced this pull request Apr 10, 2020
The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.
dna2github pushed a commit to dna2fork/dgraph that referenced this pull request Jul 18, 2020
The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.
martinmr added a commit that referenced this pull request Aug 3, 2020
The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.

(cherry picked from commit f8beadd)
martinmr added a commit that referenced this pull request Aug 3, 2020
The full schema should be backed up during incremental backups.
During restore, the schema should be cleared before processing each backup
to end up with only the final schema.

(cherry picked from commit f8beadd)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants