Skip to content

Use txn writer to write schema postings#4296

Merged
ashish-goswami merged 2 commits intomasterfrom
ashish/issue-3916
Nov 28, 2019
Merged

Use txn writer to write schema postings#4296
ashish-goswami merged 2 commits intomasterfrom
ashish/issue-3916

Conversation

@ashish-goswami
Copy link
Copy Markdown
Contributor

@ashish-goswami ashish-goswami commented Nov 20, 2019

Fixes #3916

Although I was not able to reproduce this. Stacktrace suggests, while writing schema to Badger, it got crashed with error: Txn is too big to fit into one request. This might be because, we are using single transaction to write schema for all predicates.
This PR, replaces single transaction with TxnWriter to write schema.


This change is Reviewable

Copy link
Copy Markdown
Contributor

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small comment but otherwise it :lgtm:

Reviewed 1 of 1 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @animesh2049, @ashish-goswami, @mangalaman93, and @manishrjain)


dgraph/cmd/bulk/schema.go, line 146 at r1 (raw file):

func (s *schemaStore) write(db *badger.DB, preds []string) {
	// Write schema always at timestamp 1, s.state.writeTs may not be equal to 1

I'd move this comment right above the line where the schema is actually being written.

Also, change the comment to say "Write schema and types".

Copy link
Copy Markdown
Contributor Author

@ashish-goswami ashish-goswami left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @animesh2049, @mangalaman93, @manishrjain, and @martinmr)


dgraph/cmd/bulk/schema.go, line 146 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…

I'd move this comment right above the line where the schema is actually being written.

Also, change the comment to say "Write schema and types".

Done.

Copy link
Copy Markdown
Contributor

@mangalaman93 mangalaman93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, we should add a test that verifies this. We can change the name of the test such that it doesn't run by the CI.

Reviewed 1 of 1 files at r2.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @animesh2049, @manishrjain, and @martinmr)

Copy link
Copy Markdown
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r2.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @animesh2049 and @martinmr)

@ashish-goswami ashish-goswami merged commit a7dec1e into master Nov 28, 2019
@ashish-goswami ashish-goswami deleted the ashish/issue-3916 branch November 28, 2019 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Txn is too big to fit into one request at the very end of bulk import

4 participants