Crash replays all the value log files

[jarifibrahim]

What version of Badger are you using?

14386ac

Does this issue reproduce with the latest master?

Yes

Steps to Reproduce the issue

What Badger options were set?

opt := badger.DefaultOptions(sstDir).    
         WithValueDir(vlogDir).    
         WithTruncate(truncate).    
         WithSyncWrites(false).    
         WithKeepBlocksInCache(true).    
         WithMaxCacheSize(200 << 20).    
         WithKeepBlockIndicesInCache(true).    
         WithCompression(options.ZSTD).    
         WithCompactL0OnClose(force)    

What did you do?

Run benchmark write tool

go run main.go benchmark write -m 400 --dir ~/m2_drive/100m -l 2>&1 | tee 100m-logs.txt

What did you expect to see?

Value log replayed to replay only a few recent files

What did you see instead?

Badger is replaying file with id 0 offset 0, I have 1164 files

ls ~/m2_drive/100m/ | grep .vlog | wc -l
1164

go run main.go benchmark write -m 400 --dir ~/m2_drive/100m -l 2>&1 | tee 100m-logs.txt
Listening for /debug HTTP requests at port: 8080
badger 2020/06/11 17:28:28 INFO: 335 tables out of 1170 opened in 3.002s
badger 2020/06/11 17:28:31 INFO: 669 tables out of 1170 opened in 6.026s
badger 2020/06/11 17:28:34 INFO: 1004 tables out of 1170 opened in 9.015s
badger 2020/06/11 17:28:36 INFO: All 1170 tables opened in 10.797s
badger 2020/06/11 17:28:36 INFO: Replaying file id: 0 at offset: 0
badger 2020/06/11 17:28:37 INFO: Got compaction priority: {level:2 score:1.1463758505880832 dropPrefix:[]}
badger 2020/06/11 17:28:37 INFO: Running for level: 2
badger 2020/06/11 17:28:38 INFO: Got compaction priority: {level:3 score:1.1370206640660763 dropPrefix:[]}
badger 2020/06/11 17:28:38 INFO: Running for level: 3
badger 2020/06/11 17:28:38 INFO: Replay took: 2.271997389s
badger 2020/06/11 17:28:38 INFO: Replaying file id: 1 at offset: 0

[jarifibrahim]

The issue seems to be happening because we update the head pointer only after replay is successful

badger/value.go

Line 1193 in 14386ac

vlog.db.vhead = valuePointer{Fid: vlog.maxFid, Offset: uint32(lastOffset)}

I think we can update the head pointer as entries are being applied.
I haven't thought this through but the following might fix the issue.

diff --git a/db.go b/db.go
index 79b332d..dacd30f 100644
--- a/db.go
+++ b/db.go
@@ -136,6 +136,7 @@ func (db *DB) replayFunction() func(Entry, valuePointer) error {
 		} else {
 			nv = vp.Encode()
 			meta = meta | bitValuePointer
+			db.vhead = vp
 		}
 
 		v := y.ValueStruct{

[github-actions]

Use Discuss Issues for reporting issues about this repository.