protect database with flock

[slingamn]

We got a report of database corruption with v2.7.0. It's not entirely clear what happened but it's possible that two instances of Ergo were running concurrently, resulting in interleaving of updates to buntdb's append-only log.

Suggestion:

1. Pull in https://github.com/gofrs/flock
2. Wrap it in a package irc/flock with build flags that make it a no-op on unsupported platforms
3. Acquire an exclusive flock on the buntdb database when opening it

[slingamn]

It appears that flock(2) (and presumably the related fcntl(2) call) operate at the level of the inode, rather than the filesystem path? So if the buntdb gets atomically shrunk and renamed, mutual exclusion would be violated.

Maybe lock the config file instead?

[tacerus]

I see interest in this: Locking the database instead of the configuration file would help with safe spawning of backup instances. We run Ergo on a failover IP address and currently need to pay extra care that the original instance is stopped before the second one starts and tries to access the database file (which resides on shared storage on a separate network). A lock on the database file would reduce the risk of accidentally spawning the second one too early in an automated situation where the first instance is no longer reachable to verify whether the process is stopped.

[slingamn]

What network file system is this?

I'm starting to think it would be better to define an optional flock-path key in the config file.

[tacerus]

What network file system is this?

GlusterFS.

I'm starting to think it would be better to define an optional flock-path key in the config file.

That'd be an interesting option too.