memfs: fatal error: concurrent map writes

[faruqisan]

Hi, I was trying to run memfs create inside go routine, and faced this error:

Error:

goroutine 1035 [running]:
runtime.throw(0x21e1a74, 0x15)
	/usr/local/go/src/runtime/panic.go:1112 +0x72 fp=0xc0006aba50 sp=0xc0006aba20 pc=0x1034a22
runtime.mapassign_faststr(0x1f6cf80, 0xc000c7d680, 0xc00003f4d7, 0xa, 0xc000a5ed88)
	/usr/local/go/src/runtime/map_faststr.go:211 +0x3f7 fp=0xc0006abab8 sp=0xc0006aba50 pc=0x1015377
github.com/go-git/go-billy/v5/memfs.(*storage).createParent(0xc0003866e0, 0xc00003f4a0, 0x41, 0x1b6, 0xc000422030, 0x16366d5, 0xc0003866e0)
	/Users/myuser/go/pkg/mod/github.com/go-git/go-billy/v5@v5.0.0/memfs/storage.go:69 +0x189 fp=0xc0006abb30 sp=0xc0006abab8 pc=0x1636289
github.com/go-git/go-billy/v5/memfs.(*storage).New(0xc0003866e0, 0xc00003f4a0, 0x41, 0x1b6, 0x602, 0xc00003f4a0, 0x41, 0x1)
	/Users/myuser/go/pkg/mod/github.com/go-git/go-billy/v5@v5.0.0/memfs/storage.go:50 +0x29f fp=0xc0006abbb0 sp=0xc0006abb30 pc=0x163608f
github.com/go-git/go-billy/v5/memfs.(*Memory).OpenFile(0xc000386700, 0xc00003f4a0, 0x41, 0x602, 0x1b6, 0x2, 0xc00003f4a0, 0x41, 0xc0006abcc8)
	/Users/myuser/go/pkg/mod/github.com/go-git/go-billy/v5@v5.0.0/memfs/memory.go:49 +0xaf fp=0xc0006abc38 sp=0xc0006abbb0 pc=0x1633fff
github.com/go-git/go-billy/v5/memfs.(*Memory).Create(0xc000386700, 0xc00003f4a0, 0x41, 0xc00003f4a0, 0x41, 0x1, 0x21cdcd6)
	/Users/myuser/go/pkg/mod/github.com/go-git/go-billy/v5@v5.0.0/memfs/memory.go:34 +0x50 fp=0xc0006abc90 sp=0xc0006abc38 pc=0x1633e60
github.com/go-git/go-billy/v5/helper/polyfill.(*Polyfill).Create(0xc00012c8a0, 0xc00003f4a0, 0x41, 0xc00003f4a0, 0x41, 0x0, 0x0)
	<autogenerated>:1 +0x50 fp=0xc0006abcd8 sp=0xc0006abc90 pc=0x13b9670
github.com/go-git/go-billy/v5/helper/chroot.(*ChrootHelper).Create(0xc00012c8c0, 0xc00079ab80, 0x40, 0xc00079ab80, 0x40, 0xc000391e80, 0x6)
	/Users/myuser/go/pkg/mod/github.com/go-git/go-billy/v5@v5.0.0/helper/chroot/chroot.go:49 +0x80 fp=0xc0006abd28 sp=0xc0006abcd8 pc=0x13ba210
project/internal/services/generator.(*Engine).GeneratePostgreSQLPrimaryReplica.func1(0x0, 0x0)
	/Users/myuser/Works/myprojects/project/internal/services/generator/template_pg_primary_replica.go:175 +0x14a fp=0xc0006abf78 sp=0xc0006abd28 pc=0x163db8a
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc000c7d5f0, 0xc000e42f40)
	/Users/myuser/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:57 +0x59 fp=0xc0006abfd0 sp=0xc0006abf78 pc=0x163a6f9
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc0006abfd8 sp=0xc0006abfd0 pc=0x1064cd1
created by golang.org/x/sync/errgroup.(*Group).Go
	/Users/myuser/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:54 +0x66

truncated code:

import "github.com/go-git/go-billy/v5/memfs"
bfs := memfs.New()
eg := errgroup.Group{}

for _, val := range vals {
	eg.Go(func() error {
             f, err := bfs.Create(filepath.Join(dirPath, ns))
             if err != nil {
			return err
	     }
	     defer f.Close()
        }
}

// wait eg

Any Idea how to fix this? or it's not supposed to run inside go routine?

[ilius]

I don't think memfs is concurrent-safe (it has no locks for maps) because it's mostly made for unit testing.
If you want to use it for production, you should fork it and add mutex locks to it.