Skip to content
This repository was archived by the owner on Dec 15, 2022. It is now read-only.

RFC: use git ls to crawl all files in the project #367

Closed
wants to merge 1 commit into from
Closed

RFC: use git ls to crawl all files in the project #367

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion lib/load-paths-handler.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ const path = require('path')
const {GitRepository} = require('atom')
const {Minimatch} = require('minimatch')

const childProcess = require('child_process')

const PathsChunkSize = 100

const emittedPaths = new Set()
Expand All @@ -27,13 +29,46 @@ class PathLoader {
}

load (done) {
if (this.repo && !this.traverseSymlinkDirectories) {
this.loadFromGit().then(done)

return
}

this.loadPath(this.rootPath, true, () => {
this.flushPaths()
if (this.repo != null) this.repo.destroy()
done()
})
}

async loadFromGit () {
return new Promise((resolve) => {
const args = ['ls-files', '--cached', '--exclude-standard', '--others', '-z']

for (let ignoredName of this.ignoredNames) {
args.push('--exclude', ignoredName.pattern)
}

let output = ''

// TODO: do this via a call to GitRepository (needs to be implemented).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have quite a bit of infrastructure within atom/github to interact with external git processes - with queueing, caching, and a sidecar worker process to fix a sporadic performance issue with child_process calls... see atom/github#386 and atom/github#688 for some background on that particular mess 😆 . We also bundle a git executable, so we don't need to rely on a pre-existing user installation. Rather than calling GitRepository, we should publish a package service that allows other packages like this one to make git calls on demand. See atom/github#1089 for some thoughts on this - this might be a good motivating case 😄

const result = childProcess.spawn('git', args, {cwd: this.rootPath})
result.stdout.on('data', chunk => {
const files = (output + chunk).split('\0')
output = files.pop()

for (const file of files) {
this.pathLoaded(file)
}
})
result.on('close', () => {
this.flushPaths()
resolve()
})
})
}

isIgnored (loadedPath) {
const relativePath = path.relative(this.rootPath, loadedPath)
if (this.repo && this.repo.isPathIgnored(relativePath)) {
Expand All @@ -54,7 +89,8 @@ class PathLoader {
if (this.paths.length === PathsChunkSize) {
this.flushPaths()
}
done()

done && done()
}

flushPaths () {
Expand Down