-
-
Notifications
You must be signed in to change notification settings - Fork 32.2k
test: test make doc and verify toc #16208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
test/sequential/test-make-doc.js
Outdated
} | ||
} | ||
|
||
exec('make doc', { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On a second thought...maybe we can just add doc-only
to the dependency of make test
and then only use this file to do verification?
Ping @nodejs/testing @nodejs/documentation |
Failures on FreeBSD seem relevant. E.g., https://ci.nodejs.org/job/node-test-commit-freebsd/12335/nodes=freebsd10-64/console: not ok 1973 sequential/test-make-doc
---
duration_ms: 0.193
severity: fail
stack: |-
stderr is not empty:
make[2]: illegal argument to -j -- must be positive integer!
assert.js:709
assert.ifError = function ifError(err) { if (err) throw err; };
^
Error: Command failed: make doc
make[2]: illegal argument to -j -- must be positive integer!
at ChildProcess.exithandler (child_process.js:269:12)
at emitTwo (events.js:136:13)
at ChildProcess.emit (events.js:227:7)
at maybeClose (internal/child_process.js:944:16)
at Socket.stream.socket.on (internal/child_process.js:364:11)
at emitOne (events.js:126:13)
at Socket.emit (events.js:224:7)
at Pipe._handle.close [as _onclose] (net.js:551:12)
... |
@nodejs/build Is |
Is it worth to make this separable in a test suite and to update |
test/sequential/test-make-doc.js
Outdated
const common = require('../common'); | ||
|
||
const assert = require('assert'); | ||
const exec = require('child_process').exec; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe destructuring?
test/sequential/test-make-doc.js
Outdated
function verifyToc() { | ||
const apiPath = path.join(common.projectDir, 'out', 'doc', 'api'); | ||
const docs = fs.readdirSync(apiPath); | ||
assert.notStrictEqual(docs.indexOf('_toc.html'), -1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert.ok(docs.includes('_toc.html'));
?
test/sequential/test-make-doc.js
Outdated
|
||
const linkedHtmls = links.map((link) => link.match(re)[1]); | ||
for (const html of linkedHtmls) { | ||
assert.notStrictEqual(docs.indexOf(html), -1, `${html} does not exist`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto + the message?
test/sequential/test-make-doc.js
Outdated
|
||
for (const html of generatedHtmls) { | ||
assert.notStrictEqual( | ||
linkedHtmls.indexOf(html), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto?
test/sequential/test-make-doc.js
Outdated
} | ||
} | ||
|
||
exec('make doc', { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need a shell for make
or execFile()
suffices?
test/sequential/test-make-doc.js
Outdated
}, common.mustCall(function onExit(err, stdout, stderr) { | ||
console.log(stdout); | ||
if (stderr.length > 0) { | ||
console.log('stderr is not empty: '); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
console.error()
here and in the next line? Or stderr should not be used in tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. Any suggestions are completely arbitrary)
test/sequential/test-make-doc.js
Outdated
cwd: common.projectDir | ||
}, common.mustCall(function onExit(err, stdout, stderr) { | ||
console.log(stdout); | ||
if (stderr.length > 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't assert.strictEqual(stderr, '');
give pretty much the same results?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cjihrig The stderr
would be truncated by AssertionError
to 128 chars, personally I prefer to keep it intact
Implement common.projectDir which points to the project directory.
Tried to put the build step into Makefile to work around the BSD make. New CI: https://ci.nodejs.org/job/node-test-pull-request/10775/ |
Oops, put |
Hmm..putting
I will try to investigate later. Also cc/ @nodejs/build if anyone knows what's going on with this error... |
There is a redundant slash in the definition of Anyway, a new CI run looks close (https://ci.nodejs.org/job/node-test-commit/13219/), although there is a related error on
|
Addressed nits by @refack . CI before landing: https://ci.nodejs.org/job/node-test-pull-request/10804/ |
There were multiple unrelated inspector tests errors that should hopefully fixed by #16281. Just to be safe: https://ci.nodejs.org/job/node-test-commit/13256/ |
Implement common.projectDir which points to the project directory. PR-URL: #16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
PR-URL: #16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
CI failures are unrelated. Landed in e6dfd59...df5dc2d, thanks! |
Should we make a rule to run full CI for doc PRs now? Or would this be overkill? |
@vsemozhetbyt I think we can do it optionally, if we want to make sure. The one thing we probably really should do is run (Also: @joyeecheung Awesome work! Didn’t see this PR until just now, but nice that we have this now.) |
@vsemozhetbyt Good question. I am thinking about a |
AFAIK nodejs/build#887 and similars were caused by code changes in |
We can also consider joining linter CI and doc CI in a new CI job. Previously, we used linter CI also for benchmark PRs, but now we tend to make benchmarks fully testable, so the separate linter CI job remains useful only for the docs with code fragments. We can as well make these two CI joined in some special light CI job for docs. |
And there's the #12756 dream |
Implement common.projectDir which points to the project directory. PR-URL: #16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
PR-URL: #16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
@@ -59,6 +59,8 @@ exports.enoughTestCpu = Array.isArray(cpus) && | |||
(cpus.length > 1 || cpus[0].speed > 999); | |||
|
|||
exports.rootDir = exports.isWindows ? 'c:\\' : '/'; | |||
exports.projectDir = path.resolve(__dirname, '..', '..'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this won't work on our cross-compiling Raspberry Pi devices in CI. @nodejs/build ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmmm...but it did pass in the CI, so...hmmm....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, perhaps CI was run before that specific feature was added? I'm certainly seeing it fail 100% of the time in CI right now as in https://ci.nodejs.org/job/node-test-binary-arm/11032/.
not ok 272 sequential/test-make-doc
---
duration_ms: 2.153
severity: fail
stack: |-
fs.js:926
return binding.readdir(pathModule.toNamespacedPath(path), options.encoding);
^
Error: ENOENT: no such file or directory, scandir '/home/iojs/build/workspace/node-test-binary-arm/out/doc/api'
at Object.fs.readdirSync (fs.js:926:18)
at Object.<anonymous> (/home/iojs/build/workspace/node-test-binary-arm/test/sequential/test-make-doc.js:15:17)
at Module._compile (module.js:607:30)
at Object.Module._extensions..js (module.js:618:10)
at Module.load (module.js:526:32)
at tryModuleLoad (module.js:489:12)
at Function.Module._load (module.js:481:3)
at Function.Module.runMain (module.js:648:10)
at startup (bootstrap_node.js:191:16)
at bootstrap_node.js:609:3
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if the problem isn't common.projectDir at all but instead that the Pi devices run make test-ci-js
and not make test
or make test-ci
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(...and so it skips the make doc
....which we probably want it to skip because who knows how long it would take to run on a Pi and we'd be running it on all of them just for a test that only one of them runs...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I've got a fix in #16301 but we'll see if CI agrees....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I wonder why this passed the CI before...anyway thanks for the fix!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@joyeecheung I think it passed CI because this change wasn't in the branch when CI ran. It was run before a5262f1 landed.
Implement common.projectDir which points to the project directory. PR-URL: nodejs/node#16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
PR-URL: nodejs/node#16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
Should this be backported to |
@MylesBorins I'd say no if it does not land cleanly |
PR-URL: nodejs/node#16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
Implement common.projectDir which points to the project directory. PR-URL: nodejs/node#16208 Fixes: nodejs/build#887 Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>
Fixes: nodejs/build#887
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passesAffected core subsystem(s)
test