Skip to content

[Feat] Add descriptive messages for skipped asserts#476

Merged
ljharb merged 1 commit intotape-testing:masterfrom
r0mflip:descriptive-asserts
Jun 29, 2019
Merged

[Feat] Add descriptive messages for skipped asserts#476
ljharb merged 1 commit intotape-testing:masterfrom
r0mflip:descriptive-asserts

Conversation

@r0mflip
Copy link
Copy Markdown
Contributor

@r0mflip r0mflip commented Jun 26, 2019

Now asserts can take skip as a string or boolean, strings get printed as the reason for skipping the assert.

(ok|not ok) 3 <assert message> # SKIP <reason for skipping>

@ljharb
Copy link
Copy Markdown
Collaborator

ljharb commented Jun 26, 2019

Is this a convenience feature, or something that you'd expect as part of any tap client? Are their other TAP-producing runners that do this?

@r0mflip
Copy link
Copy Markdown
Contributor Author

r0mflip commented Jun 27, 2019

TAP indicates that any individual assert can be skipped or marked todo. node-tap allows this and this also plays really well with tap-parser. The same should work for todo, but node 0.8 has problems with the stack.

test('this skipps', { skip: true }, function (t) {
    t.fail('doesn\'t run');
    t.fail('this doesn\'t run too', { skip: false });
    t.end();
});

test('some tests might skip', function (t) {
    t.pass('this runs');
    t.fail('failing assert is skipped', { skip: true });
    t.pass('this runs');
    t.end();
});

test('incomplete test', function (t) {
    // var platform = process.platform; something like this needed
    var platform = 'win32';

    t.pass('run sh', { skip: platform !== 'win32' });
    t.pass('run openssl', { skip: platform === 'win32' });
    t.end();
});

test('incomplete test with explanation', function (t) {
    // var platform = process.platform; something like this needed
    var platform = 'win32';

    t.fail('run sh (conditional skip)', { skip: platform === 'win32' });
    t.fail('run openssl', { skip: platform === 'win32' && 'can\'t run on windows platforms' });
    t.pass('this runs');
    t.end();
});

tap-parser output

[
  [ 'version', 13 ],
  [ 'comment', '# SKIP this skipps\n' ],
  [ 'comment', '# some tests might skip\n' ],
  [
    'assert',
    Result { ok: true, id: 1, name: 'this runs', fullname: '' }
  ],
  [
    'assert',
    Result {
      ok: true,
      id: 2,
      skip: true,
      name: 'failing assert is skipped',
      fullname: ''
    }
  ],
  [
    'assert',
    Result { ok: true, id: 3, name: 'this runs', fullname: '' }
  ],
  [ 'comment', '# incomplete test\n' ],
  [
    'assert',
    Result { ok: true, id: 4, name: 'run sh', fullname: '' }
  ],
  [
    'assert',
    Result {
      ok: true,
      id: 5,
      skip: true,
      name: 'run openssl',
      fullname: ''
    }
  ],
  [ 'comment', '# incomplete test with explanation\n' ],
  [
    'assert',
    Result {
      ok: true,
      id: 6,
      skip: true,
      name: 'run sh (conditional skip)',
      fullname: ''
    }
  ],
  [
    'assert',
    Result {
      ok: true,
      id: 7,
      skip: "can't run on windows platforms",
      name: 'run openssl',
      fullname: ''
    }
  ],
  [
    'assert',
    Result { ok: true, id: 8, name: 'this runs', fullname: '' }
  ],
  [ 'plan', { start: 1, end: 8 } ],
  [ 'comment', '# tests 8\n' ],
  [ 'comment', '# pass  8\n' ],
  [ 'comment', '# ok\n' ],
  [ 'comment', '# skip: 4\n' ],
  [
    'complete',
    FinalResults {
      ok: true,
      count: 8,
      pass: 8,
      fail: 0,
      bailout: false,
      todo: 0,
      skip: 4,
      plan: FinalPlan {
        start: 1,
        end: 8,
        skipAll: false,
        skipReason: '',
        comment: ''
      },
      failures: []
    }
  ]
]

var common = require('./common');
var stripFullStack = common.stripFullStack;

tap.test('tape todo test', { todo: process.versions.node.match(/0\.8\.\d+/) ? 'Fails on node 0.8': false }, function (assert) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've temporarily marked it as todo

@ljharb ljharb force-pushed the descriptive-asserts branch from fc2e7d6 to 838d995 Compare June 29, 2019 05:02
@ljharb ljharb merged commit 838d995 into tape-testing:master Jun 29, 2019
@ljharb ljharb added the semver-minor: new stuff Any additions. label Jun 29, 2019
ljharb added a commit that referenced this pull request Jun 29, 2019
 - [New] Add descriptive messages for skipped asserts (#476)
 - [Fix] emit skipped tests as objects (#473)
 - [Refactor] use `!!` over `Boolean()`
 - [meta] clean up license so github can detect it
 - [Deps] update `inherits`, `resolve`
 - [Tests] add tests for 'todo' exit codes (#471)
@r0mflip r0mflip deleted the descriptive-asserts branch January 7, 2020 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants