fix: Refactor table tokens

[calculuschild]

Pretty small change to address #2165. header and cells are now formatted as "sub-tokens" (similar to list items in a list) rather than having separate headers and tokens.headers. This also means child tokens are now in a tokens property to follow the convention of the other lexer tokens for consistency.

Changes the Table signature though so this is breaking and should be in V3.0

Previous token signature:

{
  type: 'table',
  align: [null, null],
  raw: '| a | b |\n|---|---|\n| 1 | 2 |\n',
  header: ['a', 'b'],
  cells: [['1', '2']],
  tokens: {
    header: [
      [{ type: 'text', raw: 'a', text: 'a' }],
      [{ type: 'text', raw: 'b', text: 'b' }]
    ],
    cells: [[
      [{ type: 'text', raw: '1', text: '1' }],
      [{ type: 'text', raw: '2', text: '2' }]
    ]]
  }
}

New token signature

{
  type: 'table',
  align: [null, null],
  raw: '| a | b |\n|---|---|\n| 1 | 2 |\n',
  header: {
    text: ['a', 'b'],
    tokens: [
      [{ type: 'text', raw: 'a', text: 'a' }],
      [{ type: 'text', raw: 'b', text: 'b' }]
    ]
  },
  cells: {
    text: [['1', '2']],
    tokens: [[
      [{ type: 'text', raw: '1', text: '1' }],
      [{ type: 'text', raw: '2', text: '2' }]
    ]]
  }
}

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/markedjs/markedjs/38ErXbTFie3utmH93Wg33w7gpLRQ
✅ Preview: https://markedjs-git-fork-calculuschild-refactortablece-f3c032-markedjs.vercel.app

[calculuschild]

Bonus! CodeQL found a Regex vulnerability for an edge case that we already handled in #2126

[UziTech]

LGTM

[UziTech]

Now that I think about it the tokens property here is an array of token arrays.

Maybe instead we should do:

{
  type: 'table',
  align: [null, null],
  raw: '| a | b |\n|---|---|\n| 1 | 2 |\n',
  header: {
    items: [
      {
        text: 'a',
        tokens: [{ type: 'text', raw: 'a', text: 'a' }],
      },
      {
        text: 'b',
        tokens: [{ type: 'text', raw: 'b', text: 'b' }],
      }
    ]
  },
  rows: [ // replace `cells` with `rows`
    { // row 1
      items: [
        {
          text: '1',
          tokens: [{ type: 'text', raw: '1', text: '1' }],
        },
        {
          text: '2',
          tokens: [{ type: 'text', raw: '2', text: '2' }],
        }
      ]
    }
  ]
}

That seems to be more consistent with the rest of the tokens (including lists)

[calculuschild]

Hm. Good point. I have a thought on this then:

Is it worth it to have header and row as objects with a single property items? An object with a single property seems redundant and maybe slightly slows things with extra object accesses? What if instead header and rows are just arrays like items is an array? So a List has Items, a Table has Headers and Rows, but they use the same format of being an array of "sub-tokens" (with rows being and array of arrays of sub-tokens):

{
  type: 'table',
  align: [null, null],
  raw: '| a | b |\n|---|---|\n| 1 | 2 |\n',
  header: [
    {
      text: 'a',
      tokens: [{ type: 'text', raw: 'a', text: 'a' }],
    },
    {
      text: 'b',
      tokens: [{ type: 'text', raw: 'b', text: 'b' }],
    }
  ],
  rows: [ // replace `cells` with `rows`
    [ // row 1
      {
        text: '1',
        tokens: [{ type: 'text', raw: '1', text: '1' }],
      },
      {
        text: '2',
        tokens: [{ type: 'text', raw: '2', text: '2' }],
      }
    ]
  ]
}

TLDR:

Instead of

• Table
  • which has a property of header
    *which has as sub-token of items which seems like one level too deep.

we could have:

• Table
  • which as some sub-tokens header.

[UziTech]

That sounds good

[calculuschild]

@UziTech This is now changed. Want another look at it?

[github-actions]

🎉 This PR is included in version 3.0.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀