Run accessibility audit on CI

[jelbourn]

Tools to explore:
Chrome tools: https://github.com/GoogleChrome/accessibility-developer-tools
aXe: http://www.deque.com/products/axe/

The audit should run on the e2e examples.

[sendilkumarn]

We can use Protractor-accessibility-plugin ?

adding this to protractor.conf.js

[sendilkumarn]

if this is okay. I will do a PR for this :) @jelbourn

[devversion]

I think that makes sense. I'm not in position to confirm this, but I think a PR is always welcome, if it's not too much effort for you.

btw: I'm working a PR which uses Axe Core. So our work will not interfere :)

[sendilkumarn]

Okay then 👍

[jelbourn]

In the long term we probably just want to go with one a11y assertion system, but it could be good to compare them initially.

[devversion]

@jelbourn I think both a11y assertion systems are powerful.

When I see the simplicity of the protractor plugin, I think this one is way easier to handle.

I already got Axe Core running on the e2e app as well.
Those A11Y checks can be triggered / started manually by adding a line to the given jasmine spec.

To get Axe Core running, we have to

• Copy it to the vendors for e2e and load it in the e2e
  / Or inject it manually on the given spec (recommended)
• Manually log our violations to the console or file (really difficult to show all necessary information)

Maybe @marcysutton can help in this case, and explain whats different about axe :)

[marcysutton]

The Protractor plugin will only run the A11y checks once at the end of a test and it won't let you specify an HTML fragment to test against (it tests the whole page). aXe, on the other hand, will allow you to run checks whenever the UI has changed within a set of tests and even test a DOM subtree.

The Chrome A11y Developer tools are also a smaller, noisier ruleset than aXe, and Tenon requires an API key and paid account.

Here are a few questions you should consider:

• What should be done with the log output? (We could easily write a utility for this)
• How will developers respond to accessibility issues? (for example, would developers run the tests and look at the debugger?)
• Would the issues go into an issue tracker?
• If you are analyzing whole pages, how do you plan to de-dupe the issues that are common from test-to-test? (None of the mentioned tools do this, but it's worth thinking about.)

[jelbourn]

I'd actually like to have tests fail when the a11y audit encounters an issue.