Undocumented breaking change

[ykopstein]

Expected behaviour

#2228 extended browserConsoleLogOptions.level to filter messages on the terminal. This changes the behavior of karma so that under certain circumstances messages created with console no longer show up in the terminal without configuration changes. This should be documented.

Actual behaviour

There is no documentation. In my case I spent over 3 hours investigating source code, git issues, versions, etc before I identified the problem was this change.

Environment Details

• Karma version (output of karma --version): 1.5.0

[dignifiedquire]

Log levels keep on giving. I am sorry about that. @wesleycho could you do a small write up about the old vs the new behaviour, that we can put into the changelog?

[wesleycho]

Sure, how should I do so - should I modify the changelog?

[robsimmons]

The new behavior doesn't seem like it could be consistent with the debug info log warn error ordering of console verbosity that the previous ticket claimed. If I include this in a spec:

console.debug("AAA");
console.info("BBB");
console.log("CCC");
console.warn("DDD");
console.error("EEE");

then as of 1.5.0, on the command line I get

DEBUG: 'AAA'
INFO: 'BBB'
WARN: 'DDD'
ERROR: 'EEE'

The string browserConsoleLogOptions only appears under ./node_modules/karma.

[robsimmons]

To be clear, "the ordering that the previous ticket claimed" refers to @dignifiedquire's comment on December 7 to #2228.

[s100]

Yes, it seems that in practice the order of decreasing verbosity/increasing severity is now log, debug, info, warn, error. Since the default value of browserConsoleLogOptions.level is "debug", this means that anybody not using this option will no longer see console.log entries at all.

A workaround is to set browserConsoleLogOptions.level to "log".

[mgol]

Chrome has 4 levels of filtering output in a console: Verbose, Info, Warnings & Errors. Choosing Errors just enables console.error, choosing Warnings enables console.error & console.warn, choosing Info enables everything except console.debug and choosing Verbose enables anything.

This means while the order between LOG & INFO is not clear, they both should definitely be closer to ERROR than DEBUG and now LOG is treated as the most verbose option.

I'd advocate for the order:

exports.LOG_DISABLE
exports.LOG_ERROR
exports.LOG_WARN
exports.LOG_INFO
exports.LOG_LOG
exports.LOG_DEBUG

Also, currently (in 1.5.0) even setting logLevel in the Karma config doesn't show console.logs so we have 2 bugs here...

[robsimmons]

This doc (and I've found at least one other) says that Log + Debug are exact aliases, so contrary to what the previous issue said (and what I was kind of advocating for above), putting Info between Log and Debug probably isn't justifable. So either log > debug, log = debug, or log < debug is justifiable, but the issue at present is that the behavior introduced in 1.4.1 -> 1.5.0 wasn't backward-compatible.

[mgol]

@robsimmons What I proposed fits passes your criteria regarding the desired order. If we're talking compatibility breaks, though, previously LOG_INFO printed console.logs and now it doesn't so to be backwards compatible the order would have to be:

exports.LOG_DISABLE
exports.LOG_ERROR
exports.LOG_WARN
exports.LOG_LOG
exports.LOG_INFO
exports.LOG_DEBUG

[dignifiedquire]

So if I understand it correctly, if we move to the last option @mgol proposed brings the default behaviour back to what it was before, while matching the general expectations for the various values that can be set. Could I get some confirmation for that?

[robsimmons]

@dignifiedquire I believe that's mostly right. The sadness is there has to be a choice between backwards compatibility and complying with the spec that Console.log === Console.debug.

There's a separate and not-cleanly-answerable question of whether "general expectations" match the spec. I suspect the ordering disable / error / warn / log / info / debug may be both "incorrect" and "generally expected."

[ykopstein]

@robsimmons @dignifiedquire I first encountered this issue b/c I was working w/ the expectation that log != debug. console.log is used as the default "write output the screen" in most tutorials I've seen. It's even used in the official jasmine docs. According to the spec, most of these cases should be using console.info but I have never seen it used that way.