consider ignoring initial BOM from clients

[csmith]

Ergo 2.15 includes this change in irc-go which causes a parser error for any non-ASCII commands.

Some clients unwittingly send a UTF8 byte-order-mark when they open a connection, because that's a default for the language/framework/etc they use.

Technically I think it's a protocol violation, but it's a very easy footgun for client devs, quite hard to diagnose in some cases, and previously "just worked".

It might be nice if ergo stripped an initial BOM so these slightly-wonky clients carry on in blissful ignorance?

As an example, this change was made to a client after updating to ergo 2.15. In C#, System.Text.Encoding.UTF8 adds a BOM to the start of the stream, and you have to use a different magical invocation to avoid that (new UTF8Encoding(false), apparently).

[Mailaender]

I could workaround OpenHV/OpenHV@8a044b8 with meebey/SmartIrc4net@a018b76 which seems to try the encoding with stripped BOM first, fail silenty and then resend UTF-8 without BOM.

[slingamn]

Thanks for the report!

As knolle observed in #ergo, a BOM is invalid as part of an IRC command and will fail on other servers. However, the normal server response would be to discard that one line because its command is unrecognized, rather than disconnecting the client. Ergo being so draconian here was somewhat unintentional: ircmsg returns ErrorLineContainsBadChar, which is normally returned for the forbidden octets CR and NUL, which typically indicate a corrupted client stream that can't be recovered.

I will consider possibilities for handling this more leniently but I would emphasize that the BOM is invalid in IRC at a protocol level, and that the typical IRC server implementation will silently drop the first line, which is also hard to diagnose. All of the linked workarounds seem suboptimal at an implementation level; is it not possible to do what @csmith suggested (new UTF8Encoding(false))?

[slingamn]

Also curious if this is an issue with languages/frameworks other than C#?

[csmith]

Also curious if this is an issue with languages/frameworks other than C#?

I feel like I've hit a similar problem before (and wouldn't have been using C#), but it was long enough ago that I don't remember any details. I did come across a couple of other examples of IRC + BOM issues when googling, and they were both C#. It definitely feels like the most widely-used culprit.

I will consider possibilities for handling this more leniently [...]

I guess another option would be to keep it as a fatal error, but make the message more explicit about the invalid characters. Line contains invalid characters: 0xEFBBBF or similar would make it much easier for a client dev to realise what they're doing wrong.