CORS: arbitrary blocking of accept header based on length

[pietercolpaert]

When the accept header is larger than 128 bytes, the request will trigger a preflight request that blocks if the accept header is not explicitly set. This length looks quite arbitrary: is there a reason for this? But even when the preflight request happens, probably the accept header should not explicitly be set?

The length breaks use cases where a browser script is used to query different web resources on the fly.

Asking all open web resource servers to set the Access-Control-Allow-Headers: accept in order to circumvent this seems like much to ask for no real added security?

Reproducing in Chromium

// Test 1: works
fetch('https://graph.irail.be/sncb/connections?departureTime=2019-01-27',
   {  
      headers: new Headers({
          accept: "application/ld+json"
      })
   }).then(async (response)=> { console.log(await response.json())});

// Test 2: fails
fetch('https://graph.irail.be/sncb/connections?departureTime=2019-01-27', 
   {
      headers: new Headers({
          accept: "application/trig;q=1.0,application/n-quads;q=0.7,text/turtle;q=0.6,application/n-triples;q=0.3,application/ld+json;q=0.3,text/n3;q=0.2"
      })
   }).then(async (response)=> { console.log(await response.json())});

In other browsers

Mozilla docs have this to say:

Note that certain headers are always allowed: Accept, Accept-Language, Content-Language,
Content-Type (but only with a MIME type of its parsed value (ignoring parameters) of either
application/x-www-form-urlencoded, multipart/form-data, or text/plain). These are called the simple
headers, and you don't need to specify them explicitly.

Src: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers#Directives

The test cases do work in Firefox.

In the whatwg spec

• https://fetch.spec.whatwg.org/#cors-unsafe-request-header-names – Indicates the 1024 size problem
• https://fetch.spec.whatwg.org/#no-cors-safelisted-request-header-name – Indicates that the Accept header is safelisted

Question: is this a chromium specific bug, or is this unclear in the whatwg specification and should this be corrected?

Context

We are working on querying open web resources. Setting a long Accept header is therefore quite normal: we can parse a lot of different formats.

Related issue: comunica/comunica#373

[annevk]

It's a filed bug in Firefox and Safari. MDN will be updated by @sideshowbarker as I understand it. The rationale is that the more attacker controlled bytes there are the easier it is to determine things about the resource, which the attacker may not have access to. It's a recent change as well, which is why you are running into this now.

[pietercolpaert]

Shouldn’t the accept header be exempt from this as it is usually quite large?

[annevk]

I'm not sure why you think it's usually quite large? It's usually */* or some such, definitely not exceeding 1024 bytes. Also, no, as long as the bytes are attacker-controlled they are problematic and shouldn't exceed certain limits.

[pietercolpaert]

@annevk I’m actually not sure why this test-case fails in chromium:

fetch('https://graph.irail.be/sncb/connections?departureTime=2019-01-27', 
   {
      headers: new Headers({
          accept: "application/trig;q=1.0,application/n-quads;q=0.7,text/turtle;q=0.6,application/n-triples;q=0.3,application/ld+json;q=0.3,text/n3;q=0.2"
      })
   }).then(async (response)=> { console.log(await response.json())});

This accept header is less than 1024 bytes. Could this be a bug in chromium rather than an issue with the fetch specification?

[annevk]

Ah sorry, 1024 is the global limit, the per-header limit is 128. https://fetch.spec.whatwg.org/#cors-safelisted-request-header onward details the restrictions.

[pietercolpaert]

Ok, I confirm that with 128 bytes accept header it works, 129 bytes doesn’t work.

fetch('https://graph.irail.be/sncb/connections?departureTime=2019-01-27', 
   {
      headers: new Headers({
          accept: "application/trig;q=1.0,application/n-quads;q=0.7,text/turtle;q=0.6,application/n-triples;q=0.3,application/ld+json,text/n3;q=0.2" //this one is 128 bytes
      })
   }).then(async (response)=> { console.log(await response.json())});

So when the accept header is larger than 128 bytes, a preflight request happens, where the accept header needs to be explicitly mentioned in the access-control-allow-headers?

For open datasets, we should thus start adding to all servers (even when just files) next to an access-control-allow-origin: * header also

1. an access-control-allow-headers: *, and
2. make sure these handle OPTIONS requests?

[RubenVerborgh]

The rationale is that the more attacker controlled bytes there are the easier it is to determine things about the resource

@annevk But is that really so?
At first, I intuitively agreed, because "more space" seems equal to "more space to do bad things". But after further reflection, I'm not so sure.

It begs several questions:

1. Is space indeed an enabler for bad things? (As opposed to, let's say, bad escaping, which might also lead to trouble with smaller payloads.)
2. If it is, how does the mitigation mechanism address protect against large headers?

It seems at the moment the rationale is broken. The current reasoning seems to be:

• A large header section is dangerous.
• So let's require the server to allow specific headers.
  • Server's intended result: the client can send a specific header.
  • Actual result: the client can send an arbitrarily large header section.

So the protection mechanism "sure, I allow header X" does not seem to protect against large headers at all, which I read above is what it is supposed to do (and I doubt that necessity). Why not an "allowed header length" then?

[annevk]

The idea is that if the server consents to CORS it's supposed to mitigate any concerns with requests itself. Until it consents to CORS the browser should do its best to protect the server. I'll grant you that there's a bit of a mismatch due to the granularity CORS provides and I'm not sure we'd design it the same way if we could start with the web from scratch. It might perhaps be worth calling this out more somehow.

(Note that for bad escaping CORS also blocks a fair number of bytes in values to mitigate those concerns somewhat.)

[RubenVerborgh]

@annevk Thanks. Is there any indication that the length restriction (in general, or to the specific values of 128/1024) mitigates any concrete attacks, or is this purely speculative?

Until it consents to CORS the browser should do its best to protect the server.

That is a good principle indeed, but in absence of concrete evidence that length restrictions in HTTP scenarios provide additional protection on top of character restrictions, it is hard to conclude that it contributes to actual protection as opposed to a false feeling of security that involves overhead.

[annevk]

@RubenVerborgh specific values are somewhat speculative, but informed. Some security bugs around this haven't been opened yet, so I'd rather not discuss in too much detail.

[pietercolpaert]

@annevk Would you be open to one of the following suggestions?

Suggestion 1: set the number of bytes to a higher value?

Then at least the basic javascript querying applications using conneg intensively keep working

Suggestion 2: don’t block but alter the request

Don’t block the request, but instead, remove the accept header from the request when the OPTIONS header doesn’t allow the accept header explicitly and send it anyway.

If the server doesn’t support OPTIONS at all, send the request anyway without the headers that didn’t pass the test that triggered the preflight request.

This makes sense as this way servers that don’t support certain headers also don’t need to expose them in their allowed headers in order for clients that do support them to work.

[annevk]

2 seems wrong and very scenario-specific whether that is the right thing to do. 1 could maybe be an option, provided the overall limit doesn't change, but would require some careful analysis and a bunch of work around tests and such.

[pietercolpaert]

To me the currently implemented approach seems wrong as it breaks existing webapps. Suggestion 2 seems to keep your security measures and would not break these existing webapps.

What part of 2 is wrong in your opinion?

[pietercolpaert]

Another reason why the currently accepted approach seems wrong to me:

There are initiatives to enable CORS on public Web resources. See e.g., https://enable-cors.org/. They currently only advocate for Access-Control-Allow-Origin: *. As a result from the update in this spec, they would also start advocating for Access-Control-Allow-Headers: *. But what is safer?

1. a server allowing any page to send any header, even if it didn’t pass the preflight test, or
2. a server only allowing a page to send the headers it specifically supports?

The only way the the second option would be supported would be by accepting Suggestion 2 and don’t block but alter the HTTP requests based on the preflight response.