Support for interim responses (1xx)

[Acconut]

A server can generate one final response and multiple interim responses for a single request. From RFC 9110:

A single request can have multiple associated responses: zero or more "interim" (non-final) responses with status codes in the "informational" (1xx) range, followed by exactly one "final" response with a status code in one of the other ranges

Some informational status codes, like 100 Continue, are typically handled by the HTTP implementation themselves, but other interim responses are useful for the applications. For example, a server may want to generate an 103 Early Hint interim respone to allow the client to preload resources. Alternatively, a server may want to repeatedly generate interim responses for a long-running request to update the client on the processing progress. The client, on the other hand, may be interested in consuming those interim responses.

As far as I understand - and please correct me here if I am wrong - the interface currently does not expose capabilities for clients to receive or for server to generate interim responses. Would there be interest in adding such features?

[lukewagner]

Great question! Does anyone have any links to good examples of how this is exposed in any standard library HTTP interfaces?

[Acconut]

Looking at other API is a good idea! I have worked with interim responses in Go and Node.js, so I can share their approaches.

Go

When handling incoming request on the server-side, the handler receives a ResponseWriter value, whose WriteHeader function can be called multiple times to emit an interim (1xx) response and once for the final response (2xx-5xx):

mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
  w.Header().Set("My-Interim-Header", "hello")
  w.WriteHeader(105) // interim response

  w.Header().Del("My-Interim-Header")
  w.Header().Set("My-Final-Header", "hello")
  w.WriteHeader(200) // final response
})

Retrieving interim responses as a client is a bit more tedious. You have to attach a client tracer to the outgoing HTTP request. This tracer allows you to define a callback that will be invoked for every received interim response:

ctx := context.TODO()
ctx = httptrace.WithClientTrace(ctx, &httptrace.ClientTrace{
	Got1xxResponse: func(code int, header textproto.MIMEHeader) error {
		fmt.Printf("Got %d response\n", code)
		return nil
	},
})

req, _ := http.NewRequestWithContext(ctx, "GET", "https://example.com", nil)
res, _ := http.DefaultClient.Do(req)

Node.js

For clients in Node.js, receiving interim responses is as easy as listening for the information event on the request object:

const req = http.request(options);
req.end();

req.on('information', (info) => {
  console.log(`Got information prior to main response: ${info.statusCode}`);
});

For servers in Node.js, the support for generic 1xx responses is not great. Node.js offers dedicated methods for sending specific 1xx resposnes:

• writeContinue for 100 Continue
• writeProcessing for 102 Processing
• writeEarlyHint for 103 Early Hint

Other than that, there are no methods for sending generic 1xx responses with custom headers.

That being said, the approaches from Go and Node.js are similar:

• clients use callbacks/events to receive 1xx responses
• server call a method multiple times to send 1xx responses

Do you think these concepts are transferable to wasi-http?

[lukewagner]

Thanks so much for the detailed examples in 2 languages; that helps create a picture of what we'd need at the WASI level. I think one thing I was wondering about that your examples help to explain is how to fallback gracefully when the receiver of a response doesn't know or care about interim responses: it sounds like you just get the non-interim response and body-stream like normal and silently ignore all the interim responses.

Just to confirm: is it the case that interim responses can only be received before a single final non-interim (non-1xx) response (followed by the body stream)? If so, that suggests to me that (perhaps in a 0.3 release, when we're making breaking changes to wasi-http anyways) that handle returns a resource type that represents "the overall sequence of responses to a single request" from which you can call different methods to either (1) stream the N interim responses before 1 final non-interim response or (2) skip directly to the final non-interm response. Thus, when the Go/Node.js "information" callback is installed, the impl would use (1), otherwise, it would use (2).

[acfoltzer]

Just to confirm: is it the case that interim responses can only be received before a single final non-interim (non-1xx) response (followed by the body stream)?

Yep! (cite)

handle returns a resource type

Do you mean that handle would take a new resource type as an argument? This would be in line with other proposals floating around different ecosystems (here's one from the hyper world that uses a method on the Request argument rather than an entire new argument to avoid a breaking change). If the new resource was only accessible after handle returns (along with the final Response headers) that would be too late for many 1xx use cases.

[lukewagner]

Do you mean that handle would take a new resource type as an argument? [...] If the new resource was only accessible after handle returns (along with the final Response headers) that would be too late for many 1xx use cases.

I was thinking that (again, in a 0.3 breaking-change timeframe) the resource type returned by handle would not represent "a response" (which I agree would arrive "too late"), but, rather, would represent "whatever future response(s) I get from this request" and thus you would get it before the first information response (maybe immediately, or maybe after some basic connection setup? I dunno) and given this resource, you could use it to either ask for informational responses or skip them.

Thinking through direct component-to-component composition scenarios, it seems like the ideal here is that we're not bifurcating the types (or handle function or its interface name) so that if I have a chain of 3 components and the component in the middle has no knowledge or care of informational responses--it's just forwarding things after poking at a header, let's say--that it all composes and we don't "shear off" the information responses. There are probably multiple ways to achieve this, though and what I'm saying is just a rough sketch.

One requirement that standard libraries have that we don't (yet) is that they have to maintain backwards compatibility with existing users whereas we only need to be able to implement these library interfaces (with library impl code that can know the full 0.3 interface), which is nice.

[acfoltzer]

Ah, I see, I was still thinking about it in the framework of the current incoming- and outgoing-handler split. That makes sense.

[dicej]

I was thinking that (again, in a 0.3 breaking-change timeframe) the resource type returned by handle would not represent "a response" (which I agree would arrive "too late"), but, rather, would represent "whatever future response(s) I get from this request" and thus you would get it before the first information response (maybe immediately, or maybe after some basic connection setup? I dunno) and given this resource, you could use it to either ask for informational responses or skip them.

@lukewagner would you mind sketching out what you think the API would look like here? We're also interested in supporting interim responses in Spin, and this seems like a good time to update the wasi:http@0.3.0-draft interfaces to support them.

Naively, I could imagine something like handle: func(request: request) -> stream<response>, but I'm guessing you have something different in mind.

[lukewagner]

Taking advantage of the restricted response structure I tried to summarize my understanding of above and trying to optimize for the common case where there are no informational responses (or there are, but they're ignored), I was wondering if we could just get away with simply adding one extra method to response (and renaming response to responses), e.g.:

interface wasi:http/types {
  resource responses {
    informational: func() -> stream<informational-response>;
    final-status-code: func() -> status-code;
    body: func() -> option<body>;
  }
}
interface wasi:http/handler {
  handle: func(r: request) -> result<responses, error-code>;
}

noting that a responses can be returned by handle before final-status-code has been received because it's an async function (and thus not marked with nonblocking like most other getters/setters would be).

The question with this interface is: what happens if the client calls final-status-code or body before calling informational or if informational is never called. If stream<informational-response> could be very large (like body can be), then we would need to worry about not forcing implementations to hold onto all the informational responses in memory, so we might say that calling final-status-code and body have the side-effect of saying "I don't care about informational, so skip the rest and have informational henceforth return an empty stream". That's a valid option, but I wonder if it will lead to surprising behavior. Instead, since it seems like informational responses are small, like headers, we could say that informational can be called at any time and thus implementations have to hold onto it, like headers.

This is just a idea though; WDYT?

[Acconut]

The question with this interface is: what happens if the client calls final-status-code or body before calling informational or if informational is never called.

Is it possible to ignore received informational responses until informational is called? If the user is not interested in informational responses and informational is not called, they will just be dropped without buffering. If informational is called, it subscribes to all future received informational responses, but does not return previously received ones.

Would that be possible?

[lukewagner]

@Acconut The challenge I believe is avoiding race conditions where informational responses are lost because informational was not called soon enough but for no obvious fault of the developer. This seems especially possible when chaining multiple components together.

But thinking more about how chaining is supposed to work in practice makes me think that my previous sketch is overly simplistic and that probably we want something more like what @dicej wrote. In particular, combining all the responses into 1 exclusively-owned responses resource would mean that you couldn't have a pipeline of proxies where first the informational responses propagate down the pipeline followed by the final response propagating down the same pipeline. A stream<response> (as the result of handle) like @dicej suggested captures this much better; my only complaint was that it loses the meaningful distinction between information responses and the final response. But maybe this is fine?

FWIW, we could tighten up stream<response>. Assuming for the moment we had stream<T,U> to describe "a sequence of T values followed by a final U value" (which we could approximate in the short-term with resource types), it seems like the type of handle would be:

handle: func(r: request) -> result<stream<interim-response, response>, error-code>;

where response is defined as currently in 0.3-draft. Given this, a well-behaved chainable proxy that only wanted to poke at final-response headers (i.e., the common case) would forward all the interim-responses, receive the final response, poke at response.headers, and then forward this final response. And one concrete benefit of the separation of the interim-responses from the final response is that the former could be trivially zero-copy forwarded by whatever stream.forward we eventually add since the proxy won't have to manually inspect each one to see whether it was interim or not.

[dicej]

FWIW: I added experimental, WASIp2-compatible informational response support to a fork of Spin (which also required forking Hyper, since upstream doesn't support 1xx responses yet): https://github.com/dicej/spin-informational-demo

It uses this WIT interface:

package spin:http@3.0.0;

interface http {
  use wasi:http/types@0.2.0.{response-outparam, headers, status-code};

  /// Send an HTTP 1xx response.
  ///
  /// Unlike `response-outparam.set`, this does not consume the `response-outparam`, allowing the
  /// guest to send an arbitrary number of informational responses before sending the final response
  /// using `response-outparam.set`.
  send-informational: func(out: borrow<response-outparam>, status: status-code, headers: headers);
}

[dicej]

This addresses the outbound response half of 1xx support: #139, based on the prototype I mentioned above.

[Acconut]

FWIW, we could tighten up stream<response>. Assuming for the moment we had stream<T,U> to describe "a sequence of T values followed by a final U value" (which we could approximate in the short-term with resource types), it seems like the type of handle would be:

handle: func(r: request) -> result<stream<interim-response, response>, error-code>;

This looks like a great proposal as it uses the type system to disallow semantically incorrect operation (e.g. sending an informational response after a final one). One thing to keep in mind is that informational responses are still a niche use case, so one might want to optimize their API for the typical use case of sending one final response while also allowing advanced use cases with informational responses. Would your proposed API be a hindrance for the typical use case?

[lukewagner]

@Acconut I agree that informational responses will be rare and thus it's worth trying to avoid hurting perf in the common case; that was the origin of my original suggestion. The tricky thing seems to be supporting the incremental streaming of interim responses in a multi-proxy/component chain while allowing individual components to ignore the existing of interim responses. But I won't claim to have exhausted the design space here, so other ideas welcome.

One saving grace may be that most folks are going to be using a standard HTTP library implemented on top of wasi-http, and this library's implementation can hide interim responses (by doing the automatic forwarding I mentioned above) unless the client code indicates interest (by registering the interim-response callbacks, as you showed above). My assumption is that the (to be added) zero-copy stream.forward built-in I mentioned above should keep the overhead for the common case very low, but I guess we'll need to validate that assumption.

[dicej]

Another possible approach:

resource response {
  ...
  /// Create a new informational response; `status` must be in the range [100-199]
  new-informational: static func(status: status-code, headers: headers, next: future<response>) -> result<response>;
  /// Returns the next response if this is an informational response; otherwise return an error
  next() -> result<future<response>>;
  ...
}