Hi @s0l0ist , thanks for cutting this PR. This change seems very reasonable to me.

I do have concerns about leaking internals. This is only naming return types that are already public, but we might want to tweak their composition or otherwise shift bounds in ways that break callers. I think it would be best to type erase the returned service stack.

The easiest way to do that would be via a tower::util::BoxService / BoxLayer (or the sync/clone-bounded variants, though I don't think we need them here - but we could certainly add alternate APIs that do include the sync/clone bounds if it would be useful for how the returned struct is used by a runtime).

That adds a small amount of performance overhead due to an extra allocation and layer of dynamic dispatch. But, I think the ergonomics would be much better compared to a more complex, composable builder-style API using generics and sealed inner layer types. Note that we would probably want some sort of into_streaming_streaming_response_inner() that is what into_streaming_response() is currently, so that our run_with_streaming_response can skip the minor type erasure overhead.

How does that sound to you?

It would also be great to get a small example showing usage of this with a non-tokio runtime, if you'd be open to it! That would both let us validate the API, and make it more discoverable for users. I'd probably be ok with this landing without that, though, if you don't have cycles.

@jlizen, I think that's the right direction. Currently busy right now, but can revisit this PR in a couple of weeks from now.

@jlizen - does this API work? I'll add an example or two if I have time, but wanted to make sure this was aligned beforehand.

Hi @jlizen, I think I may have a better approach.

I noticed that Adapter, handles buffered responses by wrapping a Service<Request> and producing a LambdaResponse from anything that implements IntoResponse. Taking inspiration from that, I built a StreamAdapter that wraps a Service<Request, Response = Response<B>> and produces a StreamResponse<BodyStream<B>>.

This lets us remove the BoxService entirely while keeping the same ergonomics. The trade-off is a heap-allocated future per request (same as Adapter), but we avoid exposing BoxService in the public API.

With this change, run_with_streaming_response just mirrors the non-streaming run:

pub async fn run_with_streaming_response<'a, S, B, E>(handler: S) -> Result<(), Error>
where
    S: Service<Request, Response = Response<B>, Error = E>,
    S::Future: Send + 'a,
    B: Body + Unpin + Send + 'static,
    B::Data: Into<Bytes> + Send,
    B::Error: Into<Error> + Send + Debug,
{
    lambda_runtime::run(StreamAdapter::from(handler)).await
}

And I can selectively choose which one I want like this:

/// The main function to run the AWS Lambda
pub async fn async_main(tracer_provider: sdktrace::SdkTracerProvider) -> Result<(), BoxError> {
    let (app, _) = build_service().await;

    // For a buffered response:
    // let handler = lambda_http::Adapter::from(app);
    // For a streamed response:
    let handler = lambda_http::StreamAdapter::from(app);

    let runtime = Runtime::new(handler).layer(
        // Create a tracing span for each Lambda invocation
        OpenTelemetryLayer::new(|| {
            if let Err(err) = tracer_provider.force_flush() {
                eprintln!("Error flushing traces: {err:#?}");
            }
        })
        .with_trigger(OpenTelemetryFaasTrigger::Http),
    );
    runtime.run().await
}

Okay I think I have it:

• Add into_stream_service(...): zero-alloc builder (no boxed future/vtable) that is used by run_with_streaming_response so that it continues to have the same exact behavior.
• Keep StreamAdapter for ergonomic stacking; it still returns a boxed future for API stability.
• Factor common logic into into_stream_response(...) which is used by both StreamAdapter and into_stream_service

I added a parallel test to what Adapter already had - ensuring we can compile-check a BoxService built the same way. Once we’re happy with the shape, I’ll follow up more examples. I've already added the http-axum-streaming example.

Yes I see the CI checks failing - they're unrelated to this PR, but happy to clean them up.

Ah, ok, I didn't actually look at them - probably we got hit by some new clippy/rustfmt lints across toolchain versions. If you have time to clean them up, feel free / much appreciated (separate commit please).

It sounded like you did plan to hoist that test into an example, so I can sit tight on merging since it sounds like a new commit is coming. But just ping me if you want to ship it without another round of changes.

@jlizen , fixed CI here.

And yes, I plan on moving the test into the new example and that should wrap this PR up.

@jlizen - Okay! I think we're ready :)

Added two examples, one that showcases run_with_streaming_response and another that demonstrates configuring a custom runtime with OTeL using StreamAdapter. Both examples type-erase the service into a BoxService before converting it into a streaming service. I didn’t have a good example of type-erasing after conversion, so I kept the compile test since it directly validates Tower::Service.

I've tested manually e2e to verify everything works.

Sorry to miss this today @s0l0ist - near the top of my list for tomorrow. Thanks for the CI fix.

Sorry for the back and forth on examples @s0l0ist - I'm good to merge with that one requested change (remove the vestigial BoxService). Given that users will probably copy-paste these examples, it would be better to show the right general practice.

Sorry for the back and forth on examples @s0l0ist - I'm good to merge with that one requested change (remove the vestigial BoxService). Given that users will probably copy-paste these examples, it would be better to show the right general practice.

No need to apologize! I'm happy to iterate and collaborate on the best approach moving forward.