HttpResponseTooShort for first request after a while

[jorian]

I have this strange behaviour with a Discord bot that I'm currently making.

For a cryptocurrency project, I am using rust-jsonrpc as a client to query that cryptocurrency's blockchain. It uses the same RPC mechanism that Bitcoin uses, so I'm using this crate and it has been working fine for quite some time.

Because I have to do many RPCs, I store an instance of a rust-jsonrpc client as global data of the bot. This global data is essentially a Box::pin(async move { Data } ) where I store the client as a member of Data. (the bot is made using poise, and in this setup function is where Data gets initiated)

When I start the bot and keep it running for a while (30+ sec) and get the client from global data and do a getnewaddress RPC, I am getting a HttpResponseTooShort error: error: JsonRPC(Transport(HttpResponseTooShort { actual: 0, needed: 12 })). It happens every time I use this call after at least 30 seconds of idling and I can't figure out why this happens. A second request within 30 seconds works just fine.

I've already checked if the problem is keeping a client around for a while:

    let client = vrsc_rpc::Client::vrsc(true, Auth::ConfigFile).unwrap();

    thread::sleep(Duration::from_secs(50));

    dbg!(client.get_new_address().unwrap());

(I built a wrapper like https://github.com/rust-bitcoin/rust-bitcoincore-rpc for this cryptocurrency project)

This doesn't appear to be a problem. I can't figure out what is, though. My guess would be that I keep a connection open for too long, but I can't figure this out, as it appears to be working in a separate environment.

Any ideas?

[dpc]

Sounds like a connection reuse issue after timeout.

#80 possibly related?

[apoelstra]

I agree, it sounds like we're failing to detect that the socket has died. When we do the POST request we should be detecting errors (on the second write) and retrying.

From http://www.softlab.ntua.gr/facilities/documentation/unix/unix-socket-faq/unix-socket-faq-2.html

"If the peer calls close() or exits...I would expect EPIPE, not on the next call, but the one after."

[jorian]

I'm glad it looks like the issue is known. What would be the timeline for a fix?

[apoelstra]

This week.

[jorian]

What is the status on this?

[apoelstra]

No update. I've had it as an open tab for 8 weeks. It's near the top of my TODO list. Sorry.

[apoelstra]

@jorian opened #84 as a fix.

[jorian]

Thanks! No need to say sorry :D

[jorian]

I checked out the branch that this PR uses and use it in my Cargo.toml:

jsonrpc = {git="https://github.com/apoelstra/rust-jsonrpc", branch = "2023-01--detect-epipe", version = "0.14.0"} 

Unfortunately, I still get the error message: RPC error: transport error: HTTP response too short: length 0, needed 12.

EDIT: I just noticed it got merged to master branch, but it seems to be needing some more investigation.

[apoelstra]

Damn. We haven't merged it yet (but we will, since I've run into the issue that it fixes). Frustrating that it doesn't fix your issue.

Are you able to use tcpdump or something to get more information?

[jorian]

Well I decided to just stop reusing clients for now as there is no requirement for me to do so.

I don't have the skills to use tcpdump to find out more, sorry.