TimeoutError: Connection timed out after 120000ms

[oran1248]

Describe the bug
When I run serverless command, after something like ~6 minutes, I get the following error:
TimeoutError: Connection timed out after 120000ms

api-lambda folder size is 189MB
default-lambda folder size is 100MB

To Reproduce
I just run serverless.

Expected behavior
Deployed successfully.

Screenshots

Desktop (please complete the following information):

• OS: Windows
• Version 10

[dphang]

I think this may be due to AWS Lambda upload timeout, your api-lambda folder size seems pretty big so maybe there are too many files to upload, or you may have a slow network.

I don't think you're hitting Lambda 50 MB limit as there should be a different error message.

I believe you should be able to set the AWS_CLIENT_TIMEOUT: serverless/serverless#937 and see if that helps.

[oran1248]

I think this may be due to AWS Lambda upload timeout, your api-lambda folder size seems pretty big so maybe there are too many files to upload, or you may have a slow network.

I don't think you're hitting Lambda 50 MB limit as there should be a different error message.

I believe you should be able to set the AWS_CLIENT_TIMEOUT: serverless/serverless#937 and see if that helps.

@dphang Thanks, I will try it and update. I dont understand why files are so big, my project is a standard project. Why aws has this limitation anyway? facing too many issues for simple deploy :(

[danielcondemarin]

What's the output when you run serverless --debug?

[oran1248]

What's the output when you run serverless --debug?

[danielcondemarin]

What's the output when you run serverless --debug?

You're missing a dash, should be --debug

[oran1248]

What's the output when you run serverless --debug?

You're missing a dash, should be --debug

You are right but it is the same message at the end. the last DEBUG message is:
DEBUG ─ Creating lambda X in the us-east-1 region. and than what in the screenshot above.

[danielcondemarin]

Do you have a .env file at the same directory as serverless.yml?

AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

[oran1248]

Do you have a .env file at the same directory as serverless.yml?

AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

I have already fixed the credentials issue, the deployment process passed that phase. I set the right environment variables. now trying with AWS_CLIENT_TIMEOUT=600000 as @dphang suggested.
Unfortunately because of all the failed deployments, I've got an email from aws saying that: Your AWS account X has exceeded 85% of the usage limit for one or more AWS Free Tier-eligible services for the month of September
😭

[oran1248]

I think this may be due to AWS Lambda upload timeout, your api-lambda folder size seems pretty big so maybe there are too many files to upload, or you may have a slow network.

I don't think you're hitting Lambda 50 MB limit as there should be a different error message.

I believe you should be able to set the AWS_CLIENT_TIMEOUT: serverless/serverless#937 and see if that helps.

Still got the same error... even after running export AWS_CLIENT_TIMEOUT=600000 before serverless

[dphang]

I think this may be due to AWS Lambda upload timeout, your api-lambda folder size seems pretty big so maybe there are too many files to upload, or you may have a slow network.
I don't think you're hitting Lambda 50 MB limit as there should be a different error message.
I believe you should be able to set the AWS_CLIENT_TIMEOUT: serverless/serverless#937 and see if that helps.

Still got the same error... even after running export AWS_CLIENT_TIMEOUT=600000 before serverless

Ah, sorry about that, I've been using regular serverless too much as well so I confused the two, I think AWS_CLIENT_TIMEOUT is just for regular serverless..

Since this is a serverless component we call aws-sdk to create Lambda directly in this component. The code is here:

serverless-next.js/packages/serverless-components/aws-lambda/utils.js

Line 119 in d0a6300

const res = await lambda.createFunction(params).promise();

maybe you can try to clone and build this repo and update the aws-sdk client timeout in a similar way as serverless/serverless#937? (We should make the change in this component, but it would need a PR).

Besides this, I don't think it should normally be taking more than 120 seconds to upload the Lambda. Do you have a slow network connection? API lambda folder size of 189 MB is probably close to 50 MB zipped, which is 400 Mbits, to upload in 120 seconds you need at least 3.33 Mbit/s upload speed. It would be fine in CI/CD, but on your computer, I understand it may not. Alternatively, try to reduce any large dependencies you may have to reduce the API Lambda size.

[oran1248]

I think this may be due to AWS Lambda upload timeout, your api-lambda folder size seems pretty big so maybe there are too many files to upload, or you may have a slow network.
I don't think you're hitting Lambda 50 MB limit as there should be a different error message.
I believe you should be able to set the AWS_CLIENT_TIMEOUT: serverless/serverless#937 and see if that helps.

Still got the same error... even after running export AWS_CLIENT_TIMEOUT=600000 before serverless

Ah, sorry about that, I've been using regular serverless too much as well so I confused the two, I think AWS_CLIENT_TIMEOUT is just for regular serverless..

Since this is a serverless component we call aws-sdk to create Lambda directly in this component. The code is here:

serverless-next.js/packages/serverless-components/aws-lambda/utils.js

Line 119 in d0a6300

const res = await lambda.createFunction(params).promise();

maybe you can try to clone and build this repo and update the aws-sdk client timeout in a similar way as serverless/serverless#937? (We should make the change in this component, but it would need a PR).
Besides this, I don't think it should normally be taking more than 120 seconds to upload the Lambda. Do you have a slow network connection? API lambda folder size of 189 MB is probably close to 50 MB zipped, which is 400 Mbits, to upload in 120 seconds you need at least 3.33 Mbit/s upload speed. It would be fine in CI/CD, but on your computer, I understand it may not. Alternatively, try to reduce any large dependencies you may have to reduce the API Lambda size.

I have a fast internet connection. I really don't understand, it seems a common use-case to me. Am I the only one that uploads next.js project into aws lambda? Something doesn't make any sense. should I switch to Google Cloud function?
Which service should I deploy to such that it will work?

[dphang]

I see, thanks for clarifying. If you have a fast connection, then I really don't understand why it should take more than 120 seconds either.

Yes, uploading to AWS Lambda is what this component does and is a common use case. What I meant is that I've not seen a case where it took more than 120 seconds to upload to aws-lambda except when it's due to having slow network and trying to upload a large file. So that's why I first thought it was your network or some problem with your API handler's contents or the Lambda ZIP itself...

As for other options to help debug (besides what I've given above):

1. Manually zipping and uploading the API Lambda via S3 and see if that works.
2. Create a script to zip and use aws-sdk to upload the Lambda and set a timeout > 120 sec (if you don't want to build and locally modify this project, you can try that, maybe it's quicker).
3. Does a basic project (e.g create-next-app) deploy fine for you?

In the meantime, I'll see about creating a PR to allow users to set the aws-sdk timeout to higher than default of 120 sec.

Also, Serverless-next.js currently just targets AWS, and I don't have experience with Google Cloud, so unfortunately I can't help much with that...

[oran1248]

I see, thanks for clarifying. If you have a fast connection, then I really don't understand why it should take more than 120 seconds either.

Yes, uploading to AWS Lambda is what this component does and is a common use case. What I meant is that I've not seen a case where it took more than 120 seconds to upload to aws-lambda except when it's due to having slow network and trying to upload a large file. So that's why I first thought it was your network or some problem with your API handler's contents or the Lambda ZIP itself...

As for other options to help debug (besides what I've given above):

1. Manually zipping and uploading the API Lambda via S3 and see if that works.
2. Create a script to zip and use aws-sdk to upload the Lambda and set a timeout > 120 sec (if you don't want to build and locally modify this project, you can try that, maybe it's quicker).

In the meantime, I'll see about creating a PR to allow users to set the aws-sdk timeout to higher than default of 120 sec.

Also, Serverless-next.js currently just targets AWS, and I don't have experience with Google Cloud, so unfortunately I can't help much with that...

Thank you for the help!
Few questions about the options that you've listed above:

1. How it is going to help me? It isn't a full deployment right?
2. I guess I can try it.
3. I tried it and it worked:
   
   For some reason I couldn't set bucketName, this is my serverless.yml:

# serverless.yml

example-app:
  component: "@sls-next/serverless-component@1.15.1"
  input: 
    bucketName: "example-app-static"
    

Also, why there are no any functions listed in the lambda functions section after deployment?

[dphang]

@oran1248 yeah, for (1) that was just to see if your zip and API Lambda works if uploaded another way e.g using S3, maybe aws-sdk which hits the Lambda upload server was having trouble reading it hence timing out after 120 secs.

69 seconds seems ok to me (assuming it had to create a new CloudFront distribution?).

Another small thing I thought of, maybe even if your connection is fast but you are quite far from us-east-1 (North Virginia), that could possibly affect things too, especially when having to upload a large ZIP. You could also try to setup a simple CI workflow e.g GitHub Actions workflow and see if the deployment works from there. The machines run in the US and I believe have at least 1 Gbps connections. I am in Seattle (us west) and have a 100 Mbps/50 Mbps upload connection, so it's not a problem for me either way, though I mostly don't manually deploy (I use GitHub Actions workflows).

Sorry for all the trouble, but at least we now know of some places to improve upon.

PS: Also I noticed you are using 1.15.1, it might be worth upgrading to 1.17 for the latest fixes and features. Though I don't think the Lambda upload code was modified then, so it probably won't fix this issue.

[dphang]

Also, why there are no any functions listed in the lambda functions section after deployment?

The lambdas should all be in us-east-1, check if you are in the right region. This component deploys on Lambda@Edge so it gets replicated across all of CloudFront locations but the base Lambda is created in us-east-1.