Query Parameters with space.

[dhcrees]

Not sure if this is the right place to ask this question, but I can’t seem to find much help on the subject. We’ve been using the container now for months and everything works great. We’ve used this for a simple API which only contains a single Query Parameter. We’ve noticed that when we put a space into the parameter then the API Gateway returns an Internal Server Error immediately. So for example

/search?identifier=XXXXX (works fine)
/search?identifier=XXX XXX (internal server error)

If I run the API from the AWS Console test facility and replace the space with %20 then it works. So XXX%20XXX, but if I test the API from Postman, then with the space or the %20 I get the internal server error.

From looking at the logs, it appears that the request is being sent to the Lambda. At first we assumed it was the gateway returning the error straight away, but it looks like it might be the way the Lambda is handling the space in the param.

Hope someone can help because we’re stumped. From reading some stuff online, for a non Proxy API the suggestion is to use a message template on the parameters and use the $util.urlEncode(), but I can’t see how to apply this with a proxy.

Is there something we have to put into the Java code to take care of this?

[sapessi]

Hi @dhcrees, just to confirm, you are using the serverless-java-container library right? Are you using Spring, Jersey, or Spark?

I'm just in the process of fixing #53 so I can take a look at this one too. When you run the test call from the API Gateway console with the space, what does the event look like in the logs on the right, could you paste it here?

[dhcrees]

Stefano, Yes, we’re using the Serverless-Java-Container. (Jersey). Its been working great since the first release, but we’ve just noticed this happening (well its the first time we’ve put a space in to be fair). We’ve been able to use the container library with Jetty which allows us to run the framework locally, where we don’t see the error. It only seems to happen when we deploy the lambda to AWS.

…

On 14 Aug 2017, at 20:57, Stefano Buliani ***@***.***> wrote: Hi @dhcrees <https://github.com/dhcrees>, just to confirm, you are using the serverless-java-container library right? I'm just in the process of fixing #53 <#53> so I can take a look at this one too. When you run the test call from the API Gateway console with the space, what does the event look like in the logs on the right, could you paste it here? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEMu32qiq0lM6IednGUayNOHzim6XIpJks5sYKatgaJpZM4O20pQ>.

[sapessi]

Great thanks. Let me try and replicate. I'll update you here.

[dhcrees]

Stefano, Can you confirm if you think the issue is likely to be the framework. I’ve opened a case with AWS earlier because I thought it was the gateway after I saw all the suggestions to use $util.urlEncode(). If it is then I can close the support ticker with AWS, not that people are taking his from different angles. Many thanks.

…

On 14 Aug 2017, at 21:07, Stefano Buliani ***@***.***> wrote: Great thanks. Let me try and replicate. I'll update you here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEMu39ZY1qc6iS_kaD2Jm86YsTQ1EQ8Eks5sYKj2gaJpZM4O20pQ>.

[sapessi]

Hey @dhcrees, I found the issue and fixed it. The issue is that the JerseyRequestReader creates a new URI with the raw values from API Gateway. Any unencoded query string parameter cause it to throw an exception. I have changed it to use Jersey's UriBuilder object that will apply encoding automatically. I've also added some additional logging to make this more obvious and some new unit tests.

I will commit my changes in a few minutes to the servlet-improvements branch. If you'd like to test, once I've committed the changes, you need to:

1. Clone the servlet-improvements branch from GitHub:

$ git clone -b servlet-improvements https://github.com/awslabs/aws-serverless-java-container.git

2. Use maven to install the new version in local:

$ mvn install 

3. Update your pom.xml to use version 0.7-SNAPSHOT

[dhcrees]

Stefano, Many thanks, I will try it when I get to work in the morning. Just to be clear, I do the mvn install inside the clone of serverlet-improvement.? Sorry I normally just let maven handle this stuff :)

…

On 14 Aug 2017, at 21:30, Stefano Buliani ***@***.***> wrote: Hey @dhcrees <https://github.com/dhcrees>, I found the issue and fixed it. The issue is that the JerseyRequestReader creates a new URI with the raw values from API Gateway <https://github.com/awslabs/aws-serverless-java-container/blob/master/aws-serverless-java-container-jersey/src/main/java/com/amazonaws/serverless/proxy/jersey/JerseyAwsProxyRequestReader.java#L85>. Any unencoded query string parameter cause it to throw an exception. I have changed it to use Jersey's UriBuilder object that will apply encoding automatically. I've also added some additional logging to make this more obvious and some new unit tests. I will commit my changes in a few minutes to the servlet-improvements branch. If you'd like to test, once I've committed the changes, you need to: Clone the servlet-improvements branch from GitHub: $ git clone -b servlet-improvements https://github.com/awslabs/aws-serverless-java-container.git Use maven to install the new version in local: $ mvn install Update your pom.xml to use version 0.7-SNAPSHOT — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEMu38csX4YxfEyUSdy3Ll28S8Dn3xcmks5sYK5LgaJpZM4O20pQ>.

[sapessi]

@dhcrees Yes, clone the servlet improvements branch and run mvn install from that folder. This will install 0.7-SNAPSHOT in your local maven cache. Then you'll be able to refer to it from your other projects. Hopefully the official 0.7 will be out before the end of the month.

[dhcrees]

Excellent, once again thanks. Will try it first thing in the morning and get back to you

…

On 14 Aug 2017, at 21:35, Stefano Buliani ***@***.***> wrote: @dhcrees <https://github.com/dhcrees> Yes, clone the servlet improvements branch and run mvn install from that folder. This will install 0.7-SNAPSHOT in your local maven cache. Then you'll be able to refer to it from your other projects. Hopefully the official 0.7 will be out before the end of the month. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEMu35bmPag5mDtZHT_o9S4uzb8zAohQks5sYK-DgaJpZM4O20pQ>.

[dhcrees]

Stefano, I’ve cloned and tested the 0.7-SNAPSHOT into our project and everything is working as intended. Many thanks

…

On 14 Aug 2017, at 21:35, Stefano Buliani ***@***.***> wrote: @dhcrees <https://github.com/dhcrees> Yes, clone the servlet improvements branch and run mvn install from that folder. This will install 0.7-SNAPSHOT in your local maven cache. Then you'll be able to refer to it from your other projects. Hopefully the official 0.7 will be out before the end of the month. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEMu35bmPag5mDtZHT_o9S4uzb8zAohQks5sYK-DgaJpZM4O20pQ>.

[sapessi]

Thanks for testing @dhcrees, I'll close this and merge the changes into master today.