Saturday, August 6, 2016

Websocket + SockJS + Apache as proxy

In a previous post, I mentioned about my journey into Understanding websocket with the Spring framework that I realised turned out more about exploring spring-security. Nevertheless, it was still part of the same journey, since I probably wouldn't have used spring-security if it weren't for its presence as a spring-websocket dependency.

That said, throughout my coding, it was generally uneventful. That is, until I tried deploying the package into the cloud. The WildFly was sitting behind Apache acting as the proxy, and this was where the problem manifested itself. The issue was compounded by the use of HTTPS, but only running on the proxy; with WildFly only listening for HTTP.

It took quite some time for me to figure out a solution on my own. If your setup is using the stack from Bitnami similar to mine then I certainly hope you'll find this post useful.

Assumptions:
  1. Your Bitnami stack is up and running correctly;
  2. WildFly is listening internally on :8080 port;
  3. You have complete access to your WildFly management console;
  4. Your app can be deployed successfully to WildFly;
  5. Both WildFly and Apache are sited on the same machine/instance;
  6. Apache has been configured with a SSL certificate for proper HTTPS operation;
  7. Apache is functioning normally for regular HTTP/HTTPS traffic
  8. Apache error log should indicate an excessive amount of traffic due to the websocket 
  9. Apache access log should indicate HTTP 502 or similar erroneous codes
  10. httpd.conf should already have uncommented the LoadModule lines for mod_proxy and all its modules, especially mod_proxy_wstunnel.so
 Next, add the following lines to your /opt/bitnami/apache2/conf/bitnami/bitnami.conf file:
  <IfModule proxy_wstunnel_module>
  RewriteCond %{HTTP:Upgrade} =websocket [NC]
  RewriteRule ^/(.*)$ ws://localhost:8080/$1 [P,L]
  </IfModule>
 Do the same for both HTTP (port :80) and HTTPS (port :443) traffic. And then restart Apache.

It took me a while to realise, after enabling the Apache debug log level, that
  1. There is never any wss:// used by WildFly because I'm do not have the HTTPS listener setup; just the vanilla HTTP;
  2. Redirecting to ws://www.mysite.com/$1 would not work either, because that's still throwing the same request to Apache;
  3. Redirecting to ws://localhost/$1 would not work, because that's equally requesting to Apache itself;
If you have SSL for both Apache and WildFly, I guess it'd be possible to utilise SSLProxyEngine that would probably circumvent all of the above, although it'd potentially add overheads to a small cloud instance.

Monday, August 1, 2016

FileUploadException: UT000020: Connection terminated as request was larger than 10485760

In the course of our limits testing, it turns out that it wasn't enough to just set the file upload limit in our own application for the CommonsMultipartResolver, using our spring-*context.xml configuration. WildFly had other ideas of its own. It comes out of the box with the default limit of 10485760 bytes.

While I managed to locate a couple of results such as this, this, this and this, which pointed me in the right direction, they were all referencing older versions, namely WildFly 8. And being lazy that I am, I'm not about to poke around the XML making changes manually, much less use the CLI to amend the value. I wanted to make the change via the WildFly Admin Console UI.

Thus I had to make explorations of my own. Based on those clues, I've identified the whereabouts to change said value.

Navigate to the Configuration tab > Subsystems > Web/HTTP - Undertow > HTTP and click View


HTTP Server tab > "default-server" > View

HTTP Listener > Edit

Then edit the "Max post size" to your desired value. Naturally, I'd think that the value should match whatever you've configured in your own application.

And don't forget to restart your WildFly!

Edit: Also, don't forget to tweak your database e.g. MySQL for the max_allowed_packet alongside this setting!

Friday, July 29, 2016

Understanding websocket with the Spring framework (feat. spring-security)

I needed to learn how websockets worked in the Spring framework. I did this by researching how the different parts work together with the resulting package here.

It was a rabbit hole that turned out to be much deeper than I expected. The first step to a problem I had for instantaneous notifications on the client browser, was turning to websockets, since long-polling seemed a little backward at this stage. My first encounter was the pure JSR-356 implementation by Oracle. But I quickly realised that, because I'm already using the Spring framework, there was already spring-websocket. @Endpoint doesn't seem to play nice with Spring, so now I'm using what Spring has: @MessageMapping for receiving messages, @SendToUser for sending messages to a specific user, and @SubscribeMapping for accepting subscriptions from users. The documentation is still rather sparse even now, despite the few years it had already come into being.

Next up was the browser-end. That required a combination of STOMP and SockJS, both of which are already kind of integrated into spring-websocket. But getting the jQuery send/subscribe/callback functions correct still took a bit of time getting used to.

The websocket uses HTTP:Upgrade to convert a HTTP:// into a WS:// request. The problem came when it was time to test this with TLS. My setup involves fronting the Wildfly with Apache, and the HTTPS connection is handled by this proxy. The usual mod_proxy required an additional mod_proxy_wstunnel. While SockJS is handling the switch from HTTPS:// to WSS:// on its own, the proxy introduced a new set of problems. Apache was having difficulty translating the upgrade request. Part of the solution included this. But since I have a hundred and one issues to solve, the fix is still pending.

Another portion of the setup requires spring-security. And while I totally appreciate what resources mkyong has to offer, I'm still trying to wade through the swamplands on my own. You see, instead of tapping into the login module that spring-security had to offer, I decided to stick to my own version that I'd established before this. I realised from this post that you could manually set the authenticated user with
SecurityContextHolder.getContext().setAuthentication(authentication);

and prepare your own list of Granted Authority for it.

A static 403 error page was prepared and added into the spring-security setup. Turned out that I was forgetful enough to add RequestMethod.POST on top of the standard RequestMethod.GET for the page. Took me a while to hunt for this issue. But now, I'm still trying to figure out why is spring-security redirecting an authenticated user to this 403 page due to a POST form submission.

Now that I've gotten the static 403 page displaying properly, instead of the miserable "Request Method 'POST' not supported", I wanted to trace the next stage of the problem. In order to do my detective work, I needed to follow the trail from the output logs, and spring-security doesn't make it any easier. I got the hint eventually, that not only is it needed to add <security:debug /> into the XML file, because I'm using Log4j, I also had to add these lines in:
    <logger name="org.springframework.web.security">
        <level value="debug" />
    </logger>
Notice my mistake? I didn't for a while, wondering why is the log not outputting anything relevant. It ought to have been this instead:
    <logger name="org.springframework.security">
        <level value="debug" />
    </logger>
Great. The console was outputting a whole bunch; too much of a bunch in fact. I had to enable the log file that will be output on to disk and trace from there. Finally getting some where, I identified the offending line:
DEBUG csrf.CsrfFilter - Invalid CSRF token found for...
I was already suspicious about the CSRF portion, but thought it to be a dead-end find. Now I can be even more certain as I investigate further. Returning to the source, I continued reading up the documentation on spring-security, and came across (again) the section for CSRF protection. I noticed a particular section that discusses the use of the Multipart (file upload) content type. This jumped out at me this time round, because I realised that the form I'm encountering an issue with CSRF, involves file uploads, which certainly requires the multipart feature.

A more detailed answer (despite it being over 2 years old) was found on StackOverflow here. And it was pretty straightforward. The documentation under spring-security for adding support for CSRF to multipart forms had missed out one issue. MultipartFilter defaults to "filterMultipartResolver". At this stage, I'd already configured the CommonsMultipartResolver in my spring-mvc-context.xml as an existing bean.

Now, in order to complete the loop, I could have simply named the bean the default "filterMultipartResolver". Of course, in order to prove that the linkage was correct, I did this instead:
<filter>
    <filter-name>springMultipartFilter</filter-name>
    <filter-class>org.springframework.web.multipart.support.MultipartFilter</filter-class>
    <init-param>
        <param-name>multipartResolverBeanName</param-name>
        <param-value>ubiomiMultipartResolver</param-value>
    </init-param>
</filter>
<filter-mapping>
    <filter-name>springMultipartFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
which was then placed before the springSecurityFilterChain lines of configuration in my web.xml file.

I'm happy to report that the CSRF protection remains in place, and the multipart content passes through spring-security properly!

The journey continues.