XMLHttpRequest fails on Internet Explorer 11 with "Access is denied"

Steps to reproduce:

1. Open Internet Explorer 11
2. Access "google.com"
3. Browser should redirect to "https://www.google.com"
4. Load Developer Console
5. Enter this line into the console
1. new XMLHttpRequest().open("GET", "http://www.google.com", true)
6. Console returns "Access is denied."
7. Enter this line into the console
1. new XMLHttpRequest().open("GET", "https://www.google.com", true)
8. Console returns "undefined"

When I first encountered the problem, what I found from here was this

var xhr = new XMLHttpRequest();
xhr.open('GET', 'http://localhost/', true); // This line will trigger an error
xhr.send();

 
What I didn't notice (until much later, now) on the very same page was this

In addition to the trusted site requirement I found that the problem was not fixed until I used the same protocol for the request as my origin, e.g. my test site was hosted on a https but failed with any destination using http (without the s).
This only applies to IE, Chrome just politely logs a warning in the debug console and doesn't fail.

Which led me to finding this which mentioned

Requests must be targeted to the same scheme as the hosting page
This restriction means that if your AJAX page is at http://example.com, then your target URL must also begin with HTTP. Similarly, if your AJAX page is at https://example.com, then your target URL must also begin with HTTPS.

This was a pain in my butt for the past few weeks now. It didn't have anything to do with the hardened workstation, whitelisting of URLs, or firewall configuration. The "Acces is denied" could have been a little bit more helpful with clues. It wouldn't show up in development, until you start deploying your codes into staging or production environments that stuff like SSL starts getting in the way with this kind of issues.

I'd been swamped with working on something massive for the past half a year. It hasn't been easy at all with the long drawn marathon of development work that I've been involved in. But I still think that computers are easier to understand than humans.

MySQL intermittent disconnects

Something happened. I'm not sure what. I didn't think much of it in the beginning, and for the previous few days, assumed that the exceptions being thrown out were because of bug in my codes. I was wrong.

The main problem revealed itself as a message of "SQL Error: 0, SQLState: 08S01- SqlExceptionHelper:144 - Communications link failure", which doesn't really say much.

Upon closer inspection, I realised that, even without the application running, my connection to the database over TOAD was failing. It was failing especially when I mouseover a specific table. Bad news. It was useless to run mysqlcheck because the connection would kill itself the same.

Extra parameters for autoReconnect didn't work too. Obviously, because the table was corrupt. There was no easy way to recovering it. Fortunately, as it was an ARCHIVE table, I made the decision to drop the table entirely, and recreate it anew. And that put me back on track... to fixing other bugs.

Transforming a Wikidump archive into usable content

It had been pretty peaceful/brain-wrecking for the past few months as I was working on a portion of my project that will provide the foundation content for our platform. It is essentially the meat to our bones. The Wikipedia foundation/community had basically allow anyone to download all of their archived content. When I did manage to finish downloading one of their fortnightly .bz2 files, it was only just the beginning. From there, I learnt that I had to somehow extract the data out of the massive XML file. Finding the mwdumper.jar helped, but I encountered a bug that made me re-run the entire archive more than 20 times, with each time possibly taking hours. Fixing the bug with a quick hack in one of the class files allowed me to proceed.

Now that the MySQL database had been populated, next was to figure out how to map each article into our database. Using a combination of bliki and JSoup, I was able to parse the objects into usable content. The multitudes of date formats, "magic words", and infobox templates did throw me off a little, but the data transformation program was finally working. Now I just needed to transfer the wikidump over, from database to database.

The main project was setup to receive the data at a unique URL. A secret key was required in order for the server to accept and process the submitted data. While testing during development was done locally, this was being done over HTTPS for the live platform.

This is where it got slightly tricky. Although I eventually felt that this was almost a non-issue, it might save somebody a bit of time if I could offer a documentation of my experience. My exporter/importer (I called it "transporter", duh) was a desktop client, using RestTemplate, (for now) running off my workstation IDE. A trial run had the error coming up as "PKIX path building failed" and a search turned up this, this and this. Everybody was basically saying that adding the certificate into the JRE cert store was the best course of action.

Since my project was tapping on the service provided by the generous Let's Encrypt Certificate Authority, instead of resorting to a self-signed certificate, I had a cert signed by an actual CA. But because Let's Encrypt is not recognised by Java as a trusted CA by default, we have to add it in ourselves. While the above links suggested using the CLI to add their cert, I'd personally find it a lot more convenient by using KeyStore Explorer.

First, configure its cacert store to the correct JRE location. It defaults to your non-JDK JRE, so you should navigate into the JDK_HOME/jre/lib/security path to get to your correct cacerts file.
File menu > Tools > Preferences > Authority Certificates > CA Certificates KeyStore > Browse

Once you have the cacerts file in your KeyStore Explorer view, import the Let’s Encrypt Authority X3 cert you've downloaded into the CA keystore.
File menu > Tools > Import Trusted Certificate

You may be prompted to view and verify that the information in the cert is correct first. Click okay with the suggested alias after that to complete the process.

Oh, and don't forget to amend your RestTemplate constructor:
RestTemplate restTemplate = new RestTemplate(new HttpComponentsClientHttpRequestFactory());

Because otherwise, it defaults to using the SimpleClientHttpRequestFactory that does not support SSL.

Have fun!