Once again and probably for the last time, I’ve updated my .NET Struct Performance article with results for the latest batch of compilers and runtimes. Microsoft’s and Mono’s current CLRs are unchanged as expected, and so is the excellent MinGW gcc. Visual C++ 2013 gave a nasty shock with massive optimizer failures for user-defined types in both 32- and 64-bit mode. Oracle’s JDK 7u51 produced a smaller unpleasant surprise with an optimizer regression for primitive values that must have happened at some point after 7u13. Sadly, thus are the dangers of sensitive microbenchmarks. These unpredictable changes are one reason why I think it’s time to put my little test to bed. The other is my plan to finally upgrade the computer I had been testing on, so results would no longer be comparable anyway.
But for now, let’s look at what has become the most interesting part of a benchmark originally designed for the .NET Framework: the JavaScript results. All tested browsers (Chrome, Firefox, Internet Explorer) were once again faster in almost every test. Save for one small IE regression concerning primitive values, all current browsers now perform well within the ballpark of the Mono runtime. Chrome and Firefox are within 50% of equivalent tests for user-defined types, and even IE11 matches at least the slowest Mono result. The following chart, roughly organized by release years, illustrates the massive progress JavaScript has made towards other languages.
Click the image for its original size. As you see, only good C++ compilers (i.e. gcc in this test) are still significantly faster than JavaScript. The other languages are getting uncomfortably squeezed into a dwindling space between this universally available runtime and the domain of native code experts.
Firefox 27 delivered an especially remarkable result for “naked” double values (no user-defined types), taking only 2,270 msec compared to 1,030 for the C++ compilers. I suspect the runtime may have “cheated” by reinterpreting the specified floating-point variables as integers, as permitted by the test semantics. That would be a nice exploit of JavaScript’s flexible type system.
FYI, I tried the benchmarks using the VS14 preview. The C++ compiler seems to be ok now — 32 bit gives slightly slower results than GCC, and 64 bit gives exactly the same results as GCC. The C# numbers are the same.
Excellent, thanks for testing! I’ll add a note to the main article.