Performance comparison of JAX-RS 2.0 implementations

I was looking for a short comparison of JAX-RS 2.0 implementations and I could not find one that compared their response times, so I had to do it myself.

I'm sure some will argue that speed is not the most important characteristic of a REST framework, and things like stability, maturity, advanced features matter more. Well, they may or may not be right, but in any case performance is an important aspect.

Tested frameworks

While surely I missed someone's pet JAX-RS framework, I think I have selected the three mainstream ones:

• CXF - I use it for several years, I have always been a happy user
• Jersey - The reference implementation from Sun/Oracle
• Resteasy - The JBoss implementation of JAX-RS

Test method

Project layout

• I have implemented a single Hello service annotated with JAX-RS annotations, this is in a separate maven module and used by framework-specific webapps.
• All web modules are configured with Jackson 2
• For simplicity, the services are registered in spring, however Jersey does not allow it the way Resteasy and CXF do
• All tests are performed using Jetty 9.2.2.
• For warm-up, I gave each test 1M requests on a single thread.
• The test tool was the Apache httpd benchmark utility 'ab'. Each result was performed separately with 100.000 requests and 1,2,4,8,16 concurrent threads.
• Source code of the tests available on github.
• All tests performed on the same laptop. Fedora 20, Oracle Java 1.7.0_45, AMD E2-1800 wreck

Results

Comments

I was kind of surpried to see how much better Resteasy performed and looked under the hoods of the frameworks to figure out what they do when I hit the URL. Well, after a few hours of digging I concluded that most of it is the abstraction layer. CXF is not only for REST but also for JAX-WS, it can consume messages with JMS and so on, and that abstraction layer takes it.

I have also implemented a very thin experimental JAX-RS 2.0 implementation to see how that works and with the same tests it outperformed the mainstream frameworks with roughly 2000 request/second. I would not dare to replace CXF with that :-D but kind of interesting to think about it.

Hello Babel!

Lucas van Valckenborch's painting
source: commons.wikipedia.org

Even though the new Java 8 is out with long expected features like lambda expressions (closures, for simplicity), millions of software developers are looking for a better java.

There are tons of different reasons why you may consider switching to another language, performance is just one of those. I was interested in a very basic aspect of performance: how quickly is the code compiled from the source code able to create a string.

The tested languages are:

• Java 1.7
• Scala 2.10.0
• Kotlin 0.7.271
• Groovy 2.3.0
• Clojure 1.3.10

I am really missing ceylon from the list, but it does not have a maven plugin, it is not even published to maven central or any public repositories, so I skipped it.

And the results are...


Well, the single-threaded performance and simple String handling may not be a good reason to switch at the moment. As you see, java far outperformed the other languages. If you look into the bytecode generated by Scala, Groovy and Clojure, the reason is obvious. It does so many things, that it just can't perform quick.
While Kotlin  performed only about half as quick as the code compiled from java, the problem is a bit harder to spot. So let me highlight the problem...

       0: new           #19                 // class java/lang/StringBuilder
       3: dup          
       4: invokespecial #23                 // Method java/lang/StringBuilder."<init>":()V
       7: ldc           #25                 // String Hello
       9: invokevirtual #29                 // Method java/lang/StringBuilder.append:(Ljava/lang/Object;)Ljava/lang/StringBuilder;
      12: aload_1      
      13: invokevirtual #29                 // Method java/lang/StringBuilder.append:(Ljava/lang/Object;)Ljava/lang/StringBuilder;
      16: ldc           #31                 // String !
      18: invokevirtual #29                 // Method java/lang/StringBuilder.append:(Ljava/lang/Object;)Ljava/lang/StringBuilder;
      21: invokevirtual #35                 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;

The problem is that Kotlin compiler generated code to append method accepting Object arguments, while it is known that the argument is going to be String. Should be an easy fix, I registered a bug in Kotlin's bugtracker.
Update: I played a little with it and found where the code is generated in the compiler. The compiler with my patch generated line by line the same code as the java compiler, and therefore it performed the same.

The truth is, even javac generated suboptimal code, it could be beaten. And next time I will give it a try.


Test environment:

• AMD E2-1800
• Oracle java 1.7.0_45
• Fedora linux

Test code on github.

CloudStack UI speed

I do some CloudStack development in my free time and the user interface is my favourite part from CloudStack. It looks nice and practical when in use and it is very clean in the inside, something to learn from. It is a Single-page web-application, so you will not have to wait for full page reloads.

There is a drawback though. When you load the user interface, all the templates and javascripts are loaded. At the moment (cloudstack-4.5.0-SNAPSHOT) this is 5.4 MB, quite a lot for a login page...

Solution 1: Dynamic compression

John Kinsella wrote an excelent blog post on how to make the page loading faster by configuring Apache httpd to do so. The drawback of this solution is that it will try to compress all the 5.4 MB data when you are downloading it.

Solution 2: Static compression

Well, dynamic compression is usually good, but why do you want to compress that big lot of javascript and css again and again? You can compress those files at packaging and serve the compressed version if the browser accepts it. This is a win-win situation because no CPU-time is wasted while network bandwidth is saved. This is what my patch is doing.

I wanted to show you some results...

Without compression: 5.4 MB

With static gzip compression: 1.2 MB

4.4 MB saved each time you go to your cloud, and more importantly a few seconds from your users lives. I hope you like it :-) But I have to admit there is an obvious drawback: static compression does not compress dynamic content. This is not only true the ajax interactions, but also for the index html page dynamically generated by some JSP files, it weights about 200 KB.

Right solution: combine dynamic and static

I think the combination of static and dynamic compression is the perfect solution for the size problem. No more wasting CPU-time on static compression, no more wasting network on uncompressed dynamic content. Now you have both.