Performance comparison of JAX-RS 2.0 implementations

I was looking for a short comparison of JAX-RS 2.0 implementations and I could not find one that compared their response times, so I had to do it myself.

I'm sure some will argue that speed is not the most important characteristic of a REST framework, and things like stability, maturity, advanced features matter more. Well, they may or may not be right, but in any case performance is an important aspect.

Tested frameworks

While surely I missed someone's pet JAX-RS framework, I think I have selected the three mainstream ones:

• CXF - I use it for several years, I have always been a happy user
• Jersey - The reference implementation from Sun/Oracle
• Resteasy - The JBoss implementation of JAX-RS

Test method

Project layout

• I have implemented a single Hello service annotated with JAX-RS annotations, this is in a separate maven module and used by framework-specific webapps.
• All web modules are configured with Jackson 2
• For simplicity, the services are registered in spring, however Jersey does not allow it the way Resteasy and CXF do
• All tests are performed using Jetty 9.2.2.
• For warm-up, I gave each test 1M requests on a single thread.
• The test tool was the Apache httpd benchmark utility 'ab'. Each result was performed separately with 100.000 requests and 1,2,4,8,16 concurrent threads.
• Source code of the tests available on github.
• All tests performed on the same laptop. Fedora 20, Oracle Java 1.7.0_45, AMD E2-1800 wreck

Results

Comments

I was kind of surpried to see how much better Resteasy performed and looked under the hoods of the frameworks to figure out what they do when I hit the URL. Well, after a few hours of digging I concluded that most of it is the abstraction layer. CXF is not only for REST but also for JAX-WS, it can consume messages with JMS and so on, and that abstraction layer takes it.

I have also implemented a very thin experimental JAX-RS 2.0 implementation to see how that works and with the same tests it outperformed the mainstream frameworks with roughly 2000 request/second. I would not dare to replace CXF with that :-D but kind of interesting to think about it.

JSON vs XML

A few days ago I asked a few guys at work that if they were given a choice, would they choose JSON or XML as a format for a REST API. Everyone responded in favor of JSON. What is surprising in this is that they were all java developers, not a single javascript guy. Maybe it is too early to say this, but it seems that after the initial success of XML in the previous decade and stagnation in the last few years, the XML as format is slowly getting to decline, while more compact and simple, schemaless formats seem to rise: JSON and YAML.
JSON and YAML implementations are just as simple as their formats, you can choose from very many of them.

Question: How do JSON parsers compare to XML parsers from performance perspective?

I wrote sample inputs in XML and JSON. The same data, basically. For XML, I have made two input files. The first is the 'traditional' XML, no attributes at all. The second uses some attributes. The third is the JSON version. I used very few XML apis, only the ones packaged into JDK, this is a bit unfair because the alternative XML apis may be a little faster than the default. I have made a test a few years age and that showed them not so much different, so I was more interested in JSON. With JSON format, I used 5 different parsers: json.org, jackson, jsonlib, jsonj and gson.



Even the slowest JSON parser is faster than the XML parsers of JDK, also the format is more compact, looks like it is win. Jackson is much faster than the rest. It's website is right, it is very fast.

Test source code: here.