I hate microbenchmarks, normally, because they're typically used to draw all kinds of conclusions that aren't valid in the real world.
That hasn't changed.
Yesterday I blogged about a small test I made of Orion and Tomcat, and some data that came out of it. The data is interesting, but only to a degree - and I have to admit, I was stupid for putting timing data into the test. The timing data is entirely irrelevant. Nobody really uses HTTP 1.0, and nobody hits an app server like I was hitting it.
The goal was, pure and simple, to reap session identifiers. Past that, the fault list is amazing:
- I ran the containers with identical settings (-Xmx128m -Xms128m -server -verbosegc, for Orion, Tomcat, Resin, and Jetty - and left WebLogic and WebSphere untested). This means that the servers might not be getting reasonable configurations, which affects speed. The timing results were of secondary interest - and aren't valid!
- I ran the test for a small sample (100000 entries).
- Of those 100K entries, the summarization of the session ids was in order BY THREAD, so I don't actually see the variation of session ids as they're generated - only as each individual thread sees them. You could say it breaks down like this:
| Order of Occurrence | Summary Order |
| Thread | Session |
| 1 | AAAAAA
| | 2 | AAAAAB
| | 1 | AAAAAC
| | 2 | AAAAAD
|
|
| Thread | Session |
| 1 | AAAAAA
| | 1 | AAAAAC
| | 2 | AAAAAB
| | 2 | AAAAAD
|
|
This is suboptimal, especially when the variance between session ids is larger.
And yes, I'm planning on doing more analysis and exposition.