epesh
I'm Joseph Ottinger, editor of TheServerSide.com.

Calendar

««Nov 2009»»
SMTWTFS
1234567
891011121314
15161718192021
22232425262728
2930

My Bookmarks

My Top Tags

                                       

Mailing List

My RSS Feeds








Search Box

 

More on the microbenchmark

posted Wednesday, 15 December 2004
I hate microbenchmarks, normally, because they're typically used to draw all kinds of conclusions that aren't valid in the real world.

That hasn't changed.

Yesterday I blogged about a small test I made of Orion and Tomcat, and some data that came out of it. The data is interesting, but only to a degree - and I have to admit, I was stupid for putting timing data into the test. The timing data is entirely irrelevant. Nobody really uses HTTP 1.0, and nobody hits an app server like I was hitting it.

The goal was, pure and simple, to reap session identifiers. Past that, the fault list is amazing:

  1. I ran the containers with identical settings (-Xmx128m -Xms128m -server -verbosegc, for Orion, Tomcat, Resin, and Jetty - and left WebLogic and WebSphere untested). This means that the servers might not be getting reasonable configurations, which affects speed. The timing results were of secondary interest - and aren't valid!
  2. I ran the test for a small sample (100000 entries).
  3. Of those 100K entries, the summarization of the session ids was in order BY THREAD, so I don't actually see the variation of session ids as they're generated - only as each individual thread sees them. You could say it breaks down like this:
    Order of OccurrenceSummary Order
    ThreadSession
    1AAAAAA
    2AAAAAB
    1AAAAAC
    2AAAAAD
    ThreadSession
    1AAAAAA
    1AAAAAC
    2AAAAAB
    2AAAAAD

    This is suboptimal, especially when the variance between session ids is larger.

And yes, I'm planning on doing more analysis and exposition.