[jdom-interest] New benchmark

[Dennis Sosnoski]

As I remember there are some minor quirks that show up in my memory 
measurements, but overall the measurements are very consistent. The way 
that I get this stability is by repeatedly executing a loop that first 
requests garbage collection and then waits for a brief time. This slows 
testing but gives good accuracy with at least the Sun 1.3 Linux JVM 
(much better than just requesting garbage collection once and then 
waiting). You might want to try this approach in your benchmarks, Kumar.

How do I know the results are accurate? I can't be absolutely certain, 
but the fact that the increase in memory usage is consistent for 
consecutive builds of the same document is very encouraging. The times 
where I've seen something that looked like an anomaly turned out to have 
a basis in the code (such as JDOM allocating and holding Lists for 
attribute values the first time you walk the document representation, if 
there are no attribute values initially present).

I don't remember seeing the JDOM memory usage issue that showed up in 
Kumar's test results when I've tried JDOM beta 8, so there may be a 
measurement problem with this.

I've been planning to run the next set of my tests for publication using 
the IBM JVM as well as Sun's, and I'll probably also try both the 1.3.1 
and 1.4 JVMs from Sun. It'll be interesting to see how consistent the 
memory usage is across JVMs. I'd been hoping to see JDOM beta 9 out in 
time for the tests, though - how's progress?

  - Dennis


Bradley S. Huffman wrote:

>Elliotte Rusty Harold writes:
>
>  
>
>>I'm not so sure. I don't trust his technique for measuring memory 
>>usage at all. I've done things like that in the past, and on further 
>>digging I've always found that semantically insignificant changes can 
>>produce wild, several hundred percent swings in memory usage, 
>>sometimes even sending the memory negative. Unfortuantely I don't 
>>have any better way to measure memory, but I really don't trust 
>>numbers based on System.freeMemory().
>>    
>>
>
>One thing interesting is Dennis Sosnoski tests don't show explosive memory
>usage on a large document (he used a document a little over 2M) and his
>tests also use Runtime.freeMemory (IIRC), except for the StringBuffer bug
>that caused the modify test to go outrageous.
>
>But even his tests sometimes show weird results like negative memory. However
>he does warn you in the docs and his reports that getting accurate results
>from a VM is damn hard.
>
>I with you, wish there was a better way :(
>
>Brad
>_______________________________________________
>To control your jdom-interest membership:
>http://lists.denveronline.net/mailman/options/jdom-interest/youraddr@yourhost.com
>
>  
>