ORMBattle.NET

Guys,

Today we've got a new leader It is BLToolkit. Few days ago its author (Igor Tkachev) has contacted us, and we provided him with full access to our source code repository & Google groups. Today I was able to run his performance tests, and actually was QUITE surprised - his results are almost ideal; BLToolkit is nearly as fast as plain SQL!

Of course I immediately studied the tests, and found nothing "criminal" there except the fact he uses special API on "CUD Multiple" tests. On the other hand, I'm not sure if we must reject this: its usage looks very "native" here. Let's listen the opinion of other participants.

BLToolkit API is likely the closest one to ADO.NET API in comparison to other tools we've seen (although may be I'm wrong - I viewed only its part shown in our test): any command it sends is explicit; there is a special API for batching multiple commands together and so on. Its LINQ query translation pipeline is definitely the fastest one among other competitors.

Compare its results to e.g. Subsonic - it provides API of similar level of abstraction, but its numbers are quite different

There are no LINQ tests for it yet, but Igor already works on them. As far as I can understand, finally they must be working.

So... Now you may find that I was right telling you What kind of ORM tool can win in this benchmark.

P.S. Why I'm so glad?
- Hopefully, this finally proves the fact we didn't design all the tests especially for DO4. On contrary, you may find these tests are likely the worst ones for complex ORM tool. I think we'll be able to get quite close to BLToolkit on most of tests (I wrote there is still a plenty of room for optimizations), but we have almost zero chances to e.g. get similar results on materialization. Hopefully this will allow us to return DO4 back to this web site.
- BLToolkit is designed by Russians as well That's a good reason to be proud of
- And finally, I hope other ORM vendors won't fear of being listed here. Yes, you can show average results here, and this is acceptable, if features you pay for (in particular, by performance) are good enough. No one will argue that the fastest car must be the lightest one. The same is true for ORM, and our test just proves this. Earlier it was hard to be a "heavy car" here. Now it's hard to be a "lightweight car" as well

Forgot to publish the main part - preliminary results:

Btw, guys...

Think what Igor did: he simply made a test. No talks, no argues, no discussions, no any crying. We talked about ORMBattle.NET with him at RSDN.ru forums, and there were no any complains.

I think he has perfectly shown the best way to prove that product is mature & fast.

Compare this to what Oren, Frans & Rob were doing. Someones prefer to speak; others - to act.

Compare this to what you have been saying? I read comments you left in massive discussion wtih Oren/Frans on Ayende's blog and you keep saying you will remove any reference to NHibernate as they don't want you using their product name to market your own product. I'm not commenting on or caring about accuracy or real-world relevance etc. I simply see you saying, half a year ago now, you will remove NHibernate references as requested.

What happened to honesty?

First of all, I don't remember anyone who was officially asking me to remove NHibernate references. If so, I'd like to see the link.

Furthermore, we're not going to remove any tests of open source products here. This is explicitly stated in rules, and I consider this is more then honest.

I removed LLBLGen, as it was requested, because it is a commercial product. But the same won't work with NH.

Finally, I recommend you to read e.g. these posts:
- ormbattle.net/index.php/blog/93-great-qu...m-todays-e-mail.html
- Honest comparison made by lead NH developer: ormbattle.net/index.php/blog/105-a-hones...-and-nhibernate.html

Btw, I followed NHibernate-development group discussion related to ORMBattle.NET test suite to at least track the the opinion of authors and the community.

I fully agree that generally any test (if it isn't completely wrong / inverse), must be available. If it is wrong, it will be either criticized or updated. Our test is really designed to show maximal performance difference you may expect, and this is explicitly stated everywhere. And I don't think this is bad.

Moreover, NH team always have a very good option: fix slow materialization pipeline in NH. Having just this done, NH results would look _much_ better. But I feel they aren't going to do this - at least, because this could expose a loyalty to benchmark, which was previously criticized by key people there. As far as I can judge, Oren isn't recognized as a person that easily changes his opinion in discussions (btw, is there any important acknowledgement made by him stating that he was wrong?).

P.S. I'm sorry for such a late reply.