Yes, they are nearly optimal, and there is no any cheating.
Initially there were some mistakes:
- We didn't use adonet.batch_size option. It is set to 25 now.
- We didn't use session.FlushMode = FlushMode.Commit to prevent pre-query dirty checking. So NHibernate was making linear scan of fetched object graph before each query (e.g. in query tests), that was leading to ~ O(N) dependency of query execution time (each subsequent query was fetching one more object, and thus increasing the time of the next query).
- We didn't use transaction.Rollback() (we could use session.FlushMode = FlushMode.Never as well) in materialization and query tests. So NHibernate was making a linear scan of fetched object graph on transaction.Commit() (dirty checking), that was adding a constant time for each fetched object in the end of such tests. Initially materialization ratio was lower by ~ 25% because of this.
This is done now, but I consider this as doubtful. This prevents users from freely modifying some of these objects in the same transaction later (so this limits them in available features), i.e., this seems impractical. - Doubtful: We could use IStatelessSession in CUD tests, but didn't. Usage of regular ISession leads to the same issue on "CUD Multiple" sequence - more objects you add to it, more time it spends on detecting the changes.
Doubtful, because IStatelessSession offers neither automatic dirty checking, nor even identity map. So you can use it in very limited set of cases. I'd like to hear community opinion about this. In fact, this is equal to usage of a specialized API instead of common, which is generally not acceptable @ ORMBattle.NET.
Anyway, now all of these tweaks are implemented. We can assure you initial bugs were not intentional.
Any comments related to doubtful tweaks are welcome.
If you're NH expert, we'd appreciate if you could review NH performance tests code.
P.S. Original discussion is here.
Kind regards,
Alex Yakunin





