Performance

For each ORM, we ran these tests several times with the same random seed and chose +/- the median results. The deviations of these results comparing to the most recent results are insignificant. The biggest deviation was measured for the tests "Scheme2-100-roots". 

Database Schemes

1. Scheme1: 250,000 rows/independent-table, 200,000 rows/dependent-table
   • Big data with less relations - not so many joins, no so many additional collection rows to root rows in queries. 
2. Scheme2: 25,000 rows/independent-table, 50,000 rows/dependent-table
   • Less data, many relations - many joins, many additional collection rows to root rows in queries. 
3. Scheme3: 200,000 rows/independent-table, 300,000 rows/dependent-table
   • Big data, many relations... 

Test Types

• The load tests itself has 2x5 iterations, each iteration takes 20 seconds. So, 5 iterations should be performed in 01:40 minutes (100 seconds). 
• We also measure the real test time, that is usually longer, because of multitasking of a single computer that the tests are run on. Many of tasks has to be switched between from one to another, because of limited number of CPU threads. 
• For every test, The first 5 iterations simulates 50 users while the last 5 iterations 100 users. 
• Filtering is done by index range lookup of primary or foreign keys. 
• None of the tests generates an empty query result. 

1. Scheme1-100-roots
   • Generates heavy relation (join) queries for 100 roots upon the Scheme1. Expected about 500 rows. 
2. Scheme1-400-roots
   • Generates heavy relation (join) queries for 400 roots upon the Scheme1. Expected about 1500 rows. 
3. Scheme2-100-roots
   • Generates heavy relation (join) queries for 100 roots upon the Scheme2. Expected about 1200 rows. 
4. Scheme2-400-roots
   • Generates heavy relation (join) queries for 400 roots upon the Scheme2. Expected about 7000 rows. 
5. Scheme3-100-roots
   • Generates heavy relation (join) queries for 100 roots upon the Scheme3. Expected about 900 rows. 
6. Scheme3-400-roots
   • Generates heavy relation (join) queries for 400 roots upon the Scheme3. Expected about 6000 rows. 

ORM Overview

Many developers use Dapper and critic Entity Framework Core for performance. Dapper declares in its Nuget package description the following: "A high performance Micro-ORM..." The only truth is, that it is micro-ORM, because the only thing it does is the mapping of columns from a flat single table SELECT into entity properties of the same name (and you have to define SQL by yourself). So, if it selects only a few data, it is surely faster. 

Entity Framework and D3ORM are also able to select only a few columns in queries... :-D

The developers should think about how to tune or simplify their queries due to performance issues rather, then searching for another ORM. Entity Framework and D3ORM are both high performance tools in this field. 

D3ORM

+

Entity aggregations, light layered architecture of a target project, quick development, support of own or generated entities, stored procedures, JSON serialization, DTOs, asynchronous and anonymous bulk operations. No nuget package dependencies. High performance - among other things, thanks to caching entities once aggregated for going through repeating columns much more faster and still consuming not much RAM... 

-

Learning new technology, poor support over the Internet, yet. Some claims on db schema design (no support of complex primary keys). All the versions so far contains obfuscated compilations and may be slightly slower than declared. No support of lazy loading (it is a bad proof of concept for loading data from a database and we want to keep your project clean). 

Dapper

+

Maps single table column names and values into entity or DTO properties. 

-

Does not aggregate entities, does not help with constructing SQL in any way, hard to maintain those SQL queries (plain text), consumes a lot of RAM according to data it selects, low performance... 

Entity Framework

+

Entity aggregations, support of complex primary keys, sophisticated query building using LINQ, wide support over the Internet, quick development, high performance, caching entities once aggregated. 

-

Some small impact on an architecture through all layers - less usable for well designed project (harder separation of queries and repositories and also queries and database connections), database first concept generated entities are hardly convertible into JSON in an automated way, enables beginners to build performance issue queries by an accident. 

NHibernate (DNF)

Since we were being giving parsing exceptions for such a complex queries we use in these tests after conversion from SQL -> HQL, this ORM is not included in the tests. We were only able to parse HQL and retrieve results for eager loading from not so complicated nested queries. 

+

Entity aggregations, support of complex primary keys, bulk operations, eager and lazy loading, many years of experience. 

-

Very hard to maintain: User have to maintain duplicitous information of mappings in xml (or in entity attributes using Fluent NHibernate) when entity changes (no generator), eager loading possible only with several queries or by one query using HQL, which is similar to maintain as SQL (plain text). Slow and architecture impacting development in most of the cases. HQL cannot be used for some more complex queries with nested selects - you might join collections in a flat way, that could mean an performance issue or inability to resolve some queries. Pure documentation, pure support of .NET 6.0 (i.e.: Fluent NHibernate is missing completely). 

Pure SQL

+

Full control of queries, direct and quick data fetching 1:1. 

-

No mapping between SQL table <-> entity, hard to maintain queries (plain text), no caching of repeating columns' data. 

Telerik OpenAccess (DNF)

Since this ORM is not supported for .NET 6.0, it has not been included in the tests. 

+

Uses LINQ. 

-

Heavy impact on any architecture using classes such as TrackedList or DataSetResult, no support of eager loading - collections are loaded in a lazy way in separate queries into TrackedList - useful only for a few cases... every time you need to respond with a full result, the database is called many times. No support for .NET 6+. 

MySQL Performance Test Results

Benchmark

Benchmark computation include all the criteria in the following pattern. 

• Benchmark = Transactions - RAM/4 - (Errors)*8 - (multitaskingIteration(seconds) - 140)*4