Wednesday, February 19, 2014

Déjà Vu: TokuMX is to MongoDB what InnoDB is to MySQL

While I was writing about MongoDB lacks the granularity of document-level locking or even collection-level locking, I came across TokuMX. It is touted as an open source, high-performance distribution of MongoDB with 20x performance improvement. On top of the that, it almost corrects all major problems developer community complains about MongoDB.

Besides, the DB level locking, other major complaints about MongoDB are:
  1. Poor space management with fragmented data. You can run "compact" command to rewrite and defragment all data in a collection, as well as all of the indexes on that collection. However, it is a slow process and will introduce significant down time, because the operation blocks all activities to the same database. Prior to MongoDB 2.2, it was even worse -- all activities on the mongod were blocked. 
  2. Difficult to keep working set in the memory. If the search has to hit the disk, the performance degrades dramatically. Usually the idea is to buy more memory and/or use SSD, but that's just throwing money at the problem. Besides, because of the poor space management, the cost of SSD is going to be higher. 
The root cause of the above 2 problems was that the poor underlying storage layer implementation. The storage layer is linked-list BSON documents stored in data files and all data files are memory mapped to virtual memory by OS. MongoDB just reads/writes to RAM and let the OS take care of the rest. It relies on the kernel page caching algorithm like LRU, thus it is possible to evict hot data out of memory for less frequently-accessed data. Also LRU algorithm does to prioritize things like index. After all, OS does not know the underlying storage is for a database. In one of the presentation in MongoDB London 2013, "Understanding MongoDB storage for Performance and Data Safety", one of the pros that was touted was "No complex memory/disk code in MongoDB, huge win!". As a matter of fact, it is a huge win for MongoDB engineers who do not want to implement a state of art storage layer themselves. It is a huge loss to MongoDB customers.

That brings back the memory of MySQL. MySQL originally used MyISAM as its main storage implementation. It does not have transactional support. It maintains table level locking,. It is prone to data corruption and takes long repair time after crash. MySQL 5.5 started to use InnoDB as the default storage engine, which has transactional support, row level locking and faster performance. Only then MySQL became a much desired choice in many enterprises.

TokuMx is to MongoDB what InnoDB is to MySQL. It keeps the most appealing feature of MongoDB such as dynamic schema, JSON-style document format, rich document based query, sharding support, replication, etc. It also supports all the MongoDB drivers the same way thus it is a drop-in replacement for MongoDB to your application.  The key benefit of TokuMx is that it replace the underlying storage engine for MongoDB. It uses Fractal Tree indexes instead of B-tree index, thus delivering better and consistent performance when data size outgrows memory. It provides ACID transaction support. It has MVCC support and allows more concurrency read/write to the database with document-level locking. Compression of all data is by default thus save storage space without downtime (up to 90% space save). There is no data fragmentation and there is no need for "compact" command. The current 1.4 release is fully compatible with MongoDB 2.4, except no text indexes and search and no Geospatial index and query support.

It would be interesting to see how MongoDB and TokuMx relationship play out in the future. 

No comments: