Dr. Michael Stonebraker is a legend in the database world. He was the architect of Ingres, Postgres, Vertica, Illustra, Cohera, Streambase, H-Store and VoltDB. In this interview, he dives deep into database technologies. He talks about evolution of databases and certainly provokes you to think about the technology, the industry and how application needs are driving development in database technologies. A few nuggets from the interview.
- The traditional (row oriented) database technology is becoming obsolete
- The database market is at a tipping point
- DB market can be viewed as three segments – data warehouse market, OLTP market (both new and old) and a potpourri of other stuff
- The potpourri consists of various types of NOSQL, Graph databases, Column Stores, In Memory databases
- Databases currently spend 90% on overheads and only 10% on value producing work (big Gasp!)
- The overhead consists of – feeding buffer pool, mapping buffer pools to disk, lock management, logging, multi-threading
- There are ways to reduce these overheads using shared data structures, removing the record level locking, implementing multi-version concurrency, logging commands instead of data changes, using replication, getting rid of buffer pools, getting rid of expensive multi-threading, optimistic concurrency control etc.
- He talks about Distributed concurrency vs eventual consistency
- Most database books cover traditional database architectures. He advocates teaching – column stores, in memory databases, array databases and graph database architectures.
- He talks about embarrassingly parallel databases
- VoltDB evoloved from HStore prototype from MIT
- Applications are multiplayer games, high speed OLTP
- Some examples of NewSQL databases – Hana from SAP, Hekaton from Microsoft, SQLFire by VMWare, NeuroDB, NetSQL
- Most of the db systems written in the past year are open source
He takes a lot of digs at the closed source sales models. Deep dive into databases is an intoxicating topic and this was a great interview. Thanks to IEEE Software Engineering Radio folks for their indepth, high quality interview questions.
Here is a link to Dr Michael Stonebraker on Current Developments in Databases
Meta
- I was lucky to spend a lot of time in late 80s and early 90s reading Michael Stonebraker’s papers.
- I met him twice (once when I attended an RDB seminar at Palo Alto and once when we walked into Illustra offices (we built the ODBC driver for Illustra in my previous startup)
- I have been following him when I was working on complex event processing and looking at Telegraph (built on top of Postgres) and Streambase
- I lost touch even though I kept hearing about Vertica and other developments
- Michael Stonebraker inspires me and lots of other data geeks like me