What is TrailDB?

TrailDB is designed to be a core building block for systems that need to store and process a large number of discrete events, organized by a primary key. It is complementary to existing relational and time-series databases and key-value stores.

What makes TrailDB different is immutability: Immutable data enables deeper compression, scalability, and architectural decisions, which would not be feasible with existing databases. This is especially true for cloud environments with object stores like Amazon S3 that are a perfect match for compressed, immutable files.

Developer productivity is another main motivation of TrailDB. Individual files are easy to manipulate using standard filesystem tools. The easily portable C library has only a few easily available dependencies, making it easily deployable. The API is clean and minimal by design. Language bindings are provided for Python, Go, Haskell, R, and D.

TrailDB is a perfect match for use cases that involve detecting patterns over time, such as web/mobile analytics, anomaly detection, and various machine learning models. Since 2014, AdRoll has used TrailDB to store and query over 20 trillion events that power a number of products at AdRoll. TrailDB was open-sourced in May 2016.

Tutorial

This is a hands-on tutorial teaching all you have to know about TrailDB and how to use it in Python. You can bring your own data set consisting of events with a timestamp, a primary key and additional fields, or you can use one of the public data sets provided.

Outline

TrailDB 101

1. Install TrailDB on your laptop or a server
2. Build your first TrailDB
3. Querying TrailDBs

TrailDB 201

1. Optimizing performance
2. Designing an architecture around TrailDB
3. TrailDB internals and contributing back