How we built our livescore platform using Clojure and Datomic

@stijnopheide

BeClojure

Hi, we are

What is Datomic?

The fully transactional, cloud-ready, immutable database...

...with an awesome query language...

...and yes, a schema!

A very short introduction to

Datomic

cheap comparison with other databases

Fully Transactional

ACID

Atomicity

Consistency

Isolation

Durability

Transactional

Cloud-Ready

(conj web-site buzz-words)

An intermezzo on

Architecture

Deconstruct

Query - Transactions - Storage

Cloud-ready

Peers

application servers can be scaled based on query demand

Transactor

commodity server (EC2, rackspace, ...)

Storage

hosted solutions

Cloud-Ready

Immutable database

An intermezzo on the Datomic

Data Format

[e a v t]

[entity attribute value transaction]

[e a v t]

[12345 :person/id 1 123]
[12345 :person/name "Wiggins" 123]
[12345 :person/first-name "Bradley" 123]
[12345 :person/title "Mr" 123]
[12345 :person/e-mail "brad@sky.com" 123]
[12345 :person/title "Sir" 124]
[124 :db/txInstant "2013-12-10T12:45:23Z"]

Immutable Database

Time

Q: When did Wiggins become a knight?

[:find ?ts
 :where [?e :person/name "Wiggins"]
        [?e :person/title "Sir" ?tx]
        [?tx :db/txInstant ?ts]]

Manage Time

Immutable Database

Read Scalability

Read Scalability

Schema

schema defines attributes and their characteristics

it does not define which attributes belong to which entity types

structured, but flexible

Schema

  • primitive data types: string, numbers, boolean, date, ...
  • references to other entities (cardinality one or many)
  • unique, indexed, fulltext index
  • but doesn't support SQL's other constraints

Result?

I get my joins back!

Schema

(Structure)

Schema

(Flexibility)

Query

SQL

SELECT e.address
FROM person as p
JOIN email as e
ON p.id = e.person_id
WHERE p.name = :name

Mongo

user = db.users.find({ name: name });
db.email.find({ user_id: user.id });

Datalog

[:find ?email
 :in $ ?name
 :where [?e :person/e-mail ?email]
        [?e :person/name ?name]]

Datalog

  • simple syntax
  • declarative
  • like logic programming. you get the joins for free
  • composable: queries are clojure data structures
  • only ask the question, not the data selection

Query

The best database in the world

In some cases

Remark

What about write scalability?

But...

Writes are pretty fast

Write Performance

Not suited for

applications with unlimited write scalability

But it's awesome for

applications that need write consistency and read scalability

Goalstrm & Datomic

Requirements

live data streaming

calculation of trending content

possibly lots of clients

STOP

Demo Time

Thank you.

Questions?