When the database can't keep up

04:30 PM - 05:25 PM on July 17, 2016, Room CR7

Adam DePrince

Audience level:


An overview of Peloton's custom leaderboard database.


When postgres couldn't keep up, redis ran out of room we found answers in an unusual place. Going "back to the basics" and building a very simple database in python using zerorpc, and operating directly on the file system using plain posix calls provided a surprisingly robust and scalable solution for us. In this presentation we'll review the use case of Peloton's leaderboard server, the specific operations we require and how a combination of simple reads, appends and memoization allowed us to achieve surprisingly high levels of performance for our application.