How we made an immutable data lake the source of everything

Full Featured (30 min.)
[Infrastructure]

Big data is a challenge that requires lots of specific skills, experience and devops. We identified that one of the main issues causing this is data management - making sure all your data is available, inexpensive and usable. By leveraging cheap storage on S3 and an immutable data lake, we managed to enable developers without specific big data skills to work with data at any scale, without skyrocketing costs. In this session, I'll describe our data lake architecture, and how it supports a large variety of data use cases without increasing costs beyond the basic cost of data storage.