DNS Exfiltration or Why I threw away my supervised learning models for anomaly detection

Full Featured (30 min.)

Supervised learning is great, but is it always the optimal solution? When telling the difference between cats and dogs there's always enough training examples. But what happens when you’re looking for an extraordinary phenomenon such as a unicorn? Our unicorns are DNS exfiltration attacks, such as the 2014 cyberattack on Home Depot, resulting in the theft of 65M credit card numbers. In this talk we will discuss the advantages of Anomaly Detection in the absence of training samples and the challenges we faced migrating it to large-scale Spark Scala. This is the story of how we had to take a different approach to our problem and how we got to catch a live ‘white-hat’ cyberattack on a client.