At data-driven companies, the findings from the data science team, or the outputs, influence the APIs that power software used by the other teams. Perhaps an inventory manager monitors the predicted sell-through rate of all new items added to an e-commerce platform, or an engineer builds a client application that leverages machine learning to recommend music. These are examples of how companies can integrate, leverage and disseminate machine learning and data science.
A challenge a lot of companies face exist around the handoffs — how can other teams interact with and build applications on models developed by data scientists? Taking a page out of the software engineering playbook, complex systems can speak to each other through APIs. I wanted to leverage the DataScience.com Platform to construct a machine learning API to power other applications. I also wanted to leverage Skater, our model interpretation library, to help disconnected teams understand how shared machine learning models work.
I looked at the problem of network intrusion, and detecting malicious activity on a network. Once I built a good model, I wanted to:
- Set up an API that processed sequences of connections, determined if there was an attack, and what type of attack existed.
- Build a notification system that, in the event of a detected attack, sent notifications to stakeholders with a link to a report that indicated why the system believed an attack was underway.
I gave myself 48 hours to complete the project, using the DataScience.com Platform to help me stitch together all the components.