Machine learning (ML) models are as good as the data you give them. This is true during training, but also once a model is put into production. In the real world, the data itself can change as new events occur and even small changes in the way databases and APIs report and store data could have implications for how models react. Since ML models will simply give you wrong predictions and not generate errors, it is imperative that companies monitor their data pipelines for these systems.
This is where tools like Aporia The Tel Aviv-based company announced today that it has raised a $ 5 million funding round for its ML model monitoring platform. The investors are Vertex Ventures and TLV Partners.
Aporia co-founder and CEO Liran Hason, after five years with the Israel Defense Forces, previously worked in the data science team at Adallom, a security firm that was acquired by Microsoft in 2015. After the sale, he joined venture capital firm Vertex Ventures before launching Aporia at the end of 2019. But it was during his time at Adallom that he first encountered the issues that Aporio is now trying to solve.
“I was responsible for the production architecture of the machine learning models,” he said of his time with the company. “So that’s actually where, for the first time, I got to experience the challenges of getting models into production and all the surprises you have there.
The idea behind Aporia, Hason explained, is to make it easier for companies to implement machine learning models and to harness the power of AI responsibly.
“AI is a super powerful technology,” he said. “But unlike traditional software, it relies heavily on data. Another unique feature of AI, which is very interesting, is that when it fails, it silently fails. You get no exceptions, no errors. It gets really, really tricky, especially when going into production, because in training, data scientists have full control of the data. “
But as Hason noted, a production system can depend on data from a third-party vendor, and that vendor may one day change the data schema without telling anyone. At this point, a model can no longer be trusted – for example to predict whether a bank customer might default on a loan – but it can be weeks or months before anyone notices.
Aporia continuously monitors the statistical behavior of incoming data and when it gets too far away from the training set, it will alert its users.
One thing that makes Aporia unique is that it offers its users an almost IFTTT or Zapier graphical tool to configure the logic of these monitors. It comes preconfigured with over 50 monitor combinations and provides full visibility into how they work behind the scenes. This, in turn, allows companies to refine the behavior of these monitors for their own business case and model.
Initially, the team thought they could create generic monitoring solutions. But the team realized that it would not only be a very complex undertaking, but that the data scientists who build the models also know exactly how those models should work and what they need from a monitoring solution. .
“Monitoring production workloads is a well-established software engineering practice, and it is high time that machine learning was monitored at the same level,” said Rona. Segev, founding partner of TLV The partners. “Aporia‘s The team has solid experience in production engineering, which makes their solution stand out for its simplicity, security and robustness.