Notes from Industry

Deploying Machine Learning models to the real world is prone to uncover domain coverage issues. One way to robustify the models is by generating unseen data, which the model is expected to work on. So let’s see how property based testing can solve this issue!

Property-based testing aims to maintain a certain property given an operational domain, allowing us to estimate the robustness of the model. Image by author.

In the last article on testing Machine Learning models, we looked into drawing parallels between software testing and how those strategies can be used within the MLOps domain. We showed that static dataset-based methods for evaluating model performance are not powerful enough to fully examine the accuracy of the models in high dimensional spaces. One way to gain more insight into the robustness of a model is through methods like property-based testing that go about synthesising input data, conditioned on some specifications, in order to break the model!

First, let’s begin by talking about the expected operational domain. In other…

Notes from Industry

Let’s make ML models more robust by taking inspiration from modern software testing! Or how to confidently improve your models.

Machine learning systems are now ubiquitous in our daily lives and so the correctness of their behaviour is absolutely crucial. When an ML system makes a mistake it can not only result in an annoying online experience, but also limit your ability for socio-economic movement or, even worse, make life-threatening manoeuvres in your car. So how certain are you that a deployed ML system is thoroughly tested and you are not effectively a test user? On the flip side, how do you know that the system you’ve been developing is reliable enough to be deployed in the real world? And…

When you start your machine learning journey, you are very early on hit by new terminology. So let’s see what are tensors and what can we do with them?

If we begin our search on Wikipedia, we’ll encounter something like:

In mathematics, a tensor is an algebraic object that describes a relationship between sets of algebraic objects related to a vector space.

In other places we’ll read that they are synonymous to matrices or tables. Albeit closer, they represent a special case of a 2-dimensional tensor. …

Creating neural networks often involves debugging the structure, sequence and loss functions used in different parts of the model. What is the tool that gives the best insight? We’ll look into popular solutions with 1D, 2D and 3D options in mind!

Print statements can show you a 1D representation, Tensorboard — in 2D, with Efemarai you can explore your computational graph in 3D. Photos by author.

Building networks often involves looking through the resulting computational graph and confirming that the fundamental research idea, sketched on the whiteboard, has been transformed into the correct structure. For this experiment let’s choose a relatively complex structure — DCGAN alongside its loss function— that may show the usefulness of the different platforms for debugging large models. This is how…

See visually what needs to be altered in the model and how it impacts further training.

It doesn’t have to be scary to understand how neural network models are changed. Photo by author.

One of the great results in Deep Learning came in a method known as transfer learning. That is the situation in which using a network trained for a particular task is used as an initialization for another (close) problem. This is possible, as the first few layers of the network learn generic features, which are useful across multiple domains, with the prediction part of the network tuned for the task at hand. …

Daniel Angelov

PhD Machine Learning and Robotics @ University of Edinbrugh

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store