Deep Learning is a great tool. But, one of its problems is that it’s overly optimistic: too high confidences for unknown classes. That’s a problem in an open world where new classes may be encountered.

See the unknown fruit below. Standard deep learning (left) is shockingly certain that it is pear (which it not the case). We extended a standard model with a notion of uncertainty: it says the unknown does not look like the known classes (right).

At the Videos section, you can find more examples.

Our model is an Equidistant Hyperspherical Prototype Network. The paper was accepted at NeurIPS, at a workshop about uncertainty in decision making. It is a collaboration with Pascal Mettes from the University of Amsterdam.

The prototypes are almost for free. This model has the same backbone network, while requiring less weights for the classifier head. It just requires 5 additional lines of Python code for the prototypes and the cosine loss.

Previous methods to obtain lower confidences for unknown classes are:

Model the ‘other’ category as an extra class Bayes models: a distribution over the model parameters (Deep) Ensembles: a number of models in which the mutual differences indicate an uncertainty.

For adding a class ‘other’, you have to be lucky that an unknown class looks more like ‘other’ than a known class. Bayes and Ensembles require more compute and memory. For instance, ensembles work well for >= 5 models (see a review by Google). Running that many models on a (small, limited) system is in practice inconvenient or sometimes impossible. This is problematic for embedded systems, such as SNOW’s compute backpack for SPOT.

A very simple, cheap solution turns out to be possible: prototypes. These are representations of the known classes by means of vectors. The nice thing is that unknown pictures ‘fall’ in between these prototypes. That works especially well if the prototypes are ‘equally’ separated. Such ‘equidistant’ prototypes improve uncertainty estimation.

Our NeurIPS paper is joint research with Pascal Mettes from the UvA. In the paper, we explore the classification of known and unknown scenes. This is relevant for robots who need to understand where they are and whether they know the place.

Let me know if you want to apply it or have any questions!