Stafræni Háskóladagurinn 2021: Object detection using neural networks in your smartphone trained by a supercomputer

Helmut Neukirchen, 25. February 2021

The University of Iceland's Computer Science department is researching machine learning using the next generation's supercomputer DEEP-EST -- by the way: we are also offering a Data Science specialisation in our Computer Science programme, where, e.g., machine learning including deep neural networks is covered. To showcase what is possible if you have a supercomputer to train neural networks, we offer a web page that allows you to use the camera of your smartphone (or laptop) to detect objects in real-time.

https://nvndr.csb.app/

Just open the following web page and allow your browser to use the camera: https://nvndr.csb.app/
(Allow up to approx. 1 minute for loading the trained neural network and for initialisation. Web page works best in landscape orientation.)

While neural networks are still best trained on a supercomputer, such as DEEP-EST with its Data Analysis Module, the trained neural network even runs in the browser of a smartphone (purely running locally as Javascript in your browser without any connection to a supercomputer, i.e. completely offline after having downloaded the Javascript code and the trained neural network).

The used approach is Single Shot Detector (SSD) (the percentage shows how sure the neural network is about the classification) using the MobileNet neural network architecture. The dataset used for training is COCO (Common Objects in Context), i.e. only objects of the labeled object classes contained in COCO will get detected. The Javascript code that is running in your browser uses Tensorflow Lite and its Object Detection API.

Example object detection via a neural network

If you want learn more about the DEEP-EST project where the next generation supercomputer is developed, have a look at the poster below (click on the picture below for PDF version):

PDF of DEEP-EST poster