Google Colab Webcam streaming and processing

One of the code examples that Colab provides handles recording of an image via a webcam. The code writes a webcam frame to an image file and displays it. However, it does not directly handle life streaming of the webcam feed.

I wanted to perform deep learning-based eye-tracking within Colab using the webcam. That would require continuous processing of the video stream. The code to do so can be found here:

In short, it uses the same interface as the example to pass the frame via javascript from the browser to the Colab Runtime. Therefore, the frame data is converted to a Data URL, i.e., compressed as a jpeg and base64 encoded.

The python backend decodes the image string and decompresses the jpeg data.

As I wanted to visualize the data as well, passing data from python to the javascript caller is also demonstrated.

A word of caution: While the approach does work, the performance is clearly not good and the number of frames you are able to process per second is very limited.

Leave a Reply

Your email address will not be published. Required fields are marked *