Google Colab Webcam streaming and processing

One of the code examples that Colab provides handles recording of an image via a webcam. The code writes a webcam frame to an image file and displays it. However, it does not directly handle life streaming of the webcam feed.

I wanted to perform deep learning-based eye-tracking within Colab using the webcam. That would require continuous processing of the video stream. The code to do so can be found here: https://colab.research.google.com/gist/kueblert/64299582829c35a5579b520e0bdfa9a8/cameraaccess.ipynb

In short, it uses the same interface as the example to pass the frame via javascript from the browser to the Colab Runtime. Therefore, the frame data is converted to a Data URL, i.e., compressed as a jpeg and base64 encoded.

The python backend decodes the image string and decompresses the jpeg data.

As I wanted to visualize the data as well, passing data from python to the javascript caller is also demonstrated.

A word of caution: While the approach does work, the performance is clearly not good and the number of frames you are able to process per second is very limited.

Leave a Reply

Your email address will not be published. Required fields are marked *