Modal Title

Off-The-Shelf Hacker: Adjusting the Robotic Skull’s Vision

Sep 20th, 2018 3:00am by
Featued image for: Off-The-Shelf Hacker: Adjusting the Robotic Skull’s Vision

A major feature of Hedley the robotic skull is the JeVois smart machine vision sensor transplanted into his right eye socket. The device has a tiny color camera whose image frames are analyzed by on-device advanced artificial intelligence algorithms to track, interpret and evaluate things in its field of view. The sensor outputs an augmented-reality video feed over USB and a text data feed over a hardware serial line.

I’m always on the hunt for optimized techniques and programs that will make Hedley more capable and easier to use. Reliability and predictable behavior are also particularly important when Hedley and I are up on stage demonstrating the latest off-the-shelf hacker and physical computing trends.

Hedley has to boot up and perform properly at show time.

Demo with luvcview

Actually seeing the video from the JeVois sensor is a great way for audience members to understand exactly how the device and its intelligent algorithms work during one of my tech talks. We do this by sending the video from the JeVois device to Hedley’s Raspberry Pi 3 and then using a webcam viewing program to see the output. Hedley’s HDMI display output connects directly to a monitor or a projector. I call that the skull top.

I’ve recently switched to the luvcview program for viewing the JeVois sensor video feed.

Initially, guvcview worked fairly well for seeing what Hedley sees. Sadly, over a few Raspbian Linux software revisions the program started performing poorly on the Raspberry Pi 3. Consequently, I switched to webcamoid right before the July OSCON conference presentation.

As crazy as it sounds, webcamoid worked great on the Pi 3 until a couple of weeks ago. I was a bit surprised after the latest Raspbian update when webcamoid began pegging all four cores of the Pi at around 90 percent utilization (as shown with top) and ran very slowly. I let the program go for a couple of minutes and the little high-temperature warning icon even started appearing on Hedley’s skull top screen. Something obviously was off. I poked around in the configuration file at ~/.config/Webcamoid/Webcamoid.conf but didn’t really see what was causing the excessive CPU utilization. Searching the web didn’t turn up much either.

So now, luvcview is my solution. It runs well on the Raspberry Pi 3 at an average of about 28 percent CPU utilization. One nice thing about Linux is that there are usually several different programs that will serve your purposes. If one doesn’t work and the cause isn’t fairly apparent, use something else.

Lucview can be installed using apt-get.

The program is then easily launched via the command line in a terminal using a few options.

This starts luvcview with the USB connected JeVois sensor and a resolution of 320×260.

Algorithms are selected on the JeVois by switching the resolution in the viewing program. Here are several demo algorithms I’ve used with Hedley and their corresponding resolutions. Check the links for details or go to the main page for a complete list of things you can do with the sensor.

Luvcview has controls for brightness, image size, and so on. The controls appear as a simple row of buttons across the bottom of the skull top window. The button functions are certainly not obvious. Even stranger is that I wasn’t able to find any user documentation about each button’s function.

Fortunately, adjustments to the image aren’t needed coming from the JeVois sensor. Just passing the sensor device name and resolution options on the command line are sufficient for Hedley’s demonstration needs.

The audience just needs to see what Hedley sees and I’ll fill in the rest of the details.

Wrap up

Right now, luvcview satisfies Hedley and my presentation needs. I’ll continue to work to get guvcview and webcamoid back working again in the coming months. I like the interface with guvcview. Sometimes it just takes time to resolve bugs in new versions of the applications.

Linux usually provides at least a few options for solving any given problem. Updates and revisions also sometimes break an application. It is just the nature of the beast. In spite of a few quirks, I know Linux is the best operating system for my line of work.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.