How has the recent turmoil within the OpenAI offices changed your plans to use GPT in a business process or product in 2024?
Increased uncertainty means we are more likely to evaluate alternative AI chatbots and LLMs.
No change in plans, though we will keep an eye on the situation.
With Sam Altman back in charge, we are more likely to go all-in with GPT and LLMs.
What recent turmoil?
Edge Computing

Tutorial: Accelerate Deep Learning Models with Intel Movidius

Mar 15th, 2019 8:30am by
Featued image for: Tutorial: Accelerate Deep Learning Models with Intel Movidius

Intel Movidius Neural Compute Stick accelerates machine learning inferencing at the edge. I covered the details of this device last week. In this tutorial, we will take an existing Caffe deep learning model and optimize it for Intel Movidius.

This guide is based on Intel Movidius NCS 1 and NCSDK 2. The most recent version of the device uses Intel OpenVINO Toolkit which is not compatible with the previous versions of the SDK. But Intel is still selling NCS 1 devices while actively maintaining the SDK.

Apart from the NCS1 device, you need an Ubuntu 16.04 PC with a free USB 3 port. You can use VirtualBox or VMware Fusion/Workstation to set up and configure the SDK. Optionally, a Raspberry Pi 3 device may be used to run the optimized graph.

Installing Intel Neural Compute SDK 2

Let’s start by confirming that the Intel Movidius NCS USB device is recognized. Running lsusb should show a device with ID 03e7:2150.

We will now install the prerequisites – Python3, Pip, and Git packages on Ubuntu.

Clone the NCSDK 2 Github repository and build the SDK.

It may take a few minutes for the setup to complete. After it is done, verify that the SDK is properly installed by running the hello_ncs_py sample.

The above output confirms that the SDK is able to access the Intel Movidius NCS device.

Generating the Graph from Caffe Deep Learning Model

In one of the previous tutorials, I used NVIDIA DIGITS to build a Convolutional Neural Network (CNN) that classifies images. We will use the fully-trained model from that demo to classify the images of dogs and cats.

Download the trained Caffe model from the below links:

Before we use the model for inference, we need to generate a graph optimized for Intel Movidius. For this, we will use mvNCCompile, one of the command tools available in the NC SDK. This tool takes the trained model as an input and generates the required graph.

The first two parameters point to the Caffe model while -s 12 denotes that we are using 12 SHAVE Cores for the graph. The last parameter is the size of image, which is 227X227.

You should now find two new files – graph and output_expected.npy which can be loaded onto the NCS device for inference.

It’s time for us write Python code that loads the graph and does inference.

Let’s call the above file as Before we pass sample images to test the graph, we need to create a labels file with just two entries in separate lines.

You may want to download samples preprocessed images from this link.

Invoke the graph through the below command:

In one of my upcoming tutorials, I will demonstrate how to use Intel Movidius NCS 2 with OpenVINO Toolkit.

Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar for a deep dive on accelerating machine learning inference with Intel Movidius.

Feature image by F. Muhammad from Pixabay.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.