NVIDIA Brings Affordable GPU to the Edge with Jetson Nano
Jetson Nano is the latest addition to NVIDIA’s Jetson portfolio of development kits. The $99 developer kit powered by the Jetson Nano module that packs a lot of punch. It comes with a Maxwell GPU, Quad-core ARM processor, and 4GB RAM.
When it comes to machine learning accelerators, NVIDIA is a segment leader. Almost every public cloud provider offers virtual machine (VM) infrastructure backed by NVIDIA Tesla K80, P100, and T4 GPUs. These VMs come with pre-configured software such as NVIDIA drivers, CUDA Toolkit, cuDNN libraries, and GPU-enabled deep learning frameworks. Developers and data scientists provision these VMs to train complex neural networks.
What’s unique about Jetson Nano developer kit is the availability of NVIDIA GPU that’s compatible with its data center counterparts. It’s a less powerful platform based on AArch64 architecture while most of the servers and VMs are based on X86.
Jetson Nano is meant to be deployed at the edge running fully trained deep learning models that take advantage of local GPU acceleration. The workflow involves using the public cloud for training the models, optimizing the models for inference on NVIDIA GPU, and deploying it at the edge.
NVIDIA’s inference engine is based on TensorRT, a runtime that’s optimized for forward propagation which is a technique used for accelerating inferencing. TensorFlow, PyTorch, and Caffe2 models can be converted into TensorRT to exploit the power of GPU for inferencing.
Jetson Nano developer kit makes it easy to develop, test, debug, and deploy TensorRT modules at the edge. It acts as the carrier board to program the GPU module. The Jetson module powering the kit can be easily detached and deployed in production environments.
The development kit brings the best of IoT and AI development to the community. It has Raspberry Pi-compatible GPIO pinout that can support a broad range of shields and hats from the Pi ecosystem. Most of the Python and Node.js programs that talk to Raspberry Pi GPIO can be easily ported to Jetson Nano.
Jetson Nano supports standard USB cameras as well as the Raspberry Pi camera modules through Camera Serial Interface (CSI).
Let’s take a closer look at the hardware specifications of Jetson Nano.
The OS and software are packaged into an image that can be downloaded and flashed into an SD card much like the Raspbian OS. NVIDIA calls this platform as JetPack SDK which includes a flavor of Ubuntu, drivers, runtimes, tools, and frameworks to develop and deploy neural network models.
The latest version of JetPack SDK is 4.2 which includes LTS Kernel 4.9 support, the new Jetson.GPIO Python library, TRT Python API support, and a new accelerated renderer plugin for GStreamer framework. Developers can get started as soon as they boot the developer kit with the JetPack 4.2 image.
JetPack SDK 4.2 is built on top of Ubuntu 18.04 with a modified kernel. It includes TensorRT, which is a high performance deep learning inference runtime for image classification, segmentation, and object detection neural networks. TensorRT accelerates deep learning inference as well as reducing the runtime memory footprint for convolutional and deconvolutional neural networks.
It also includes the CUDA Toolkit that provides a comprehensive development environment for C and C++ developers building GPU-accelerated applications. The toolkit comes with a compiler for NVIDIA GPUs, math libraries, and tools for debugging and optimizing the performance of applications. CUDA Deep Neural Network, cuDNN, a library built of on top CUDA, provides high-performance primitives for deep learning frameworks. It includes support for convolutions, activation functions, and tensor transformations.
JetPack also comes with Docker Engine. But unfortunately, it doesn’t support NVIDIA-Docker, a fork of Docker which aware of GPUs. But the Docker on JetPack is capable of running images built for ARM that are based on AArch64 architecture. With some tweaks and additional parameters, containers will be able to talk to the underlying GPU. This workaround makes it possible to run mainstream edge computing platforms. AWS IoT Greengrass officially supports Jetson family of products while Microsoft is yet to extend Azure IoT Edge to ARM64 and AArch64.
The easiest way to get started with Jetson Nano is to train models with NVIDIA DIGITS, convert the models to TensorRT, and deploying them at the edge. For a detailed walkthrough of setting up, configuring, and training models with NVIDIA DIGITS, refer to one of my previous tutorials.
The edge computing market is growing rapidly. With NVIDIA, Intel, Google, Qualcomm investing in custom hardware accelerators for AI, deploying deep learning models for inferencing has become the key driver for edge computing. This trend combined with software platforms from AWS, Microsoft, Google, and startups such as FogHorn, IoTium, and Swim, edge computing is transforming into a multimillion-dollar opportunity.
In one of my upcoming tutorials, I will demonstrate how to optimize and deploy deep learning models trained in the public cloud on Jetson Nano. Stay tuned.
Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.