Tutorial: Real-Time Object Detection with DeepStream on Nvidia Jetson AGX Orin

Last month, Nvidia unleashed the next-generation edge computing hardware device branded as Jetson AGX Orin at GTC. Courtesy of Nvidia, I was fortunate enough to get a Jetson AGX Orin Developer Kit to evaluate and experiment with it.

The Jetson AGX Orin Developer Kit has everything you need to run AI inference at the edge with ultra-low latency and high throughput. As a successor to the most powerful Jetson AGX Xavier, AGX Orin packs a punch.

Below are the specifications of the Jetson AGX Orin compute module:

The developer kit comes with a carrier board that makes it easy to connect various peripherals.

The Jetson AGX Orin Developer Kit comes with a preview of JetPack SDK 5.0, which is based on the Ubuntu 20.04 root filesystem and Linux Kernel 5.10. It comes preloaded with CUDA 11.4, cuDNN 8.3.2, TensorRT 8.4.0, and DeepStream 6.0.

This tutorial will walk you through the steps involved in performing real-time object detection with DeepStream SDK running on Jetson AGX Orin.

Step 1 – Install TensorFlow on JetPack 5.0

Since we use a pre-trained TensorFlow model, let’s get the runtime installed.

sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v50 'tensorflow<2'

This installs TensorFlow 1.15 which works best for this tutorial. The --extra-index-url points to a wheel optimized for JetPack 5.0.

Check if the GPU is accessible from TensorFlow.

Step 2 – Download Pre-trained TensorFlow Inception Model

We will use the TensorFlow Inception V2 model trained on the COCO dataset.

Step 3 – Converting the TensorFlow Model to TensorRT

This step optimizes the model by converting the TensorFlow frozen model to a serialized TensorRT UFF MetaGraph model.

Step 4 – Compiling the Custom Object Detector Sample Application

DeepStream SDK comes with a sample application that can be integrated with various models. We will now compile the application to use our Inception V2 model.

Step 5 – Edit DeepStream Application Configuration File

Open /opt/nvidia/deepstream/deepstream/sources/objectDetector_SSD/deepstream_app_config_ssd.txt and replace the [source0] section with the below contents:

This essentially configures the USB webcam as the input source. Once you connect the camera, make sure it’s visible to JetPack.

Step 6 – Run the DeepStream Inference Pipeline

Finally, run the inference to perform object detection.

We can increase the inference performance and the number of frames per second by switching to the inbuilt Camera Serial Interface (CSI) of the Jetson AGX Orin Developer Kit.