Edge Inference Benchmark Lab — FP32 vs FP16 vs INT8 on Jetson Orin Nano

contact@davidvincze.com (David Vincze) — Sun, 27 Sep 2026 09:00:00 +0200

Introduction

In this article I am going to use the MobileNetV2 deep learning model from the ONNX model zoo [1] to benchmark the inference performance on NVIDIA Jetson Orin Nano Developer Kit. The main focus is to compare the performance of FP32, TF32, FP16 and INT8 precision modes in terms of latency, throughput, accuracy and power consumption.

The goal is to provide insights into efficient model deployment on edge devices, and to share lessons learned from the benchmarking process. The article is intended for Edge AI enthusiasts, engineers, and researchers who are interested in optimizing deep learning models for edge devices.

Tensorrt - Tag - David Vincze

Edge Inference Benchmark Lab — FP32 vs FP16 vs INT8 on Jetson Orin Nano

Introduction