Overview

Neutron NPU Passthrough on NXP i.MX95

Overview

The Neutron-NPU-Passthrough-on-NXP-i.mx95 provides a ready-to-use inference environment for validating ONNX models on the NXP i.MX95 platform with full support for Neutron NPU hardware acceleration.

This container is intended for developers, system integrators, and solution architects who need a fast and reproducible way to evaluate CPU vs NPU inference performance on i.MX95 without dealing with complex dependency resolution, driver alignment, or runtime configuration.

All required runtimes, libraries, and demo assets are pre-configured, enabling immediate execution of AI inference workloads.

Key Features

ONNX Runtime on i.MX95
Pre-integrated ONNX Runtime optimized for NXP i.MX95 architecture.
Neutron NPU Acceleration
Native support for i.MX95 Neutron NPU via ONNX Runtime Execution Provider.
Out-of-Box Inference Demo
Includes a complete image classification example using MobileNetV2.
CPU / NPU Comparison
Simple runtime switch for validating functional correctness and performance.
Lightweight & Reproducible
Focused container design for benchmarking, validation, and PoC workflows.

Hardware Specifications

Component	Specification
Target Hardware	Advantech AOM-5521
SoC	NXP i.MX95
GPU	Arm® Mali™ G310
NPU	eIQ neutron N3-1034S
Memory	8 GB LPDDR5

Operating System

Environment	Operating System
Device Host	Yocto 5.2 Walnascar
Container	Ubuntu:24.04

Software Components

Component	Version	Description
ONNX Runtime	1.22.0 from NXP BSP v6.12.20	Inference runtime
Python	3.13	Demo application runtime
NumPy / Pillow	Latest	Image preprocessing

Included Models & Assets

Image Classification Model

Model	Format	Note
MobileNetV2	ONNX	Lightweight CNN for inference validation

Demo Assets

File	Description
`grace_hopper.bmp`	Sample RGB image
`labels.txt`	ImageNet classification labels

Hardware Acceleration Support

Accelerator	Support Level	Notes
NPU (Neutron)	INT8 (primary)	Best performance-per-watt on i.MX95
CPU	FP32	Reference & fallback execution

Precision Support

Precision	Supported On	Notes
INT8	NPU / CPU	Recommended for deployment
FP32	CPU	Debug & reference

Quick Start Guide

Prerequisites

Please ensure docker & docker compose are available and accessible on device host OS
Since default eMMC boot provides only 16 GB storage which is in-sufficient to run/build the container image, it is required to boot the Host OS using a 32 GB (minimum) SD card.

For container quick start, including the docker-compose file and more, please refer to Advantech Container Github Repository

Best Practices & Known Limitations

Minimum storage Storage required for running Docker containers is 32 GB
NPU passthrough Neutron EP error sometimes, reboot the development board may solve this problem.

Neutron: NEUTRON_IOCTL_BUFFER_CREATE failed!: Cannot allocate memory