[OMNIML-5774] Add BEVFormer ONNX PTQ and evaluation example

[ajrasane]

What does this PR do?

Type of change: new example

Adds an end-to-end BEVFormer-tiny ONNX PTQ example under examples/onnx_ptq/bevformer. The example provides a pinned container environment, temporal calibration-data generation, INT8 and FP8 quantization, TensorRT engine building, full nuScenes accuracy evaluation, and latency benchmarking.

The workflow extends NVIDIA's DL4AGX BEVFormer INT8 example with FP8 support and reuses the shared bounded NpzCalibrationReader introduced by #2180. It also adds temporal-state unit coverage, links the example from the ONNX PTQ index, and updates the changelog.

Usage

python examples/onnx_ptq/bevformer/quantize.py \
  --onnx=/artifacts/bevformer_tiny_epoch_24_cp2_op13.onnx \
  --calibration-dir=/artifacts/calibration \
  --trt-plugins=/workspace/BEVFormer_tensorrt/TensorRT/lib/libtensorrt_ops.so \
  --quantization-mode=fp8 \
  --output=/artifacts/bevformer_tiny_epoch_24_cp2_op13.fp8.onnx

Testing

• Ran the targeted CPU-only unit tests: 10 passed.
• Ran all applicable pre-commit hooks, including Ruff, mypy, Bandit, markdownlint, license checks, and requirements validation.
• Generated 600 temporal calibration samples and validated the INT8 and FP8 ONNX graphs.
• Built FP16, INT8, and FP8 TensorRT 10.14.1.48 engines on an NVIDIA RTX 6000 Ada Generation GPU.
• Evaluated all 6,019 nuScenes validation samples.

Precision	Median GPU compute time	NDS	mAP
FP16	4.597 ms	0.3546	0.2515
INT8/FP16	3.180 ms	0.3512	0.2505
FP8/FP16	4.106 ms	0.3526	0.2489

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ✅
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: ✅
• Did you write any new necessary tests?: ✅
• Did you update Changelog?: ✅

Additional Information

This PR is stacked on #2180 to reuse its shared calibration reader and should be retargeted to main after #2180 merges.

Reference workflow: NVIDIA DL4AGX BEVFormer INT8 example.

🤖 Generated by Codex (AI agent).

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)

• main
• release/.*
• feature/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9d4d1ef0-b353-487c-a197-1974ca790b22

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[cjluo-nv]

Bot review (claude-opus-5) — DM the bot to share feedback.

New BEVFormer-tiny ONNX PTQ example (8 files, +593). Structure closely mirrors the existing examples/onnx_ptq/petr and far3d examples (Dockerfile + prepare_calibration.py + quantize.py + pinned requirements.txt + README), reuses the shared NpzCalibrationReader from examples/onnx_ptq/quantization_utils.py rather than adding a new reader, and the modelopt.onnx.quantization.quantize() call uses valid kwargs. build_inputs has a focused unit test that covers the scene-boundary reset and the can_bus delta math, and the temporal bookkeeping in that function (deep-copying position/angle before mutating can_bus) looks correct. Size is small and cohesive. No prompt-injection content found in the PR text.

Flagging for owner sign-off rather than approving:

• Licensing / OSRB (blocks auto-approval). prepare_calibration.py is explicitly "Adapted from" NVIDIA/DL4AGX bevformer-int8-eq/tools/calib_data_prep.py and carries a non-canonical header (Copyright (c) 2024, 2026 … plus an attribution line) that diverges from LICENSE_HEADER. The Dockerfile additionally clones DerryHub/BEVFormer_tensorrt (which vendors OpenMMLab-derived code under third_party/), applies a DL4AGX patch, and builds mmcv/mmdetection; the README instructs users to download a third-party checkpoint and nuScenes/CAN-bus data. This is the same third-party-provenance review that was requested on the sibling PETR PR (#2180) — it needs a human/OSRB sign-off, not an automated approval.
• Stacked on an unmerged base. The base branch is ajrasane/petr (#2180), whose own review has open items and which owns NpzCalibrationReader. The diff can only be judged final once that lands and the reader API is settled; the PR should be retargeted to main before merge.
• README/code mismatch on prev_bev. The README says the script "carries prev_bev across frames while resetting it between scenes", but main() never resets prev_bev — only use_prev_bev is set to 0 and can_bus[:3]/can_bus[-1] are zeroed, so the last frame of the previous scene's BEV is what gets written into the first NPZ of the next scene and therefore into the calibration statistics for the prev_bev input. Please confirm this matches the DL4AGX reference and either zero prev_bev on scene change or fix the README wording.
• Deferred imports without a stated reason. from mmcv import Config and from third_party.bev_mmdet3d.datasets.builder import … sit inside main(), while the PETR/FAR3D prepare_calibration.py scripts import at module top. The likely (and legitimate) reason is that the new unit test imports build_inputs on a CPU-only runner without mmcv/mmdet3d — please add a one-line comment saying so, otherwise it reads as a style deviation.
• Fragile container pin. onnxruntime-gpu==1.24.0.dev20260123002 comes from an Azure nightly feed; those feeds prune old dev builds, so the image build will silently start failing. A note in the README (or a documented fallback) would help. python setup.py install for pytorch-quantization is also deprecated on modern setuptools.
• Test scope. Only build_inputs is covered; the sample loop (prev_bev propagation, the saved != num_samples failure path, which currently leaves a partially populated output dir) and quantize.py's defaults are untested. Acceptable for an example, but a small fake-session test of the loop would be cheap.

[gcunhase]

@ajrasane thanks for creating this PR.

Can you please elaborate a bit on what "temporal-state unit coverage" means?

Also, what's the perf expectation between INT8 and FP8? My understanding is that they should be on par but in this example, INT8 is faster than FP8, which matches my own experience with this model too. Do we need to investigate this on ModelOpt (Q/DQ node placement differences between INT8 and FP8) or on TensorRT (kernel improvements / tactic selection in FP8)?

Finally, would it make more sense to just add the FP8 results to https://github.com/NVIDIA/DL4AGX/tree/master/AV-Solutions/bevformer-int8-eq or move it here as you're proposing? If the latter, should we deprecate the workflow on DL4AGX so that there's a single source of truth?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.41%. Comparing base (67a8657) to head (4c232d7).

Additional details and impacted files

@@              Coverage Diff               @@
##           ajrasane/petr    #2208   +/-   ##
==============================================
  Coverage          78.41%   78.41%           
==============================================
  Files                523      523           
  Lines              60903    60903           
==============================================
  Hits               47756    47756           
  Misses             13147    13147           

Flag	Coverage Δ
examples-torch_onnx	21.76% <ø> (-0.03%)	⬇️
examples-torch_trt	15.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ajrasane]

@gcunhase,

“Temporal-state unit coverage” means unit tests for BEVFormer’s frame-to-frame input bookkeeping.
The test simulates three frames:

• First frame of scene A: disables history and zeroes motion deltas.
• Second frame of scene A: enables history and calculates translation/yaw deltas.
• First frame of scene B: disables history and zeroes deltas again.

Regarding the perf difference, I think it would be best to loop in the TensorRT team for debugging this. For now, I will keep the FP8 measurements and add a TODO for investigating the perf differences.
I don't have a strong opinion on keeping 2 copies of this example or keeping one source of truth. At the moment I think the environment setup of this as well as the PETR and Far3d environments is a bit convoluted because we try to reuse functionality from the DL4AGX repo. I was thinking of keeping this example independent of that code and simplifying the dependencies a bit.