OpenCV 5 Adds Native GPT, Gemma And Qwen Model Support

0
1
OpenCV 5 Rebuilds Its AI Stack With New DNN Engine, Expands ONNX Coverage Beyond 80%
OpenCV 5 Rebuilds Its AI Stack With New DNN Engine, Expands ONNX Coverage Beyond 80%

OpenCV has released OpenCV 5 with a rebuilt DNN engine, native LLM and VLM support, and expanded ONNX compatibility. The update strengthens the open-source computer vision platform’s position as a broader AI inference framework across Intel, Arm, Qualcomm and RISC-V hardware.

OpenCV has released OpenCV 5, introducing a completely rebuilt Deep Neural Network (DNN) engine that significantly expands the capabilities of the open-source computer vision library as an AI inference platform.

The new graph-based DNN engine increases ONNX operator coverage from about 22% in OpenCV 4.x to more than 80%, enabling support for dynamic and symbolic shapes, subgraphs, operator fusion, shape inference and constant folding. As a result, many AI models that previously failed to run in OpenCV can now execute without extensive modification.

OpenCV 5 also adds native support for large language models (LLMs) and vision-language models (VLMs), allowing models such as Qwen 2.5, Gemma 3, PaliGemma, and GPT-family models to run directly within the DNN module. Native tokenisation and KV-cache support eliminate the need for a separate runtime.

Performance benchmarks on an Intel Core i9-14900KS system showed OpenCV 5 outperforming ONNX Runtime across multiple models, with gains ranging from 4.4% to 36.6%.

The release further introduces a redesigned Hardware Acceleration Layer based on Universal Intrinsics 2.0, enabling optimisation across Intel IPP, Arm KleidiCV, Qualcomm FastCV and RISC-V Vector architectures. Arm implementations reportedly achieve up to 3-4x faster performance on operations such as resizing and warping.

Additional enhancements include FP16 and BF16 data types, real N-dimensional support, up to 2x faster mathematical workloads, modernised C++ and Python support, and a new 3D vision framework targeting robotics, reconstruction and SLAM applications.
The OpenCV 5 source code is available on GitHub.

LEAVE A REPLY

Please enter your comment!
Please enter your name here