OpenVINO and ONNX: What You Need to Know About Supported Operations

As a developer working with deep learning models, you’re likely familiar with OpenVINO, an open-source library that allows you to optimize and run your models on a variety of devices. But did you know that OpenVINO doesn’t support all ONNX operations? In this article, we’ll dive into the specifics of what’s not supported, and what you can do about it.

The Importance of ONNX Operations in OpenVINO

ONNX (Open Neural Network Exchange) is an open format used to represent deep learning models. It’s widely adopted across various frameworks and libraries, including OpenVINO. When you convert your model to the ONNX format, you expect it to be compatible with OpenVINO. However, that’s not always the case.

OpenVINO supports a vast majority of ONNX operations, but there are some exceptions. Understanding which operations are not supported is crucial to avoid compatibility issues and ensure seamless model conversion.

SequenceConstruct and ConcatFromSequence: The Unsupported Operations

So, what are these mysterious operations that OpenVINO doesn’t support? Let’s take a closer look:

• SequenceConstruct: This operation creates a sequence from a list of tensors. It’s often used in natural language processing (NLP) models to construct sequences of words or characters.
• ConcatFromSequence: As the name suggests, this operation concatenates tensors from a sequence. It’s commonly used in models that require concatenating outputs from different layers.

These two operations are not supported by OpenVINO, which means that if your model uses them, you’ll encounter compatibility issues during the conversion process.

Why Are These Operations Not Supported?

There are several reasons why OpenVINO doesn’t support SequenceConstruct and ConcatFromSequence:

1. Limited Hardware Support: These operations are not efficiently supported by most hardware accelerators, making it challenging for OpenVINO to optimize them.

2. Complexity: SequenceConstruct and ConcatFromSequence are complex operations that require significant computational resources. OpenVINO’s primary focus is on optimizing models for inference, and these operations don’t fit well into that scope.

3. Alternative Solutions: In many cases, you can achieve similar results using alternative ONNX operations that are supported by OpenVINO. We’ll explore some of these alternatives later in this article.

How to Handle Unsupported Operations

So, what can you do if your model uses SequenceConstruct or ConcatFromSequence? Don’t worry; there are ways to work around these limitations:

Model Refactoring

One approach is to refactor your model to avoid using these operations. This might require significant changes to your model architecture, but it’s a viable solution if you’re willing to put in the effort.

  
  # Original model architecture
  seq_construct = onnx.helper.make_node('SequenceConstruct', 
                                        inputs=['input_1', 'input_2'], 
                                        outputs=['output'])

  # Refactored model architecture
  concat_node = onnx.helper.make_node('Concat', 
                                      inputs=['input_1', 'input_2'], 
                                      outputs=['output'])
  

Using Alternative ONNX Operations

Another approach is to use alternative ONNX operations that achieve similar results. For example, you can use the Concat operation instead of ConcatFromSequence. Similarly, you can use Stack or Unstack operations to replace SequenceConstruct.

Unsupported Operation	Alternative Operation
SequenceConstruct	Stack or Unstack
ConcatFromSequence	Concat

Custom OpenVINO Plugins

If you’re unwilling or unable to refactor your model, you can create custom OpenVINO plugins to support the unsupported operations. This approach requires a deep understanding of OpenVINO’s internal workings and the capabilities of the target hardware.

By creating custom plugins, you can extend OpenVINO’s functionality to support SequenceConstruct and ConcatFromSequence. However, this approach is typically reserved for advanced users and requires significant development effort.

Conclusion

OpenVINO’s lack of support for SequenceConstruct and ConcatFromSequence might seem like a limitation, but it’s essential to understand the reasoning behind this decision. By refactoring your model or using alternative ONNX operations, you can work around these limitations and ensure seamless model conversion.

Remember, OpenVINO is a powerful tool that can optimize and run your deep learning models on a wide range of devices. With a little creativity and flexibility, you can overcome the limitations imposed by unsupported operations.

So, go ahead and explore the world of OpenVINO and ONNX. With this knowledge, you’ll be well-equipped to tackle even the most complex model conversions!

Frequently Asked Question

Get your questions answered about OpenVINO’s ONNX operation limitations!