Reshape Input for Dynamic Batch Size

[akurniawan]

Based on this tutorial http://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html, we need to specify the batch_size while exporting the model from pytorch to onnx. In some cases we need a dynamic batch_size for inference, do you have any advice of how we can do this?

[ezyang]

This is a PyTorch export limitation. One idea we had for relaxing this restriction was this: you export the model twice with two different batch sizes. Then, looking at the dimensions which changed, you generalize your ONNX model accordingly. This code doesn't exist yet but it seems reasonably likely to work. One potential difficulty is that the symbolic batch size, while supported in ONNX spec, is not very well tested.

[akurniawan]

Sorry, I still don't really get it by "export the model twice with two different batch sizes". I need to create two different protos with different batch sizes? But that would result still in static batch size right? Do you mind to elaborate more?