reshape output performance improve advice

[YeWenxuan64]

you can store the output shape to aviod a function calling

  def __init__(self, 巴拉巴拉什么的):
     self.outputshape_list = self.getOutputShapes()
     self.output_num = len(outputshape_list )

  def Inference(self, input, perf_profile = PerfProfile.DEFAULT, graphIndex = 0):
      output = self.m_context.Inference(input, perf_profile, graphIndex)
      
      for i in range(self.output_num):
          output[i].reshape(outputshape_list[i]) # although it reshapes the outputs but numpy.reshape returns a view

      return output

although it reshapes the outputs but numpy.reshape returns a view, not a C-continuous array, may cause so problem in downstream processing.（但其实大多数处理还是要再reshape一次，所以C不C连续都没什么问题）
so please reshape the outputs in C/C++ level, (pybind is suitable for connecting C/C++ to numpy)

[quic-zhanweiw]

@YeWenxuan64

Could you please share the detail about the issues in above implementation? I think reshape only modified the meta shape data in the numpy struct in C level, it will not modify the data in the array.
If really has issue here, may you help share a patch for this? Thanks in advance!

[YeWenxuan64]

i tried to put a reshape view in qnncontext.inference like this, then the inference is wrong.

input_data = [a_array.reshape((1,96,16,16))]
qnncontext.inference(input_data)

and tranpose view

input_data = [a_array.tranpose(0,3,1,2)] # nhwc to nchw
qnncontext.inference(input_data)

one day i make it C-continuous, then the inference correct

input_data = [a_array.tranpose(0,3,1,2).copy(order='C')] # nhwc to nchw
qnncontext.inference(input_data)

so i think return a view could cause some problem when put last inference's output data in another inference

output_a = qnncontext_a.inference(input_data) # A unlucky person who doesn't know the return is a view
output_b = qnncontext_b.inference(output_a)

so i think we need to notice the inference returns a view

---

but also i think it doesn't matter, bucause many model's post-processing need to use slice or reshape, like remove the batch-dim (i only use computer-vision model) , and in numpy's world: numpy will handle all of the view-array convention.

---

last but not least, i still advice put the CPU-intensive tasks in C/C++ level, if add a ‘“return a reshaped C-continuous array” function. because it needs one times of copy.

[quic-zhanweiw]

Thanks @YeWenxuan64 for your update! We'll study this.

[tim202503]

Our current reshape operation in inference does not need copy, just returns a view. As you mentioned, many model's post-processing still need to use slice or reshape besides our default reshape.

[YeWenxuan64]

self.outputshape_list = self.getOutputShapes()

my final suggestion：move the self.outputshape_list = self.getOutputShapes() out of the Inference() method to reduce 400 micro second “function calling” performance.