Question about expected matrix order in iplInstancedMeshUpdateTransform

[mellinoe]

I'm currently trying to integrate Steam Audio in my engine. I've added support for static geometry without much of an issue, and am moving on to dynamic objects using IPLInstancedMeshes. I've been using iplSceneSaveOBJ as an easy way to reflect an IPLScene's state back into my editor view, and it has been very helpful for static meshes, to make sure they are placed correctly. If I feed my engine's row major local-to-world matrices into iplInstancedMeshUpdateTransform, the OBJ file mesh data looks good and perfectly matches the engine's mesh view. However, the occlusion results don't match, and seem to act as if the mesh is at the origin (e.g. an identify instance transform matrix).

If I transpose the matrix before passing it to iplInstancedMeshUpdateTransform, the audio occlusion seems to work properly, but the dumpOBJ mesh data no longer looks correct. According to the header file, an IPLMatrix4x4 is row major, so I wouldn't expect to need to transpose it. Stepping through the implementation a bit, I see that iplInstancedMeshUpdateTransform immediately transposes the input matrix, and Scene::dumpObj also transposes the instance matrix before dumping vertex data. What is the expected format here?

Version: 4.6.1

[SkacikPL]

I can only answer how it works in UE but static geometry is exported with its final "in engine" transforms whilst dynamic objects are always exported with identity transform (0,0,0 location, 0 rotation, 1 scale) and they're updated each tick using transform of their in-engine equivalent.

This does reflect in your findings that static scene does match but importing dynamic objects does not.

[lakulish]

@mellinoe There's definitely a bug here, in Scene::dumpObj, it should not be transposing the matrix. I will commit a fix for that. However, when I tried to reproduce the issue, and passed a row-major matrix to iplInstancedMeshUpdateTransform, I got correct occlusion and an incorrect .obj file, which is consistent with the bug being in Scene::dumpObj but the opposite of what you are observing. Can you double-check whether the matrix you are passing is in fact row-major?

[mellinoe]

@lakulish Thank you. If Scene::dumpObj is fixed then my project will be in a working state, but yes it's strange that it is necessary to transpose the matrix to get correct results.

I'm using the 4x4 matrix type in the .NET standard library, which is documented here: https://learn.microsoft.com/en-us/dotnet/api/system.numerics.matrix4x4?view=net-9.0. To be even more specific, here's the raw layout for an example matrix I'm using and trying to pass as-is to iplInstancedMeshUpdateTransform:

// Translation: 7.95, 0, -2.48
// rotation: -49 degrees around Y
// Scale: 2, 2, 2

1.3046997, 0, 1.515836, 0
0, 2, 0, 0
-1.515836, 0, 1.3046997, 0
7.95, 0, -2.48, 1

[lakulish]

@mellinoe Ah, I see what the mismatch is. Steam Audio uses the convention (like OpenGL) where in a matrix-vector multiplication, the vector is a column vector, i.e. multiplying a matrix M with a vector v is Mv. On the other hand, .NET uses the convention (like Direct3D) where the vector is a row vector, i.e., multiplying a matrix T with a vector v is vT. The matrix used in the Direct3D/.NET convention is the transpose of the matrix used in the OpenGL/Steam Audio convention. So although the matrix is actually stored in row-major order in .NET, it is the transpose of the matrix that you need to pass to Steam Audio, so you'll want to transpose the matrix obtained from .NET before passing it to any Steam Audio API functions.