Use hardware skinning esp. for animated meshes

[ExeVirus]

Currently, on mobile devices and desktops, animated meshes are limited excessively by CPU, and not GPU. For example, when I load up 15 animated waterwheels with 120 faces each, Mobile (1.2 Ghz CPU, 400 MHz GPU with 128 KB ram) FPS drops to 20 from 28.

When I do the same amount of animated windmill wheels with 700 faces each, The fps drops to the same exact 20 fps. Therefore animated meshes are CPU bound and not GPU bound.

Now, Looking at our .cpp code, I see a lot of processing is being done with each step of an animated mesh (entity). Most of this is highly related to texture changing and updating.

Why have we not move these calculations to GPU processing instead? Irrlicht supports animated meshes with textures on GPU already, so I am wondering why this has not been implemented yet?

I have some experience in GPU work, so I can help a core dev with the project, but I doubt I can make the needed changes alone.

[paramat]

why this has not been implemented yet?

Assuming there is nothing preventing that, i expect the reason is because MT was written in a CPU-orientated way (many aspects of MT are like this) and no-one has changed that yet.
Many of the current active core devs are not too good with rendering stuff, so i wonder if we could ask celeron55 for thoughts?

[hecktest]

I don't understand, is this about tile animations or skinning?

[ExeVirus]

Skinning specifically. If one doesn't change the texture, and doesn't update the animation (via set animation), there should be minimal draw time processing (if any) because that is normally done on the GPU end. The issue is, minetest does a lot of calculation each step specifically for animated meshes, when it probably shouldn't be.

…

On Thu, Dec 19, 2019, 3:27 AM hecktest ***@***.***> wrote: I don't understand, is this about tile animations or skinning? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#9218>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKT7N2VY7SRA2T5NKPE63DDQZMV7PANCNFSM4J4RPNXA> .

[hecktest]

I agree that skinning should be done with a vertex shader.
But this will not eliminate the overhead of animating the transforms. There are ways to delegate that to the GPU too, but it's painful on anything less than GL 4.6

And skinning is still not the worst bottleneck of the client env; GUI issues, poor culling and lack of LOD are far worse.

[ExeVirus]

Hecktest, I'm relatively green when it comes to GPU processing. (Though I am learning). Thoughts:

1. LOD makes perfect sense in terms of making things run faster, how would this be implemented generally?
2. Poor culling surprises me, is this not handled by the Irrlicht engine?
3. What kind of GUI issues?
4. So you would recommend we stay away from GPU controlled animation transforms correct?

[ExeVirus]

Also, further update: I think the problem is not actually the animated meshes themselves, as I went deep into irrlicht and found that those are animated via mostly GPU already. Instead, what I found is I think every global step, animated meshes have their visual-data fully invalidated and are therefore re-initialized every step. I.e. If there are 15 animated meshes in view, every game step the game deletes 15 meshes and tries to initialize 15 meshes.

[hecktest]

GPU evaluated animation curves are fine but it will only benefit a narrow target, only help games that use a lot of skinned meshes (not many of those around) and there are more pressing things to work on. So you won't find many folks willing to take care of that. And that's assuming irrlicht isn't handling it efficiently already. A better reason to rewrite animation would be to fix bugs and provide new features.

LOD for the map itself is a very complex task. It would probably require some form of mipping for far mapblock data. This is something that's sorely needed but it's a gargantuan task, reaching as deep as the map storage format itself. But there's hundreds of thousands of vertices to save here, while allowing a greater draw distance, so it's worth doing.

LOD for entities would probably involve extra models provided by the modder, or an imposter system for very far things. Since this only cuts down on the skinned vertex count, and we're not bound by that, as you've noticed, this also isn't worth working on right now.

The GUI issues I've mentioned involve simple HUD elements like statbars, text or the radar cutting the framerate in half for no good reason. Those things have no right to eat up so many miliseconds! Something's rotten on that end and it would be a good idea to investigate.

Occlusion culling is not handled by irrlicht; irrlicht has no knowledge of the kind of voxel world that minetest renders, and no idea how to cull it. The current culling has its problems and leaves a lot of dead geometry, often drawing twice or three times as much as is needed.
I do believe the culler lives here... it's pretty crude.
https://github.com/minetest/minetest/blob/53ebd803c613949a3f30fb4a905192c85dc73851/src/map.cpp#L1131

Please elaborate on the "deleting and reinitializing 15 meshes every frame" bit. What exactly is happening that shouldn't be? Something like VBOs being deleted and remade?

[lhofhansl]

FWIW, I tried GPU skinning in MT using Irrlicht some weeks ago, and result was garbage... The mesh was there, but distorted and in the wrong places, faces were oriented the wrong way, etc, etc. I concluded that it's not ready for use, yet.

That said... I only spent a few hours on it, so it's possible that I missed something.

I did see a marked performance increase.

[hecktest]

If your mesh was distorted, then you either forgot to invert the bindposes or you inverted them twice. Had the same problem trying to build .x meshes. Link me to your branch for this and I'll have a look.

[paramat]

For world LOD see #3502 , there are some videos on sofar's youtube channel.

[lhofhansl]

Hey @hecktest, I got rid of the code :(
Lemme see what I can remember and I'll create a branch and point you to it.

Note, I did nothing other than turning on Irrlicht's HW skinning, assuming Irrlicht would do the rest :)

Edit: Just this (as I said, I didn't try very hard).

diff --git a/src/client/content_cao.cpp b/src/client/content_cao.cpp
index ce0e3183..e506947e 100644
--- a/src/client/content_cao.cpp
+++ b/src/client/content_cao.cpp
@@ -23,6 +23,7 @@ with this program; if not, write to the Free Software Foundation, Inc.,
 #include <ITextSceneNode.h>
 #include <IMeshManipulator.h>
 #include <IAnimatedMeshSceneNode.h>
+#include <ISkinnedMesh.h>
 #include "client/client.h"
 #include "client/renderingengine.h"
 #include "client/sound.h"
@@ -686,6 +687,11 @@ void GenericCAO::addToScene(ITextureSource *tsrc)
                infostream<<"GenericCAO::addToScene(): mesh"<<std::endl;
                scene::IAnimatedMesh *mesh = m_client->getMesh(m_prop.mesh, true);
                if (mesh) {
+                       if (mesh->getMeshType() == scene::E_ANIMATED_MESH_TYPE::EAMT_SKINNED) {
+                               scene::ISkinnedMesh *sm = static_cast<scene::ISkinnedMesh *>(mesh);
+                               sm->setHardwareSkinning(true);
+                       }
+ 
                        m_matrixnode = RenderingEngine::get_scene_manager()->
                                addDummyTransformationSceneNode(nullptr);
                        m_matrixnode->grab();

[lhofhansl]

@celeron55 worked on "farmaps" a long time ago (akin to LOD, though not the same).