Struggling to get asset replacement to work

[SurDno]

Heya! So I am currently trying to get a simple tool that will iterate through mod folders (named the same way as actual assets files they are trying to modify) and for each file inside, will try to replace it in the assets file. I went through the docs but it did not cover asset replacement unfortunately, and most projects that use AssetTools.NET seem to use 2.0 or whatever version still had AssetsReplacer class. I decided to start with audioclips cause with textures I'll have to care about formatting and so many more things and it all seems unnecessarily complicated.

The code below works (hooray!) but it corrupts the assets file in the process and neither the game nor stuff like AssetStudio can open it after that. And I've got no idea what I am doing wrong.

using AssetsTools.NET;
using AssetsTools.NET.Extra;

namespace P2ModLoader.Helper {
    public static class AssetsFilePatcher {
        public static bool PatchAssetsFile(string assetsFilePath, string modAssetsFolder) {
            AssetsManager? manager = null;
            AssetsFileInstance? assetsFileInstance = null;

            // Create a copy for modification
            var newFile = assetsFilePath + ".temp";
            File.Copy(assetsFilePath, newFile, true);

            try {
                manager = new AssetsManager();
                manager.LoadClassPackage("C:/classdata.tpk");

                using var fs = new FileStream(
                    newFile,
                    FileMode.Open,
                    FileAccess.ReadWrite, 
                    FileShare.None 
                );
                assetsFileInstance = manager.LoadAssetsFile(fs, newFile);
                var assetsFile = assetsFileInstance.file;

                manager.LoadClassDatabaseFromPackage(assetsFile.Metadata.UnityVersion);

                var modAssetFiles = Directory.GetFiles(modAssetsFolder, "*.*", SearchOption.AllDirectories);
                foreach (var modAssetFile in modAssetFiles) {
                    var assetName = Path.GetFileNameWithoutExtension(modAssetFile);
                    var assetData = File.ReadAllBytes(modAssetFile);
                    var extension = Path.GetExtension(modAssetFile).ToLower();

                    switch (extension) {
                        case ".wav" or ".ogg" or ".mp3" or ".aif" or ".aiff": {
                            if (!ReplaceAudioAsset(manager, assetsFileInstance, assetName, assetData)) {
                                Console.WriteLine($"Failed to replace audio asset: {assetName}");
                                return false;
                            }
                            break;
                        }
                        default:
                            Console.WriteLine($"Unsupported asset type for file: {modAssetFile}");
                            break;
                    }
                }

                fs.Position = 0; 
                using (var writer = new AssetsFileWriter(fs)) {
                    assetsFileInstance.file.Write(writer, -1);
                }

                fs.Close();

                // Copy over original.
                File.Copy(newFile, assetsFilePath, true);
                Console.WriteLine($"Successfully patched {assetsFilePath}");
                return true;
            } catch (Exception ex) {
                Console.WriteLine($"Error patching assets file: {ex.Message} {ex.StackTrace}");
                return false;
            } finally {
                File.Delete(newFile);
                manager?.UnloadAll();
            }
        }

        private static bool ReplaceAudioAsset(
            AssetsManager manager,
            AssetsFileInstance assetsFileInstance,
            string assetName,
            byte[] assetData) {
            try {
                var afile = assetsFileInstance.file;
                var audioAssets = afile.GetAssetsOfType((int)AssetClassID.AudioClip);

                foreach (var assetInfo in audioAssets) {
                    var baseField = manager.GetBaseField(assetsFileInstance, assetInfo);
                    var name = baseField["m_Name"].AsString;

                    if (name != assetName) continue;
                    
                    var externalFilePath = assetsFileInstance.path.Replace(".temp", "") + ".resS";
                        
                    // Backup external file.
                    BackupManager.CreateBackup(externalFilePath);

                    long newOffset;
                    using (var stream = new FileStream(
                               externalFilePath,
                               FileMode.Open,
                               FileAccess.ReadWrite,
                               FileShare.None)) {
                        newOffset = stream.Length;
                        stream.Position = newOffset;
                        stream.Write(assetData, 0, assetData.Length);
                    }

                    var resourceField = baseField["m_Resource"];
                    resourceField["m_Offset"].AsULong = (ulong)newOffset;
                    resourceField["m_Size"].AsUInt = (uint)assetData.Length;

                    var replacer = new ContentReplacerFromBuffer(baseField.WriteToByteArray());
                    assetInfo.Replacer = replacer;

                    Console.WriteLine($"Successfully replaced asset {assetName}");
                    return true;
                }

                Console.WriteLine($"Audio asset with name {assetName} not found.");
                return false;
            } catch (Exception ex) {
                Console.WriteLine($"Error replacing audio asset: {ex.Message}");
                return false;
            }
        }
    }
}

I understand issues is probably not the best place to seek help but I've got no diea where to go. If I missed something in the doc or you have a working example with assets file patching I would really appreciate it.

[nesrak1]

This seems mostly fine. The only major issue I see is using files like wav/ogg/mp3 in the resource file. Unity takes audio in fmod format, not raw audio files. There is no C# library to re-encode anything into that format right now, so most people use the official native fsb library to do it (but of course, that comes with native library complications).

I'm curious what AssetStudio is showing for the error. Also check with UABEA and see if it prints an error or not (you may need to click view data on the audio clip asset to get it to throw an error.)

If you don't set the replacer but write the file, does it work fine? If it still breaks everything, then it's an AssetsTools issue. If writing with no changes doesn't break anything, it's probably the specific changes you made to the audio clip.

[SurDno]

I doubt it's AssetStudio issue as the game was stuck on loading with the modified .assets file, implicating the file was indeed corrupted from this code. Either way, AssetStudio was giving following exceptions:

When trying to open the same assets file, UABEA just never loaded anything. Seemed to be stuck on non-responding for a while.

I figured it's because the .assets file is 1.5GB in size though so maybe I just never waited long enough to see the error message, so I tried repeating the process on a 20MB package, and after a while it finished (to my surprise) and gave me this:

First 30-40 entries were corrupted like that but the rest of the file seemed fine. No error messages or anything.

Moving onto the code, removing ReplaceAudioAsset() completely led to file being overwritten without being corrupted, but it obviously applied no changes.

After playing around for a bit, I got it to work by removing this code:

                fs.Position = 0; 
                using (var writer = new AssetsFileWriter(fs)) {
                    assetsFileInstance.file.Write(writer, -1);
                }
                
                // Copy over original.
                File.Copy(newFile, assetsFilePath, true);
                Console.WriteLine($"Successfully patched {assetsFilePath}");

and changing externalPath to be acquired from m_Resource to account for cases where the resources may not be in .resS file.

                    string externalFilePath;
                    var resourceField = baseField["m_Resource"];
                    var resourcePath = resourceField["m_Source"].AsString;                        
                    var assetsFilePath = assetsFileInstance.path; 
                    var assetsDirectory = Path.GetDirectoryName(assetsFilePath);

                    if (string.IsNullOrEmpty(resourcePath)) {
                        externalFilePath = assetsFilePath.Replace(".temp", "") + ".resS";
                    } else {
                        externalFilePath = Path.Combine(assetsDirectory, resourcePath);
                    }

Which meant the resources.resources file was updated while resources.assets was left alone. As I am not completely set on the structure of both files now, I assume it would work fine for when I am just replacing assets without adding or removing anything? Or will I bump into issues later if I am not properly overwriting the original assets file? What's the necessity of updating .assets if it's just references to actual data in a different file?

Suprisingly, .wav loaded just fine and was played in the game, but it seems like it got its length wrong (?). When used for replacing a looping music track, it started from the beginning before the actual track was over, about halfway through. I'll look into re-encoding it in fmod format, is there ready-to-use example of doing that? I suppose UABEA AudioClipPlugin handles that?

Also: will I have similar issues when I extend this implementation to support texture replacement? Will I have to convert jpg/png into formats used by Unity? What about meshes, etc?

[nesrak1]

The corrupted asset names are not supposed to happen. On second look of your code, it looks like you are saving the bundle to the stream you have open? One of the things you can't do is save the assets file onto the file that's already open. From the wiki:

Make sure that you save with a different filename than the file currently open. AssetsTools.NET copies unmodified assets from the original file and writing is not possible when the original file is already open for reading.

In this scenario, it looks like asset data was rearranged and all assets up to the location where your new asset got stored was corrupted.

Which meant the resources.resources file was updated while resources.assets was left alone. As I am not completely set on the structure of both files now, I assume it would work fine for when I am just replacing assets without adding or removing anything? Or will I bump into issues later if I am not properly overwriting the original assets file? What's the necessity of updating .assets if it's just references to actual data in a different file?

If you edit only the .resource file to edit audio, you wouldn't need to touch the assets file assuming the audio data is the exact same size as the original. Most of the time it's not the same, so you'd need to edit the assets file to modify the offset and size into the resource file.

is there ready-to-use example of doing that? I suppose UABEA AudioClipPlugin handles that?

UABEA's plugin doesn't support writing because it uses Fmod5Sharp which is all C# but decode only. I know there are non-unity/non-official fmod gui tools that can do it. I haven't used it, but one I've seen going around is fmod bank tools and there was also another game on gamebanana I've seen has their own command line tool.

Also: will I have similar issues when I extend this implementation to support texture replacement? Will I have to convert jpg/png into formats used by Unity? What about meshes, etc?

The only format that unity uses that isn't an engine specific format are fonts (they can use ttf/otf but don't have to). Everything else from textures to meshes will use a non-traditional format. For texture encoding, you can use AssetsTools.NET.Texture. Check the dev branch if you want to see the in-progress port of UABEA's old texture plugin to stock AssetsTools.NET. As for meshes, nothing exists to turn an fbx or similar complex format into a unity mesh. Most people build a project with a mesh inside and copy the asset dump from the built game into the file they want to mod.