PaddleSpeech 1.5.0 Release Note

[luotao1]

Full Changelog: 748a5f9...develop 18 contributors

Version Adaptation

Upgrade and adapt PaddleSpeech from Paddle 2.5.1 to Paddle 3.0.0-beta. Address incompatibility issues caused by the new version upgrade of Paddle, perform adaptation development and regression testing on the models in PaddleSpeech, and ensure the suite operates normally without loss of model functionality or accuracy.

• Ensure the adaptation of 80+ existing models in the demo and example directories.
• Ensure the adaptation and accuracy alignment of 10+ core models in the example directory.
• Support the re-export of 20+ dynamic-to-static models using the PIR + predictor approach and ensure successful inference.

New Features

• Implement the third-party library audio tools used in DAC (Descript-Audio-Codec) training.
• Reproduce the losses required for DAC training: MultiScaleSTFTLoss, GANLoss, and SISDRLoss.

Bug Fix

Others

• Clean up dependencies and support using PaddleSpeech in Python>3.8 environments

Acknowledgements

Special thanks to contributors including @wanx7130 @warrentdrew @DrRyanHuang @cchenhaifeng @undefined-ux @zxcd @GreatV @yinfan98 @Liyulingyue @megemini @SuiYunsy @Netrvin @enkilee @tianshuo78520a and others for their support.

New Contributors

• @wanx7130 made their first contribution in Add GCU Backend #3875
• @cchenhaifeng made their first contribution in 【Hackathon 8th No.9】在 PaddleSpeech 中复现 DAC 训练需要用到的 loss #3988
• @undefined-ux made their first contribution in 添加量词和单位符号 #3837
• @DrRyanHuang made their first contribution in [Hackathon 7th No.55] Add audiotools to PaddleSpeech #3900
• @SuiYunsy made their first contribution in Fix a bug of streaming_tts_server #3865
• @Netrvin made their first contribution in fix pydantic dependency #3715
• @guspan-tanadi made their first contribution in docs: text frontend intended links #3958
• @enkilee made their first contribution in [Hackathon 7th] 修复 opencopop的svs1中安装依赖包问题 #3889

[GreatV]

Hardware Support

• Add GCU Backend support Add GCU Backend #3875 (@wanx7130)
• SpeedySpeech code adaptation for NPU speedyspeech code adapt for npu #3804 (@warrentdrew)
• SpeedySpeech code adaptation for MLU speedyspeech code adapt for mlu #3828 (@warrentdrew)

New Models and Features

• Add AudioTools toolkit [Hackathon 7th No.55] Add audiotools to PaddleSpeech #3900 (@DrRyanHuang)
• Add FFT convolution layer implementation [Hackathon 7th No.55] Add fft_conv1d to PaddleSpeech -part #3947 (@DrRyanHuang)
• Implement loss functions required for DAC training 【Hackathon 8th No.9】在 PaddleSpeech 中复现 DAC 训练需要用到的 loss #3988 (@cchenhaifeng)
• Add quantifiers and unit symbols support 添加量词和单位符号 #3837 (@undefined-ux)
• Add multiple PIR models 【TTS】add some PIR model #3956, add some pir model #3982 (@zxcd)
• Separate paddle.logsumexp 【CI】separate paddle.logsumexp #3897 (@zxcd)

Compatibility and Optimization

• Enhance NumPy compatibility NumPy compatibility enhancements #3907 (@GreatV)
• Fix Whisper model support under Paddle 3.0 【Hackathon 7th】fix whisper at Paddle 3.0 #3880 (@yinfan98)
• Remove dependency on paddlepaddle-gpu Remove Dependence of Paddlepaddle-gpu #3898 (@Liyulingyue)
• Remove paddleaudio from PaddleSpeech 【audio】remove paddleaudio from paddlespeech #3986 (@zxcd)
• Support new inference interface support new inference interface  #3927 (@zxcd)
• Add chunk configuration for tal_cs 【asr】add chunk config for tal_cs #3936 (@zxcd)
• Lower installation requirements lower the install requirements #3985 (@Liyulingyue)
• Modify inference to be compatible with Paddle 3.0 [Hackathon 7th] 修改 inference 兼容 paddle 3.0 #3963 (@megemini)
• Optimize Python version compatibility 【Hackathon 8th No.7】Python版本适配 1 #3965, 【Hackathon 8th No.7】Python版本适配 2 #3967, 【Hackathon 8th No.7】Python版本适配 3 #3969, 【Hackathon 8th No.7】Python版本适配 4 #3970, 【Hackathon 8th No.7】Python版本适配 5 #3972 (@Liyulingyue)
• Define PythonDetermine in setup.py def PythonDetermine in setup.py #3975 (@Liyulingyue)
• Adapt for librosa fit with librosa #3989 (@Liyulingyue)
• Modify basemodel.py 【audiotools】remove package load and save  #3994 (@zxcd)

Bug Fixes

• Fix type promotion issues [Fix] 修改 type promotion 以兼容 paddle 3.0 版本 #3817, fix type promotion TypeError in tensor_utils.py #3883, [Hackathon 7th] 修复 voxceleb 的类型提升问题 #3944, [Hackathon 7th] 修复 vctk 的 ernie_sat 训练时出现的类型提升问题 #3943 (@megemini @GreatV)
• Fix shape error in layer normalization [Hackathon 7th]fix the shape error in layer_norm #3884 (@Liyulingyue)
• Resolve scipy import error fix scipy import error #3874 (@GreatV)
• Fix streaming TTS server issues Fix a bug of streaming_tts_server #3865 (@SuiYunsy)
• Fix matplotlib version incompatibility 【TTS】fix matplotlib version for incompatible upgrade #3841 (@zxcd)
• Fix pydantic dependency issues fix pydantic dependency #3715 (@Netrvin)
• Fix 0D tensor to 1D issue [Hackathon 7th] 修复 tal_cs 测试中 0D tensor to 1D #3913 (@megemini)
• Fix CSMSC vits [Hackathon 7th] fix vits with CSMSC #3920 (@Liyulingyue)
• Fix fastspeech2 0d issue [Hackathon 7th] 修复 fastspeech2 的 0d 问题 #3951 (@megemini)
• Fix emb initialization [Hackathon 7th] 修复 Embedding 层的初始化参数 #3962 (@megemini)
• Fix max between int and value [Hackathon 7th] 修复 int 与 Value 取 max 的问题 #3903 (@megemini)
• Replace view with reshape replace view with reshape in aishell/asr1 #3887, [Hackathon 7th] 使用 reshape 代替 view #3939 (@GreatV, @megemini)
• Fix audiotools file path 【audio】fix file path of audiotools. #3968 (@zxcd)
• Fix duplicated argument [Hackathon 7th] 修复 asr4 测试命令中重复的 arg #3934 (@megemini)
• Fix vctk spk_emb dimension issue [Hackathon 7th] 修复 vctk 中 spk_emb 维度问题 #3916 (@megemini)
• Fix aishell3/vctk vc0/ernie [Hackathon 7th] fix aishell3/vctk vc0/ernie #3928 (@Liyulingyue)
• Add missing keywords for aishell3/vits-vc [Hackathon 7th] add missing keywords of aishell3/vits-vc #3932 (@yinfan98)
• Fix data traversal error caused by empty folders without *.npy files [Hackathon 7th] 修复不存在 *.npy 文件的空文件夹导致的数据遍历错误 #3948 (@megemini)
• Use numpy for transpose [Hackathon 7th] 修复 paddle 3.0 transpose 的输入不能为 tensor 的问题 #3933 (@megemini)
• Fix package dependency issues in opencopop svs1 [Hackathon 7th] 修复 opencopop的svs1中安装依赖包问题 #3889 (@enkilee)
• Fix shape issues in opencopop svs1 [Hackathon 7th] 修复 opencopop的svs1中的shape问题 #3912 (@enkilee)

Documentation and Examples

• Add Squeezeformer information to README 【doc】add Squeezeformer info in README #3860 (@zxcd)
• Add README documentation for TIMIT/ASR1 [hackathon7] Add timit/asr1 readme.md #3930 (@enkilee)
• Fix multiple examples and demos 【DOC】fix demos bug #3830, Fix wav2vec error in Demos/ssl #3872, 【PaddleSpeech No.4】补全合成系列中的脚本中参数缺失 #3998 (@zxcd @Liyulingyue @enkilee)
• Complete missing parameters in synthesis series scripts 【PaddleSpeech No.4】补全合成系列中的脚本中参数缺失 #3998 (@enkilee)
• Fix tess readme [Hackathon 7th] 修复 tess readme #3882 (@megemini)
• Update README.md Fix missing xzvf #3890 (@Liyulingyue)
• Fix Example/tiny documentation errors [Hackathon 7th] 修复Example/tiny的文档错误 #3892 (@Liyulingyue)
• Update tal_cs readme [Hackathon 7th] 修复 tal_cs 中的 readme #3911 (@megemini)
• Fix librispeech asr readme [Hackathon 7th] 修复 librispeech 中 asr 的 readme #3917 (@megemini)
• Fix CSMSC voc1 readme.md [Hackathon 7th] fix voc1 readme.md in CSMSC  #3915 (@enkilee)
• Fix s2t example errors [Hackathon 7th] 修复 s2t 示例错误 #3950 (@megemini)
• Fix led_en_zh st1 example [Hackathon 7th] fix example led_en_zh st1 #3955 (@GreatV)
• Text frontend intended links docs: text frontend intended links #3958 (@guspan-tanadi)
• Update Tiny README.md [Hackathon 7th] Update Tiny README.md #3896 (@Liyulingyue)
• Fix acs demo 【ASR】fix acs demo #3826 (@zxcd)

Scripts and Configuration Fixes

• Fix g2p run.sh [Hackathon 7th] 修复 g2p run.sh 中错误的命令 #3886 (@megemini)
• Fix deepspeech2online export issue [Hackathon 7th] 修复 deepspeech2online 的导出问题 #3935 (@Liyulingyue)
• Fix asr5 test.sh script path error [Hackathon 7th] 修复 asr5 的 test.sh 脚本路径错误 #3941 (@megemini)
• Fix asr4 test_wav redundant arguments [Hackathon 7th] 修复 asr4 的 test_wav 多余的 argument #3940 (@megemini)
• Add synthesize_e2e.sh for csmsc/voc1, fix run.sh [Hackathon 7th] 添加csmsc/voc1的synthesize_e2e.sh，修复run.sh #3945 (@enkilee)
• Add synthesize_e2e.sh for csmsc/voc5, fix run.sh [Hackathon 7th] 添加csmsc/voc5的synthesize_e2e.sh，修复run.sh #3959 (@enkilee)
• Fix CSMSC Voc5/Jets/TTS2 [Hackathon 7th] fix Voc5/Jets/TTS2 with CSMSC #3906 (@Liyulingyue)
• Update utility script paths [Hackathon 7th] updating the paths to utility scripts and modifying method parameters #3942 (@GreatV)
• Fix panns predict.py for pir json model path [Hackathon 7th] 修复 panns 中 predict.py 对于 pir 的 json 模型路径 #3914 (@megemini)

Engineering and Development

• Add server CI Add server ci #3857 (@tianshuo78520a)
• Add hints for installing with -e option 【Hackathon 8th No.7】Add hints for installing with -e #3979 (@Liyulingyue)
• Move audiotools requirements to setup.py 【audiotools】mv audiotools requirement to setup.py #3999 (@zxcd)
• Add unit tests 【Test】fix unit test #3835, Add tests #3836 (@zxcd, @tianshuo78520a)
• Update install_openblas.sh update install_openblas.sh #3876 (@GreatV)
• Add strtobool implementation 【Hackathon 7th】add implemention of strtobool #3877 (@Liyulingyue)
• Remove parser.add_argument 【Hackathon 7th】Remove parser.add_argument #3878 (@Liyulingyue)
• Remove non-existent folders and add existing folders Remove None-Existed floder and add Existed floder #3881 (@Liyulingyue)
• Fix file name fix file name #3895 (@zxcd)
• Update setup.py Remove Py3.7 in setup.py #3964, Update librosa version #3995 (@Liyulingyue)
• Fix typos Fix typos #3984 (@co63oc)
• Fix missing ' Fix missing ’ #3869 (@Liyulingyue)
• Import print_arguments [Hackathon 7th] 修复导入 print_arguments #3918 (@megemini)
• Import TimeDomainSpecAugment [Hackathon 7th] 修复导入 TimeDomainSpecAugment #3919 (@megemini)
• Run with aishell/asr3 [Hackathon 7th] 修复aishell/asr3运行脚本 #3904 (@Liyulingyue)

General Fixes

• Fix issues Fix #3821, 【cls】fix cls static model infer error. #3856, Close test_expand #3971, Fix tests/unit/tts/test_pwg.py #3974, Fix test_snapshot.py #3976, Fix typos #3980, Fix typos #3981, [Hackathon 7th] 添加csmsc/voc3的synthesize_e2e.sh，修复run.sh #3960 (@tianshuo78520a, @zxcd, @co63oc, @enkilee)

Acknowledgements

Special thanks to contributors including @wanx7130, @warrentdrew, @DrRyanHuang, @cchenhaifeng, @undefined-ux, @zxcd, @GreatV, @yinfan98, @Liyulingyue, @megemini, @SuiYunsy, @Netrvin, @enkilee, @tianshuo78520a, @guspan-tanadi, @co63oc and others for their support.

New Contributors

• @wanx7130 made their first contribution in Add GCU Backend #3875
• @cchenhaifeng made their first contribution in 【Hackathon 8th No.9】在 PaddleSpeech 中复现 DAC 训练需要用到的 loss #3988
• @undefined-ux made their first contribution in 添加量词和单位符号 #3837
• @DrRyanHuang made their first contribution in [Hackathon 7th No.55] Add audiotools to PaddleSpeech #3900
• @SuiYunsy made their first contribution in Fix a bug of streaming_tts_server #3865
• @Netrvin made their first contribution in fix pydantic dependency #3715
• @guspan-tanadi made their first contribution in docs: text frontend intended links #3958
• @enkilee made their first contribution in [Hackathon 7th] 修复 opencopop的svs1中安装依赖包问题 #3889
• @co63oc made their first contribution in Close test_expand #3971

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.