Skip to content

Conversation

@zeroRains
Copy link
Contributor

@zeroRains zeroRains commented Jul 7, 2025

pcard-71500

使用带有paddle.MmapStorage的safetensors,加速模型文件的加载。

单卡测试:ernie-4_5-21b-a3b-bf16-paddle

方法 文件加载时间 文件加载时间+模型参数导入模型时间 结论
baseline 28 44.64527976885438
MmapSotrage 4 22.69623290002346 加载时间缩短为原来的一半

8卡测试:ernie-4_5-300b-a47b-bf16-paddle

方法 文件加载时间 文件加载时间+模型参数导入模型时间 结论
baseline 175 222.6731412112713
MmapSotrage (get_slice) 168 226.66718888282776 基本不变
MmapSotrage (get_tensor) 86 131.25562098249793 加载时间缩短为原来的一半

前置PR:

提前使用方式:

  1. 编译最新develop的paddle
  2. 编译这个仓库git clone -b pp https://github.com/zeroRains/safetensors.git
  3. 拉取本pr的commit

@paddle-bot
Copy link

paddle-bot bot commented Jul 7, 2025

Thanks for your contribution!

@paddle-bot paddle-bot bot added the contributor External developers label Jul 7, 2025
@CLAassistant
Copy link

CLAassistant commented Jul 7, 2025

CLA assistant check
All committers have signed the CLA.

@zeroRains zeroRains force-pushed the loading branch 3 times, most recently from 371ac7b to 52b4ac6 Compare July 7, 2025 12:31
Change-Id: I8f6faff3d86b682ccdccc31b38eb1d6b1db5e8a1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants