Update ppo

[dddd-d]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

New features

PR changes

Models

Description

update ppo

原PR地址（含PR view）：#10884

[paddle-bot]

Thanks for your contribution!

[DesmonDay]

LGTM （备注：在原来的PR已经整体review过，新提PR主要是为了触发CI）