自动压缩中使用蒸馏训练后 模型结果异常

[dreamcatcher-zgr]

原模型精度挺高的 但使用蒸馏训练后模型精度就不对了 把模型输出结果打印出来看后发现不同输入都会得到同一个输出 大佬帮忙看看什么原因呢

训练过程中的输出
2024-11-07 09:50:47,799-INFO: start to test metric before compress
Evaluation stage, Run batch:|██████████████████████████████████████████| 447/447
2024-11-07 09:50:49,434-INFO: metric of compressed model is: 0.9664429530201343
2024-11-07 09:50:49,726-INFO: train config.distill_node_pair: ['teacher_elementwise_add_0', 'elementwise_add_0']
I1107 09:50:50.019730 3462931 interpreter_util.cc:648] Standalone Executor is Used.
2024-11-07 09:50:50,217-INFO: Total iter: 0, epoch: 0, batch: 0, loss: 6.939813137054443 l2: 6.939813137054443
2024-11-07 09:50:50,312-INFO: Total iter: 10, epoch: 0, batch: 10, loss: 6.67962646484375 l2: 6.67962646484375
2024-11-07 09:50:50,405-INFO: Total iter: 20, epoch: 0, batch: 20, loss: 4.862423419952393 l2: 4.862423419952393
2024-11-07 09:50:50,498-INFO: Total iter: 30, epoch: 0, batch: 30, loss: 5.843245506286621 l2: 5.843245506286621
2024-11-07 09:50:50,591-INFO: Total iter: 40, epoch: 0, batch: 40, loss: 1.7612680196762085 l2: 1.7612680196762085
2024-11-07 09:50:50,685-INFO: Total iter: 50, epoch: 0, batch: 50, loss: 3.830599308013916 l2: 3.830599308013916
2024-11-07 09:50:50,781-INFO: Total iter: 60, epoch: 0, batch: 60, loss: 2.4715871810913086 l2: 2.4715871810913086
2024-11-07 09:50:50,873-INFO: Total iter: 70, epoch: 0, batch: 70, loss: 3.5432209968566895 l2: 3.5432209968566895
2024-11-07 09:50:50,967-INFO: Total iter: 80, epoch: 0, batch: 80, loss: 0.6653429865837097 l2: 0.6653429865837097
2024-11-07 09:50:51,064-INFO: Total iter: 90, epoch: 0, batch: 90, loss: 2.188952922821045 l2: 2.188952922821045
2024-11-07 09:50:51,166-INFO: Total iter: 100, epoch: 0, batch: 100, loss: 2.6419882774353027 l2: 2.6419882774353027
2024-11-07 09:50:51,262-INFO: Total iter: 110, epoch: 0, batch: 110, loss: 3.7801308631896973 l2: 3.7801308631896973
2024-11-07 09:50:51,355-INFO: Total iter: 120, epoch: 0, batch: 120, loss: 0.8452658653259277 l2: 0.8452658653259277
2024-11-07 09:50:51,449-INFO: Total iter: 130, epoch: 0, batch: 130, loss: 0.6925385594367981 l2: 0.6925385594367981
2024-11-07 09:50:51,546-INFO: Total iter: 140, epoch: 0, batch: 140, loss: 1.122310996055603 l2: 1.122310996055603
2024-11-07 09:50:51,640-INFO: Total iter: 150, epoch: 0, batch: 150, loss: 1.561903953552246 l2: 1.561903953552246
2024-11-07 09:50:51,735-INFO: Total iter: 160, epoch: 0, batch: 160, loss: 1.165655493736267 l2: 1.165655493736267
2024-11-07 09:50:51,830-INFO: Total iter: 170, epoch: 0, batch: 170, loss: 1.6543434858322144 l2: 1.6543434858322144
2024-11-07 09:50:51,924-INFO: Total iter: 180, epoch: 0, batch: 180, loss: 2.673067808151245 l2: 2.673067808151245
2024-11-07 09:50:52,017-INFO: Total iter: 190, epoch: 0, batch: 190, loss: 0.8985536098480225 l2: 0.8985536098480225
2024-11-07 09:50:52,110-INFO: Total iter: 200, epoch: 0, batch: 200, loss: 1.004972219467163 l2: 1.004972219467163
2024-11-07 09:50:52,203-INFO: Total iter: 210, epoch: 0, batch: 210, loss: 1.3521126508712769 l2: 1.3521126508712769
2024-11-07 09:50:52,296-INFO: Total iter: 220, epoch: 0, batch: 220, loss: 1.9136583805084229 l2: 1.9136583805084229
2024-11-07 09:50:52,390-INFO: Total iter: 230, epoch: 0, batch: 230, loss: 1.0522427558898926 l2: 1.0522427558898926
2024-11-07 09:50:52,483-INFO: Total iter: 240, epoch: 0, batch: 240, loss: 1.7291829586029053 l2: 1.7291829586029053
2024-11-07 09:50:52,577-INFO: Total iter: 250, epoch: 0, batch: 250, loss: 1.1346322298049927 l2: 1.1346322298049927
2024-11-07 09:50:52,670-INFO: Total iter: 260, epoch: 0, batch: 260, loss: 1.5537294149398804 l2: 1.5537294149398804
2024-11-07 09:50:52,765-INFO: Total iter: 270, epoch: 0, batch: 270, loss: 1.1103237867355347 l2: 1.1103237867355347
2024-11-07 09:50:52,859-INFO: Total iter: 280, epoch: 0, batch: 280, loss: 1.165198564529419 l2: 1.165198564529419
2024-11-07 09:50:52,953-INFO: Total iter: 290, epoch: 0, batch: 290, loss: 1.611351728439331 l2: 1.611351728439331
2024-11-07 09:50:53,047-INFO: Total iter: 300, epoch: 0, batch: 300, loss: 2.0881824493408203 l2: 2.0881824493408203
2024-11-07 09:50:53,140-INFO: Total iter: 310, epoch: 0, batch: 310, loss: 1.252273678779602 l2: 1.252273678779602
2024-11-07 09:50:53,234-INFO: Total iter: 320, epoch: 0, batch: 320, loss: 1.1105802059173584 l2: 1.1105802059173584
2024-11-07 09:50:53,327-INFO: Total iter: 330, epoch: 0, batch: 330, loss: 1.5239158868789673 l2: 1.5239158868789673
2024-11-07 09:50:53,420-INFO: Total iter: 340, epoch: 0, batch: 340, loss: 1.9464023113250732 l2: 1.9464023113250732
2024-11-07 09:50:53,514-INFO: Total iter: 350, epoch: 0, batch: 350, loss: 0.2952193021774292 l2: 0.2952193021774292
2024-11-07 09:50:53,608-INFO: Total iter: 360, epoch: 0, batch: 360, loss: 0.6373522877693176 l2: 0.6373522877693176
2024-11-07 09:50:53,709-INFO: Total iter: 370, epoch: 0, batch: 370, loss: 0.74739009141922 l2: 0.74739009141922
2024-11-07 09:50:53,810-INFO: Total iter: 380, epoch: 0, batch: 380, loss: 1.026517629623413 l2: 1.026517629623413
2024-11-07 09:50:53,905-INFO: Total iter: 390, epoch: 0, batch: 390, loss: 1.248557448387146 l2: 1.248557448387146
2024-11-07 09:50:53,999-INFO: Total iter: 400, epoch: 0, batch: 400, loss: 0.1199449747800827 l2: 0.1199449747800827
2024-11-07 09:50:54,093-INFO: Total iter: 410, epoch: 0, batch: 410, loss: 1.060091257095337 l2: 1.060091257095337
2024-11-07 09:50:54,187-INFO: Total iter: 420, epoch: 0, batch: 420, loss: 0.37600892782211304 l2: 0.37600892782211304
2024-11-07 09:50:54,281-INFO: Total iter: 430, epoch: 0, batch: 430, loss: 1.43874990940094 l2: 1.43874990940094
2024-11-07 09:50:54,374-INFO: Total iter: 440, epoch: 0, batch: 440, loss: 1.549577236175537 l2: 1.549577236175537
2024-11-07 09:50:54,467-INFO: Total iter: 450, epoch: 0, batch: 450, loss: 1.316918134689331 l2: 1.316918134689331
2024-11-07 09:50:54,560-INFO: Total iter: 460, epoch: 0, batch: 460, loss: 2.0584702491760254 l2: 2.0584702491760254
2024-11-07 09:50:54,653-INFO: Total iter: 470, epoch: 0, batch: 470, loss: 2.3895020484924316 l2: 2.3895020484924316
2024-11-07 09:50:54,747-INFO: Total iter: 480, epoch: 0, batch: 480, loss: 0.838370680809021 l2: 0.838370680809021
2024-11-07 09:50:54,843-INFO: Total iter: 490, epoch: 0, batch: 490, loss: 1.3764344453811646 l2: 1.3764344453811646
Evaluation stage, Run batch:|██████████████████████████████████████████| 447/447
2024-11-07 09:50:56,525-INFO: epoch: 0 metric of compressed model is: 0.252796, best metric of compressed model is 0.252796

配置文件
Global:
model_dir: MobileNetV1_infer
model_filename: /home/ai/zgr/classification/MobileNetV4/output/resnet18_pd_1/inference_model/model.pdmodel
params_filename: /home/ai/zgr/classification/MobileNetV4/output/resnet18_pd_1/inference_model/model.pdiparams
batch_size: 1
data_dir: /home/ai/zgr/model_cut/data/flower_datasets

ChannelPrune:

pruned_ratio: 0.1

prune_params_name:

- conv2d_0.w_0

criterion: l1_norm

Distillation:
alpha: 1.0
loss: l2

TrainConfig:
epochs: 1
eval_iter: 500
learning_rate:
type: CosineAnnealingDecay
learning_rate: 0.015
optimizer_builder:
optimizer:
type: Momentum
weight_decay: 0.00002
origin_metric: 0.9664429530201343