"slow_conv2d_cpu" not implemented for 'Half' #22
Description
This work is really fancy, I am trying to execute it following the indicated steps with one exception: I cannot install Cuda in my Miniconda environment since my Mac does not support it (M3 chips). I thought it would be ok, until trying to run the test command: "bash ssh_scripts/multimodal_sample_sr.sh", with this execution:"
Logging to /Users/k/Documents/Deep_Generative_Model/MM-Diffusion/samples/multimodal-sample-sr/dpm_solver
Effective parameters:
<<< all_save_num: 64
<<< audio_attention_resolutions: -1
<<< audio_fps: 16000
<<< audio_size: [1, 25600]
<<< audio_type: 1d
<<< batch_size: 4
<<< channel_mult:
<<< class_cond: False
<<< classifier_scale: 0
<<< clip_denoised: True
<<< cross_attention_resolutions: 2,4,8
<<< cross_attention_shift: True
<<< cross_attention_windows: 1,4,8
<<< devices: 0,1,2,3
<<< diffusion_steps: 1000
<<< dropout: 0.0
<<< is_strict: True
<<< large_size: 256
<<< learn_sigma: False
<<< load_noise:
<<< multimodal_model_path: /Users/k/Documents/Deep_Generative_Model/MM-Diffusion/models/landscape.pt
<<< noise_schedule: linear
<<< num_channels: 128
<<< num_head_channels: 64
<<< num_heads: 4
<<< num_heads_upsample: -1
<<< num_res_blocks: 2
<<< output_dir: /Users/k/Documents/Deep_Generative_Model/MM-Diffusion/samples/multimodal-sample-sr/dpm_solver
<<< predict_xstart: False
<<< ref_path: /Users/k/Documents/Deep_Generative_Model/MM-Diffusion/landscape/train
<<< resblock_updown: True
<<< rescale_learned_sigmas: False
<<< rescale_timesteps: False
<<< sample_fn: dpm_solver
<<< save_type: mp4
<<< seed: 42
<<< small_size: 64
<<< sr_attention_resolutions: 8,16,32
<<< sr_class_cond: False
<<< sr_diffusion_steps: 1000
<<< sr_dropout: 0.0
<<< sr_learn_sigma: True
<<< sr_model_path: /Users/k/Documents/Deep_Generative_Model/MM-Diffusion/models/landscape_SR.pt
<<< sr_num_channels: 192
<<< sr_num_head_channels: -1
<<< sr_num_heads: 4
<<< sr_num_heads_upsample: -1
<<< sr_num_res_blocks: 2
<<< sr_resblock_updown: True
<<< sr_sample_fn: ddim
<<< sr_timestep_respacing: ddim25
<<< sr_use_scale_shift_norm: True
<<< timestep_respacing:
<<< use_checkpoint: False
<<< use_fp16: True
<<< use_kl: False
<<< use_scale_shift_norm: True
<<< video_attention_resolutions: 2,4,8
<<< video_fps: 10
<<< video_size: [16, 3, 64, 64]
<<< video_type: 2d+1d
creating model and diffusion...
models waiting to be evaluated:['/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/models/landscape.pt']
keys not initialize in model*
keys not used in state_dict*"
But then I have this error that is just under, as follow:"
Traceback (most recent call last):
File "py_scripts/multimodal_sample_sr.py", line 309, in
main()
File "py_scripts/multimodal_sample_sr.py", line 125, in main
sample = dpm_solver.sample(
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1293, in sample
x = self.singlestep_dpm_solver_update(x, vec_s, vec_t, order, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1060, in singlestep_dpm_solver_update
return self.singlestep_dpm_solver_third_update(x, s, t, return_intermediate=return_intermediate, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 819, in singlestep_dpm_solver_third_update
model_s = self.model_fn(x, s)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 449, in model_fn
return self.noise_prediction_fn(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 417, in noise_prediction_fn
return self.model(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 350, in model_fn
return noise_pred_fn(x, t_continuous)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 305, in noise_pred_fn
video_output,audio_output = model(x["video"], x["audio"], t_input, **model_kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 1085, in forward
video, audio = module(video, audio, emb)#
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 45, in forward
video, audio = layer(video, audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 694, in forward
return self.video_conv(video), self.audio_conv(audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 96, in forward
video = self.video_conv_spatial(video)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 460, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'
Traceback (most recent call last):
File "py_scripts/multimodal_sample_sr.py", line 309, in
main()
File "py_scripts/multimodal_sample_sr.py", line 125, in main
sample = dpm_solver.sample(
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1293, in sample
x = self.singlestep_dpm_solver_update(x, vec_s, vec_t, order, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1060, in singlestep_dpm_solver_update
return self.singlestep_dpm_solver_third_update(x, s, t, return_intermediate=return_intermediate, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 819, in singlestep_dpm_solver_third_update
model_s = self.model_fn(x, s)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 449, in model_fn
return self.noise_prediction_fn(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 417, in noise_prediction_fn
return self.model(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 350, in model_fn
return noise_pred_fn(x, t_continuous)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 305, in noise_pred_fn
video_output,audio_output = model(x["video"], x["audio"], t_input, **model_kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 1085, in forward
video, audio = module(video, audio, emb)#
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 45, in forward
video, audio = layer(video, audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 694, in forward
return self.video_conv(video), self.audio_conv(audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 96, in forward
video = self.video_conv_spatial(video)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 460, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'
sampling samples for /Users/k/Documents/Deep_Generative_Model/MM-Diffusion/models/landscape.pt
Traceback (most recent call last):
File "py_scripts/multimodal_sample_sr.py", line 309, in
main()
File "py_scripts/multimodal_sample_sr.py", line 125, in main
sample = dpm_solver.sample(
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1293, in sample
x = self.singlestep_dpm_solver_update(x, vec_s, vec_t, order, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1060, in singlestep_dpm_solver_update
return self.singlestep_dpm_solver_third_update(x, s, t, return_intermediate=return_intermediate, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 819, in singlestep_dpm_solver_third_update
model_s = self.model_fn(x, s)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 449, in model_fn
return self.noise_prediction_fn(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 417, in noise_prediction_fn
return self.model(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 350, in model_fn
return noise_pred_fn(x, t_continuous)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 305, in noise_pred_fn
video_output,audio_output = model(x["video"], x["audio"], t_input, **model_kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 1085, in forward
video, audio = module(video, audio, emb)#
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 45, in forward
video, audio = layer(video, audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 694, in forward
return self.video_conv(video), self.audio_conv(audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 96, in forward
video = self.video_conv_spatial(video)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 460, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'
Traceback (most recent call last):
File "py_scripts/multimodal_sample_sr.py", line 309, in
main()
File "py_scripts/multimodal_sample_sr.py", line 125, in main
sample = dpm_solver.sample(
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1293, in sample
x = self.singlestep_dpm_solver_update(x, vec_s, vec_t, order, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 1060, in singlestep_dpm_solver_update
return self.singlestep_dpm_solver_third_update(x, s, t, return_intermediate=return_intermediate, solver_type=solver_type, r1=r1, r2=r2)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 819, in singlestep_dpm_solver_third_update
model_s = self.model_fn(x, s)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 449, in model_fn
return self.noise_prediction_fn(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 417, in noise_prediction_fn
return self.model(x, t)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 350, in model_fn
return noise_pred_fn(x, t_continuous)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_dpm_solver_plus.py", line 305, in noise_pred_fn
video_output,audio_output = model(x["video"], x["audio"], t_input, **model_kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 1085, in forward
video, audio = module(video, audio, emb)#
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 45, in forward
video, audio = layer(video, audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 694, in forward
return self.video_conv(video), self.audio_conv(audio)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/Documents/Deep_Generative_Model/MM-Diffusion/mm_diffusion/multimodal_unet.py", line 96, in forward
video = self.video_conv_spatial(video)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 460, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/Users/k/miniconda3/envs/mmdiffusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
mpiexec detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[42769,1],0]
Exit code: 1
--------------------------------------------------------------------------"
Does anyone had this error and was able to solve it? Or any ideas on a further investigation?
Many thanks in advance