Add additional Neuron drivers to 6.18

[jpculp]

Description of changes:
There are situations where manually overriding the Neuron driver chosen by the PCI detection may be warranted. This provides those drivers for those times. All automatic behavior is intact, these are just present on the filesystem for those use cases.

Testing done:

• Launched a modified aws-k8s-1.35 on trn2.48xlarge with these drivers and manually loaded the driver to show it working.

Details

bash-5.2# driverdog link-modules
23:35:36 [INFO] Copied neuron.ko
23:35:36 [INFO] Copied neuron.ko
bash-5.2# ls -al /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
-rw-r--r--. 1 root root 958105 May  7 23:35 /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
bash-5.2# find . -name "neuron.ko"
./var/lib/kernel-modules/.overlay/upper/6.18.20/kernel/drivers/neuron/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2_24/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7372/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7693/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8072/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8689/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_latest/neuron.ko
bash-5.2# for ko in ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_*/neuron.ko; do echo "=== $ko ==="; rmmod neuron 2>/dev/null; cp "$ko" /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko && modprobe neuron && modinfo neuron; done
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7372/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.7372.0
license:        GPL
description:    Neuron Driver, built from SHA: ed2eca4781405ab537f14c42850ecae3fd99dad4
import_ns:      DMA_BUF
srcversion:     65499ABA5AB2A03439601DD
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           userver_pds_node_cnt:pds ultraserver node count (int)
parm:           userver_pds_server_id:pds ultraserver id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7693/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.7693.0
license:        GPL
description:    Neuron Driver, built from SHA: 559cb9bf14a5b9e98b28b11fe8c7d70d659bd308
import_ns:      DMA_BUF
srcversion:     A5AA33BD55AE2BD2B4B7BE4
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           userver_pds_node_cnt:pds ultraserver node count (int)
parm:           userver_pds_server_id:pds ultraserver id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           dma_teardown_on_exit:Reset the DMA state on user process exit (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8072/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.8072.0
license:        GPL
description:    Neuron Driver, built from SHA: 381f0e424131a6a8ab4599de7edd7170ee3388c1
import_ns:      DMA_BUF
srcversion:     614E351145FEC5E249AE9B2
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           userver_pds_node_cnt:pds ultraserver node count (int)
parm:           userver_pds_server_id:pds ultraserver id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           dma_teardown_on_exit:Reset the DMA state on user process exit (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8689/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.8689.0
license:        GPL
description:    Neuron Driver, built from SHA: 623632eb9c3d3312be233cf19079301468d42497
import_ns:      DMA_BUF
srcversion:     6575987C65645BB7D4C215D
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           pds_reservation_id:pds reservation id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           reset_top_dma:Reset top-level DMAs during TPB reset (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           dma_teardown_on_exit:Reset the DMA state on user process exit (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)

• Launched a modified aws-ecs-3 on inf2.xlarge with these drivers and manually loaded the driver to show it working.

Details

bash-5.2# driverdog link-modules
00:31:32 [INFO] Copied neuron.ko
00:31:32 [INFO] Copied neuron.ko
bash-5.2# ls -al /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
-rw-r--r--. 1 root root 976289 May  8 00:31 /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
bash-5.2# find . -name "neuron.ko"
./var/lib/kernel-modules/.overlay/upper/6.18.20/kernel/drivers/neuron/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2_24/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7372/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7693/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8072/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8689/neuron.ko
./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_latest/neuron.ko
bash-5.2# for ko in ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_*/neuron.ko; do echo "=== $ko ==="; rmmod neuron 2>/dev/null; cp "$ko" /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko && modprobe neuron && modinfo neuron; done
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7372/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.7372.0
license:        GPL
description:    Neuron Driver, built from SHA: ed2eca4781405ab537f14c42850ecae3fd99dad4
import_ns:      DMA_BUF
srcversion:     65499ABA5AB2A03439601DD
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           userver_pds_node_cnt:pds ultraserver node count (int)
parm:           userver_pds_server_id:pds ultraserver id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_7693/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.7693.0
license:        GPL
description:    Neuron Driver, built from SHA: 559cb9bf14a5b9e98b28b11fe8c7d70d659bd308
import_ns:      DMA_BUF
srcversion:     A5AA33BD55AE2BD2B4B7BE4
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           userver_pds_node_cnt:pds ultraserver node count (int)
parm:           userver_pds_server_id:pds ultraserver id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           dma_teardown_on_exit:Reset the DMA state on user process exit (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8072/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.8072.0
license:        GPL
description:    Neuron Driver, built from SHA: 381f0e424131a6a8ab4599de7edd7170ee3388c1
import_ns:      DMA_BUF
srcversion:     614E351145FEC5E249AE9B2
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           userver_pds_node_cnt:pds ultraserver node count (int)
parm:           userver_pds_server_id:pds ultraserver id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           dma_teardown_on_exit:Reset the DMA state on user process exit (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)
=== ./x86_64-bottlerocket-linux-gnu/sys-root/usr/libexec/neuron/neuron_2x_8689/neuron.ko ===
filename:       /lib/modules/6.18.20/kernel/drivers/neuron/neuron.ko
alias:          pci:v00001d0fd00007064sv*sd*bc*sc*i*
version:        2.x.8689.0
license:        GPL
description:    Neuron Driver, built from SHA: 623632eb9c3d3312be233cf19079301468d42497
import_ns:      DMA_BUF
srcversion:     6575987C65645BB7D4C215D
depends:
name:           neuron
retpoline:      Y
vermagic:       6.18.20 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:
sig_key:
sig_hashalgo:   unknown
signature:
parm:           pds_reservation_id:pds reservation id (int)
parm:           userver_ctl:ultraserver election control (int)
parm:           userver_etimeout:ultraserver election timeout (int)
parm:           force_userver:Force Neuron UltraServer (int)
parm:           force_die_flip:Force Neuron Core Mapping APIs to give back DIE flip mappings (int)
parm:           nmetric_metric_post_delay:Minimum time to wait (in milliseconds) before posting metrics again (uint)
parm:           nmetric_log_posts:1: send metrics to CW, 2: send metrics to trace, 3: send metrics to both (uint)
parm:           no_reset:Dont reset device (int)
parm:           reset_top_dma:Reset top-level DMAs during TPB reset (int)
parm:           nc_per_dev_param:Number of neuron cores (int)
parm:           dev_nc_map:Map of active neuron cores (int)
parm:           dma_teardown_on_exit:Reset the DMA state on user process exit (int)
parm:           zerocopy_trn1_override:override zerocopy for trn1 (int)
parm:           mempool_min_alloc_size:Minimum size for memory allocation (int)
parm:           mempool_host_memory_size:Host memory to reserve(in bytes) (int)
parm:           mempool_small_pool_size:Size of genpool for small allocations (in bytes) (ulong)
parm:           mempool_small_alloc_max:Threshold (in bytes) for deciding if an allocation is small (ulong)
parm:           dup_helper_enable:enable duplicate routing id unload helper (int)
parm:           wc_enable:enable write combining (int)

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.