Homelabs Pi Storage, a CM5 NAS board

[kelroy1990]

CM5 Expansion Board - NAS/Server Platform

Introduction

As member of the HomeLabs Club community i really like all of the Raspberry Pi ecosystem, so i decided to work on an expansion board for the Raspberry Pi CM5 that turns it into a proper NAS/server platform with low power consumption. After months of development, I think it's ready to share with the community.

While this started as a personal project inspired by the Raspberry Pi ecosystem and my passion for servers and homelab setups, the non-profit association Homelabs Club is planning to make this available commercially in a near future.

Hardware Features

CM5 Compatibility:
Works with both SD card and eMMC versions of the CM5
Built-in SD card slot accessible from inside the board
Internal USB-C port allows direct programming of CM5's eMMC or SD card

Sensors

• Current Sensor for the whole board, report directly by linux command "sensors"

Networking:

• 2.5G Ethernet port (Note: This NIC doesn't have a pre-burned MAC address, so the system will generate a random one at boot. I'm working on a solution to program a proper MAC address into the NIC)
• 1G Ethernet port

Connectivity:

• HDMI output
• 2x USB 3.0 ports
• 40-pin GPIO header (standard Pi layout)

Storage:

• 2x NVMe M.2 slots
• 6x SATA ports for HDDs

Power & Control:

• Hardware power button
• 2x 4-pin fan headers for case cooling
• Dedicated CM5 cooling connector
• CR2035 battery holder for RTC power

..... And a lot of LEDs, of course.

Power Requirements

The board needs a 16-21V DC power supply with at least 150W capacity. I recommend going with 180W to be safe, especially if you're planning to run multiple drives.

It must be a DC barrel jack with a 5.5x2.5mm conector, it's really easy and cheap to find. The best option its fo find a cheap laptop DC power supply adapter.

For example, i am using my old ASUS laptop DC power supply, and it work like a rock¡¡

Thermal Considerations

This thing gets hot. The CM5 absolutely needs active cooling - I'm using a heatsink with a fan. The board itself has heatsinks on the main power components, but you'll want good airflow through your case. The fan headers help with this.

Block Diagram

The CM5 has its limitations - I've tried to balance the available resources as best as possible, but there will definitely be bottlenecks. That's part of the fun though, figuring out what those limits are and how to work around them :)

Photos

It's a sleek black PCB - 6 layers of pure awesomeness! The board quality turned out really nice.

Community Project

I've released the STEP files for anyone who wants to design cases. We're 3D printing and building complete NAS units with community designs - it's been pretty cool to see what people come up with.

Notes

A few things I learned during development:

• Don't skimp on the power supply - drives spinning up can cause voltage drops
• Case airflow is critical, especially around the CM5 area

This has been a challenging and super fun project to work on. But honestly, the most exciting part will be seeing what the community creates around this board!

[alejandroperezlopez]

The project in real life is amazing!! @geerlingguy if you want more information to be added here or anything else, just let us know. Thanks @kelroy1990 for all your work, from the beginning of the project to the end.

[geerlingguy]

Very cool board! I'm excited to finally see a nice looking NAS solution with the CM5; some of the other ones require a lot of extra work to get something looking halfway decent.

Another two options that could be considered in a future version:

• Using a PCIe Gen 3 switch chip so the 2.5 Gbps NIC and HDDs don't have to share 5 Gbps max bandwidth
• Maybe having holes in the PCB for airflow, so you can have the drives oriented horizontally with the PCB in the back, like a 'traditional' NAS (though toaster style is fun!).

[kelroy1990]

Hi geerlingguy, the PCI packet Switch is a Gen 3 capable one, retrocompatible with Gen 2.
So we have Gen 3 capability if you configure it on the CM5, already tested and the bandwith increase notably.

There is the chip information.
https://www.asmedia.com.tw/product/788Yqe9SPAqi5XG0/7c5YQ79xz8urEGr1

You are right about the holes, but the problem its the complexity of this board, its a 6 layer and we need every inch of copper :D, there is a lot of High speed lanes.

[photo removed because somehow the link was broken]

This is only the top hahaha.

We have some extra boards if you want to test it, i am already trying to push it to the limit with the 3D design of another member of the community.

[geerlingguy]

@kelroy1990 - Ah I missed that it was Gen 3!

I would love to test it out — if you ship a board, I would be able to print up to the size of my Bambu P1S print bed (not sure if the case design fits in that volume).

Do you have a website or GitHub repo for the board that I could link to? Also do you have plans to sell the board on Crowd Supply or elsewhere?

Regarding the holes in the board—it would just make all the high speed signals that much more fun hahaha

[kelroy1990]

@geerlingguy Sure, how should we share the shipping information?, mail, discord, telegram...

Right now I am printing it on the H2D but yes, I think that this should fit on a P1S. In the following days some of the community members will meet at an event in Spain, so I have already printed some extra parts if you want, but... of course, you will lose the fun of printing it :) hahaha.

We will upload the 3D files from the community on the github on the following days, but i can send you in advance.

I opened a GitHub project https://github.com/kelroy1990/Homelabs-Pi-Storage where we can share more information as a public site. We will prepare a website for sure, as there are very talented people in the community.

The idea is to create a Crowd Supply campaign as a starting point, and then the Homelabs association will study whether to sell through our own website or with the support of a platform like Tindie, for example.
Any advice is welcome of course¡, feedback is gold always.

[kelroy1990]

Hi @geerlingguy where should I send the board? 🙂

We're receiving more designs from the community this week, and next week we'll update the Git repo with the new designs and additional photos.
It's amazing how the community gets involved and creates different designs... especially since I'm not great at 3D design myself haha.

[geerlingguy]

@kelroy1990 - you can email me at jeff@jeffgeerling.com (my public email) and go from there. I'm also somewhat of a noob at 3D printed design, but getting better. So far I haven't designed any PCBs, want to do that at least once by the end of this year (something really simple!).

[geerlingguy]

I have them in hand, they got a quick mention today, before I get to testing them... maybe even with TrueNAS haha: https://www.youtube.com/watch?v=GryCOpcglyI

[geerlingguy]

Just noting this project is on the site now: https://pipci.jeffgeerling.com/boards_cm/homelabs-pi-storage.html

[geerlingguy]

One strange device I haven't seen before on USB is ASMedia KD-123:

pi@pinas:~ $ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 002: ID 174c:2362 ASMedia Technology Inc. KD-123
Bus 005 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 003: ID 174c:2362 ASMedia Technology Inc. KD-123
Bus 003 Device 002: ID 04b4:6500 Cypress Semiconductor Corp. 
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 002: ID 04b4:6502 Cypress Semiconductor Corp. CY4609
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
pi@pinas:~ $ lspci
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2712 PCIe Bridge (rev 30)
0001:01:00.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:00.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:02.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:03.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:04.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:08.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:0c.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:0e.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:02:0f.0 PCI bridge: ASMedia Technology Inc. Device 806a (rev 02)
0001:03:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02)
0001:08:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0002:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2712 PCIe Bridge (rev 30)
0002:01:00.0 Ethernet controller: Raspberry Pi Ltd RP1 PCIe 2.0 South Bridge

Tested NIC at 2.2 Gbps.

Update: It looks like KD-123 are two NVMe to USB adapters on the USB bus for the two NVMe slots. I'm installing TrueNAS to an NVMe drive in one of the slots :)

[geerlingguy]

A few notes about microSD card usage:

1. The fan header is just above the microSD card slot on the design revision I'm testing. This means to install an SD card I've had to pull the cover off that slot, mount the card in it manually, and affix it with Kapton tape. It might've been easier to just remove the fan header on there since this is using a larger fan.
2. Maybe due to the mounting, when using full speed I am getting SDHCI REGISTER DUMP every now and then, with ADMA Errors, that print to the Linux kernel log. I may try slower SD speeds to see if it cleans up that issue.
3. It would be a bit easier to have the microSD card slot mounted on an edge so I could get at it without removing the back of the case, but either way is fine... just need to find a way to position it so you don't have to break it to install a card ;)

[kelroy1990]

Hi @geerlingguy maybe i should change the microSD connector, its a connector that you should open before put the microSD inside. If you pull the cover off... i think that you maybe broke it hahahhha :), maybe you can put it again on place.

There is a video example.

Video Example

The Molex 47219-2001 is a hinge-type microSD card connector that uses a simple three-step installation process:

1.-Unlock the hinge cover: Use your fingernail to gently push the small hinge cover backwards until you hear/feel a click. This releases the locking mechanism and the cover becomes loose.
2.-Open and insert the microSD card: Lift the cover to a 90-degree angle, fully opening the connector. Place the microSD card into the connector slot, ensuring proper orientation. Card polarization features ensure correct card orientation preventing misinsertion.
3.-Close and lock the hinge cover: Lower the hinged cover back down over the microSD card, then push it forward in the opposite direction until you hear/feel another click. This locks the card securely into position.

PD: The KD-123 its the ASM2362 (USB3.2 Device Controller 10G).