Install SONiC on Dell S5212F-ON: Budget 25GbE/100GbE

The flood of used Dell -ON suffix switches on the second-hand market made 25GbE and 100GbE affordable for homelabs and small businesses, but many arrived locked to Dell OS10, which needs an expensive license and can reboot every 72 hours without one. This hands-on guide shows how to escape that trap: update the switch firmware, boot into ONIE, install the open-source community SONiC image from a USB stick, and bring the switch up as a fully open network OS. Written from real experience with the Dell S5212F-ON, it covers the gotchas that the quick-start docs do not mention.

Background: Why an Alternative OS

These switches were cheap partly because they only worked with Dell OS10 (which is expensive), and OS10 would reboot the switch every 72 hours without a valid license. Non-Dell operating systems advanced quickly because of this — SONiC is fully open source and production-hardened in hyperscale data centers, making it the natural choice.

There is a second reason the used prices are low. The S5212F-ON shipped as an "open networking" switch with an ONIE bootloader, but the vendor-specific pieces — the platform driver, the fan and thermal control, the optics firmware — were always tuned for the vendor OS. Open networking promises a choice of operating systems; in practice the choice works best when the community image already carries a platform definition for your exact model. The S5212F-ON is a Broadcom Trident 3 box, which is the best-supported ASIC family in the SONiC ecosystem, and that is the main reason this project is viable at all.

What the S5212F-ON Actually Is

Before buying or flashing, confirm you have the right platform. The S5212F-ON is a half-width 1RU top-of-rack switch:

Note the port maths: 12 x 25 GbE plus 3 x 100 GbE is a genuinely useful leaf for a small cluster, and it is why these units get snapped up. Note also the word fixed next to the fans — it becomes important later.

Prerequisites and Parts List

• Console access. A USB-to-RJ45 (or USB-to-serial plus the supplied RJ45 adapter) cable and a terminal at 115200 baud, 8 data bits, no parity, 1 stop bit, no flow control. You will need this at every stage; do not start without it.
• A USB stick, 4 GB or larger, formatted FAT32. ONIE's shell understands vfat; the switch will not read exFAT or a modern default NTFS stick.
• The SONiC image for Broadcom platforms. Community SONiC publishes one image per ASIC vendor, so the Broadcom build (sonic-broadcom.bin in the official naming) is the one to download, not the virtual-switch image. Verify the checksum after downloading — a truncated image on a USB stick produces an ONIE install that hangs at 90 percent.
• Power and airflow. The unit is loud. Plan for a rack or a room where that is acceptable, and make sure you have both PSUs seated before powering up.
• Time. Firmware updates, an ONIE uninstall and a SONiC install comfortably fill an evening. Do not start this an hour before you need the switch.

Connect to the Console and Check What You Have

Power the switch and let it boot into whatever OS it shipped with. Default credentials are admin/admin for the switch OS and linuxadmin/linuxadmin for the Linux shell user on Dell OS10. Collect the current state before you destroy it — the ONIE environment is worth capturing because a wrong onie_boot_reason or a bad boot order is the usual cause of a switch that will not install anything:

show version
show image status
show system
show inventory

From the Linux shell, or from ONIE once you reach it, the ONIE utilities answer the questions that matter:

ONIE:/ # onie-syseeprom
ONIE:/ # onie-sysinfo
ONIE:/ # fw_printenv | grep -i onie
ONIE:/ # onie-support

onie-syseeprom prints the platform name, service tag and MAC base — if it returns garbage or all zeros, the EEPROM is unreadable and no NOS will identify the platform correctly. onie-support writes a support bundle to a USB stick and is the single most useful thing to capture if you plan to ask anyone for help.

Update the Firmware First

Use a serial console at 115200 baud. Update the Dell platform firmware before installing anything — the ONIE firmware was so old on some units that the update had to be applied twice, updating BMC, BIOS and FPGA firmware in different steps. Default credentials are admin/admin for the switch OS and linuxadmin/linuxadmin for the Linux shell user.

The ordering matters more than the method. ONIE itself must be recent enough to understand current image metadata and to hand the platform definition over cleanly; an ONIE from 2017 will boot a 2024 community image and then fail at the point where it tries to initialise the ASIC. Update ONIE from its own menu, or from the ONIE shell with the self-update mechanism, then reboot and confirm the version before touching the NOS:

ONIE:/ # onie-self-update tftp://192.0.2.10/onie-updater-x86_64-dell_s5212f-r0.bin
ONIE:/ # onie-sysinfo
ONIE:/ # reboot

Expect the ONIE update to release the console for several minutes. Do not interrupt it. If the switch reboots twice, that is normal on some firmware revisions — the FPGA update stage reboots into ONIE a second time to apply the next component.

Preparing the USB Stick

Format the stick as FAT32, then copy the SONiC image to the root of the drive and give it a short name to make life easier at the ONIE prompt:

# On a Linux workstation, with the stick at /dev/sdX1
sudo mkfs.vfat -F 32 -n SONIC /dev/sdX1
sudo mount /dev/sdX1 /mnt
sudo cp sonic-broadcom.bin /mnt/sonic.bin
sync && sudo umount /mnt

Some guides show writing the image with dd directly to the device. Do not do that here: ONIE wants a filesystem it can mount and a file it can read, not a raw block copy of an installer ISO. A copy is also far easier to replace when you discover you grabbed the wrong ASIC family, which happens to nearly everyone the first time.

Install SONiC via ONIE

Download the Broadcom SONiC BIN file, copy it to a USB drive, and boot the switch into ONIE. From the ONIE menu choose Uninstall OS first (to remove OS10), reboot, then select Install OS:

GNU GRUB version 2.02~beta2+e4a1fe391
+----------------------------------------------------------------------------+
|*ONIE: Install OS
| ONIE: Rescue
| ONIE: Uninstall OS
| ONIE: Update ONIE
+----------------------------------------------------------------------------+

mkdir /mnt/media
mount /dev/sdb1 /mnt/media
onie-os-install /mnt/media/sonic-broadcom-filename.bin

The installation takes a while; then log in with the default SONiC credentials admin/YourPaSsWoRd.

One correction worth knowing if your ONIE build was refreshed in the previous step: the installer binary in current ONIE releases is called onie-nos-install, and the original write-up of this procedure uses the equivalent onie-os-install. If the shell answers "command not found", use onie-nos-install with exactly the same argument — it is the same action, exposed under the older name on this platform:

ONIE:/ # onie-nos-install /mnt/media/sonic.bin

If the USB stick is not detected, check the device names rather than guessing. ONIE numbers block devices in discovery order and the internal SSD is often /dev/sda, leaving your stick at /dev/sdb:

ONIE:/ # fdisk -l
ONIE:/ # ls /dev/sd*

When you have no USB stick at all, ONIE can fetch the image over the network from rescue mode, which is also the fastest way to reinstall after a failed upgrade:

ONIE:/ # onie-discovery-stop
ONIE:/ # ip addr add 192.0.2.20/24 dev eth0
ONIE:/ # ip link set eth0 up
ONIE:/ # ip route add default via 192.0.2.1
ONIE:/ # onie-nos-install http://192.0.2.10/sonic.bin

Configuring the Switch

SONiC comes up in router mode by default, with every interface getting an IP. On some builds even show commands require sudo, which differs from the documentation — remember to prefix with sudo when you hit permission errors reading the EEPROM:

admin@sonic:~$ show interface status

If you get a PermissionError reading /sys/class/i2c-adapter/i2c-0/0-0050/eeprom, re-run the command with sudo. From there, configure interfaces, VLANs and routing with the standard SONiC config commands.

The first ten minutes after first boot are worth spending on discovery rather than configuration, because SONiC's platform layer either works or does not, and there is no point configuring a switch whose fans are about to spin to full speed:

admin@sonic:~$ show version
admin@sonic:~$ show platform summary
admin@sonic:~$ show platform fan
admin@sonic:~$ show platform temperature
admin@sonic:~$ sudo show interface status
admin@sonic:~$ show interface counters

If show platform fan returns a real fan table with speeds, the platform driver matched and the switch is functional. If it returns "Fan not detected" or an empty table, the platform definition does not cover this unit — check the community platform definitions and the build you installed before spending time on anything else.

Basic configuration follows the usual SONiC pattern. Interfaces start administratively up and get addresses assigned explicitly; VLANs are created and then members added; everything lands in the configuration database and should be saved when it is right:

admin@sonic:~$ sudo config hostname sw-leaf1
admin@sonic:~$ sudo config interface ip add Ethernet0 10.0.0.1/24
admin@sonic:~$ sudo config interface ip remove Ethernet0 10.0.0.1/24
admin@sonic:~$ sudo config vlan add 100
admin@sonic:~$ sudo config vlan member add 100 Ethernet4
admin@sonic:~$ sudo config interface autoneg Ethernet4 enabled
admin@sonic:~$ sudo config save -y
admin@sonic:~$ sudo config reload -y

The save step is the one people forget, and it is the number one cause of "my configuration disappeared after reboot". SONiC's running configuration lives in the configuration database; config save writes config_db.json to disk and makes it the startup configuration. Verify the write and the resulting state:

admin@sonic:~$ show startup-config
admin@sonic:~$ show ip interface
admin@sonic:~$ show vlan brief
admin@sonic:~$ show running-configuration

Two-port LACP and breakout are the reason most people buy this switch, so check them early. A 100 GbE uplink can be broken into four 25 GbE lanes, and the port naming after breakout is what trips people up — the lanes appear as separate interfaces and must be configured individually:

admin@sonic:~$ show interface breakout
admin@sonic:~$ sudo config interface breakout Ethernet12 '1x100G[40G]'
admin@sonic:~$ sudo config portchannel add PortChannel1
admin@sonic:~$ sudo config portchannel member add PortChannel1 Ethernet0
admin@sonic:~$ sudo config portchannel member add PortChannel1 Ethernet4

Managing the Image Later

SONiC installs as two image partitions and boots the default one, which gives you a genuine rollback path. This is a large advantage over the vendor workflow, and it is worth using properly: install the new image, set it as default, reboot, and verify before deleting the old one.

admin@sonic:~$ sudo sonic-installer list
admin@sonic:~$ sudo sonic-installer set-default sonic-broadcom.bin
admin@sonic:~$ sudo sonic-installer install http://192.0.2.10/sonic-broadcom-next.bin
admin@sonic:~$ sudo sonic-installer cleanup
admin@sonic:~$ sudo reboot

Keep at least one known-good image on the switch at all times. A community image that installs cleanly can still fail at the platform layer after a reboot, and the two minutes you spend checking sonic-installer list before rebooting is the cheapest insurance in this whole procedure.

Hardware Notes

The S5212F-ON is loud — fans ramp to over 9000 rpm when a fan is removed. If you replace fans with Noctua 40mm units, expect the switch to complain and run the remaining fans at high speed; monitor temperatures carefully after any fan swap.

Two related behaviours are worth knowing before you start modifying cooling. First, the platform driver usually reads fan presence and tachometer data from the BMC, so a fan it cannot see is treated as a failed fan, and the response is to run everything else at maximum — the noise gets worse, not better. Second, thermal monitoring depends on the same sensor path, so after any swap watch the reported inlet and ASIC temperatures for a full load cycle rather than assuming the sensors are trustworthy:

admin@sonic:~$ show platform temperature
admin@sonic:~$ show platform psu
admin@sonic:~$ sensors | head -40

If temperatures sit within vendor limits and the fan table shows the expected speeds, the swap is fine regardless of the alarm. If the ASIC temperature climbs steadily under load, put the original fans back.

Check optics compatibility early as well. Trident 3 platforms are usually happy with generic 25G SFP28 and 100G QSFP28 modules, and often fussy about QSFP-to-SFP adapters and long passive DACs. Test one link of each type before ordering a hundred of anything.

Rolling Back to OS10

The uninstall step removed OS10, so a rollback means installing it again from an image file rather than selecting an older partition. You will need the OS10 image and a valid licence from Dell support, plus the same ONIE procedure in reverse: boot to ONIE, choose Install OS, and point at the OS10 image. Keep the current OS10 image on your file server before you wipe the switch the first time — recovering it later from the second-hand listing you bought is not a plan. If you only removed the OS with a factory reset rather than an ONIE uninstall, the ONIE menu's install option is all you need.

Verification Checklist

• show version reports the SONiC build you intended to install, and not the virtual-switch or wrong-ASIC image.
• show platform summary reports the correct model and a matching platform definition.
• show platform fan and show platform temperature return real values, not empty tables.
• show interface status shows every connected port up at the expected speed.
• At least one 25 GbE and one 100 GbE link pass traffic, including one 4x25 GbE breakout.
• show startup-config contains the configuration you saved, and it survives a reboot.
• sonic-installer list shows a second, known-good image available for rollback.
• Management access and the console both still work after a cold restart.

Related open-networking articles: the SONiC troubleshooting guide, SONiC quick start: ONIE install, ONIE and Onyx MLNX-OS installation, and getting started with Mellanox switches.

原文链接：https://forum.level1techs.com/t/dell-s5212f-on-alternative-os-sonic-setup-guide-25gbe-100gbe-on-a-budget/198643