Linux drivers for communicating with the FLX-182 card through a virtual network over PCIe
These modules creates a network device on Versal card and host PC. Each FLX-182 card connected to PCIe becomes a peer in the network and is accessible from the host. FLX-182 peers cannot access each other. Host device is assigned a random MAC address on startup. FLX-182 devices are assigned a MAC address generated from IDCODE and VERSION information, so that it stays persistent between reboots.
Instructions
-
Instantiate the versal_network_device from https://gitlab.cern.ch/atlas-tdaq-felix/firmware/-/tree/phase2/FLX-1886_FLX182_virtual_network/sources/ip_cores/BNL182/ip_repo/versal_network_device_1.0?ref_type=heads
-
Connect the AXI bus to the Versal processing system
-
Connect the FIFO ports to the the corresponding ports of Wupper
-
The following modules need to be cross-compiled with petalinux for the Versal card
-
Run
petalinux-build -C flxnet
in the petalinux project to build the correct modules:- flxnet_dev.ko
- flxnet_target.ko
-
Copy Makefile.host to Makefile on host PC and run
make
to build the correct modules for the host PC:- flxnet_dev.ko
- flxnet_pcie.ko
-
Add the following device tree entry in the petalinux project (
system-user.dtsi
file inflx-petalinux/project-spec/meta-user/recipes-bsp/device-tree/files
):&amba { flxnet_instance: flxnet@20100030000 { #address-cells = <0x02>; #size-cells = <0x02>; reg = <0x201 0x00030000 0x00 0x100>; compatible = "atlas_tdaq_felix,VERSAL_virtual_network_device_v1.0"; status = "okay"; }; };
- The address (the number to the right of '@' and two first numbers in
reg
field) must match the Vivado address editor for the VERSAL_virtual_network IP
- The address (the number to the right of '@' and two first numbers in
-
Setup a DHCP server on the host to automatically configure an IP address on the Versal cards
-
For debugging messages, increase the message level of the ethernet adapter using the commands:
ethtool -s flxnet0 msglvl 4095 # display all debug messages ethtool -s flxnet0 msglvl 8191 # display all debug messages and packet data
- The exact message level may be different for different Linux versions
- For detailed explanation debug message levels see: https://www.kernel.org/doc/Documentation/networking/netif-msg.txt
The modules
- flxnet_dev.ko : network interface representation for the FIFOs, shared module between the Versal and the host, it exports a function that the other modules use for registering the device as a network peer
- flxnet_target.ko : driver for the Versal card, it obtains the IP information from the device tree, then adds the peer to the network
- flxnet_pcie.ko : driver for the host PC, it finds the correct PCIe devices and adds them as peers into the network
Update
Readme updated on 13-11-2024 by Laura Rientsma l.rientsma@nikhef.nl
Original author of the driver is Elena Zhivun, last edited on 05-07-2022
For questions, please contact Frans Schreuder f.schreuder@nikhef.nl
Biggest changes:
- Driver is adjusted for and tested with the FLX-182 card instead of the VMK180
- Driver is made compatible with modern Linux designs
- Changed FIFOs into AXI Stream FIFOs, added end of packet signal that fixed the delay of the first few packages
- The driver is written so that multiple flxnet network devices can be created and communicate with one host
- DHCP server on host automatically gives out an IP address to the Versal card instead of having to configure the IP addresses manually
- Added flxnet driver to the build_rpm.sh script from https://gitlab.cern.ch/atlas-tdaq-felix/felix-tdaq-drivers/-/tree/main?ref_type=heads, so that the modules on the host PC automatically insert on host server startup
- Added carrier function that gives a signal when the link goes up or down