Oss/libnetdevname
From DellLinuxWiki
Contents |
[edit] About
[edit] Linux Enumeration of NICs in the Enterprise
Servers in the Enterprise continue to have increasing number of network ports. Current Dell PowerEdge 11 Generation machines contain four onboard Lan-On-Motherboard (LOM) NICs and can take in additional PCIe interface cards that can contain up to four ports each. Given the large number of NICs, naming/ordering the interfaces is a major issue. There is currently no standard method for the operating system to enumerate these NIC ports in a particular order either in accordance to the BIOS or otherwise.
Dell addressed this with hardware design changes that ensure LOMs and add-on NICs appear in a desired order under PCIe.
There was also an attempt to address this problem by the BIOS passing the LOM ordering information via the SMBIOS table type 41 and the Dell developed open-source udev helper - biosdevname that reads the SMBIOS table and influences the naming of the interfaces.
Both these methods did not solve the problems entirely on all Linux distributions for all possible cases. NIC enumeration even with biosdevname and the restriction on the location of the NIC device on the PCI tree is largely non-deterministic. For instance, parallel udev threads loading network drivers is one case that biosdevname or the hardware design cannot address.
This has been a major concern for System Administrators and ISVs who spend efforts to map physical ports to the names that the OS identifies them with for large scale deployments. This also breaks images that Enterprise customers use on multiple systems that assume network device name to physical port mapping. Deployment scripts and firewall rules that have been developed with the assumption of network device name to physical port mapping would also break.
[edit] Solving with alternate network device name space
The assumption that the Linux kernel’s ethernet namespace of eth0, eth1, etc., maps to the physical layout is one of the causes for the NIC enumeration problem – this is a convention and not standardized. These names are not meaningful and they do not convey anything useful to the system administrator about the network device's properties.
The proposal is to create a parallel name space/s to that of the kernel’s in which device names are ordered based on various criteria like chassis numbering, PCI slot numbering, port speed, etc. This parallel namespace would then be mapped to the kernel’s name space by device-name management framework like udev that would make use of helper applications like biosdevname to ascertain the "correct" order of the interface ports. The kernel device names are required to appear as device nodes under /dev/ directory. The device nodes can have many-to-one mappings from various name spaces based on different criteria. Applications that conventionally use the Ethernet device names (ethN) as arguments can now use absolute paths to the alternative name space, like /dev/net/by-chassis-label/Embedded_NIC_1.
Applications would require to be modified to handle the new pathname in addition to traditional ethN names. The resolution logic would be provided as a library function that the applications can call and resolve pathnames to ethN names.
As an example:
/dev/net/by-chassis-label/Embedded_NIC_1 -> /dev/netdev/eth2
that represents eth2 in the kernel namespace. The library performs the look-up. This framework allows the assignment of more meaningful names such as Embedded_NIC_1 and Embedded_NIC_2 with symbolic links to kernel name space.
[edit] Implementation
[edit] Kernel
The standard linux kernel has been modified to create a character device node for each network interface detected on the system. kernel patch submitted to the netdev list 09-Oct-2009.
[edit] udev rules
If you have udev >= 145, it will create /dev/netdev/$kernelname for you.
If you have udev < 145, the following udev rule ensures proper mapping between the Ethernet character devices and the new names. This goes in /lib/udev/rules.d/50-udev-default.rules in the network section:
SUBSYSTEM=="net", KERNEL!="tun", NAME="netdev/%k", MODE="0600"
Additionally, biosdevname will install its own rules.
SUBSYSTEM!="net", GOTO="biosdevname_end" KERNEL!="eth*", GOTO="biosdevname_end" ACTION!="add", GOTO="biosdevname_end" PROGRAM="/sbin/biosdevname --policy=all_names -i %k", SYMLINK+="net/by-slot-name/%c", OPTIONS+="string_escape=replace" PROGRAM="/sbin/biosdevname --policy=smbios_names -i %k", SYMLINK+="net/by-chassis-label/%c", OPTIONS+="string_escape=replace" LABEL="biosdevname_end"
[edit] libnetdevname
A library that contains functions that applications can use to look-up the real device name given the absolute pathname. The library has been named libnetdevname
libnetdevname is a library that maps a pathname like /dev/netdev/ethN to an interface name like ethN. The pathname can be a char device file like /dev/netdev/ethN or symbolic links to it such as /dev/net/by-chassis-label/Embedded_NIC_1, etc. The library takes the pathname as an argument, "stat"s the pathname and retrieves the minor number which is the interface index. It then it maps this ifindex to a ethN name by making a netlink socket system call. If the argument given is not a pathname, then it just returns name as is and indicates success.
[edit] libnetdevname consumers
Modifications to commonly used network utilities to use the library functions.
Applications that traditionally use ethN names, can be patched to resolve the pathnames to the device names by issuing a call to this library like this:
char kernelname[IFNAMSZ];
path = argv[n];
if (netdev_pathname_to_name(path, kernelname) < 0)
error();
If path points to pathname, then library maps it to a kernel network name (like 'eth1') and kernelname will contain the returned value. It return success as indicator. If devname points to a traditional "ethN" name, library does nothing and returns success. Since the ioctls would require the traditional names, call to this library should be issued before issuing these ioctls.
Example of applications patched to use libnetdevname
# /sbin/ifconfig /dev/net/by-chassis-label/Embedded_NIC_1 # /usr/sbin/ethtool -p /dev/net/by-mac/00:01:02:03:04:05
[edit] Handling interface rename events
When interfaces are renamed using commands like "nameif" the corresponding device nodes and symbolic links need to be changed to reflect the new interface name. When an interface is renamed "move" event is generated. As of now, udev does not handle this event. The patch below addresses that issue.
With this patch, rules can be formed to handle move events -
SUBSYSTEM=="net", ACTION=="move", PROGRAM="/bin/mv /dev/netdev/$(basename $DEVPATH_OLD) /dev/netdev/$(basename $DEVPATH)"
[edit] Documentation
[edit] Mailing lists and Participation
- For any queries, bugs or suggestions on enhancements please post to the linux-poweredge@dell.com mailing list. You must subscribe to the list before posting.
- subscribe and archives.
