TL;DR The code used here is available on Github.
This article describes the steps I walked through in order to install an Ubuntu server on my Dell PowerEdge 110T server using Preboot eXecution Environment (PXE).
One may ask: why bother with PXE for ocasional installations? Wouldn’t be simpler to just use a bootable USB memory stick?
Well, from a practical point-of-view the answer is yes. Although, I decided to go through this process mainly for 2 reasons:
- as a lazy developer, I challenge myself to automate everything whenever possible
- I always wanted to do this, either professionally, or as a learning exercise (it looks like Iron man, and that’s cool)
That said, before we get started, it is important to understand some concepts and technologies, and how the whole thing works together.
PXE mechanics
Preboot eXecution Environment (PXE, pronounced as pixie) specification describes a standardized client-server environment that boots a software assembly, retrieved from the network, on PXE-enabled clients. – Wikipedia
On the client side it requires only a PXE-capable network interface controller (NIC), and uses a small set of industry-standard network protocols such as Dynamic Host Configuration Protocol (DHCP) and Trivial File Transfer Protocol (TFTP). In many computers, the PXE capability can be enabled in the BIOS.
When a machine is booted in PXE mode, it tries to acquire an IP address during the boot phase through DHCP, and it sends some PXE-specific options in its DHCPDISCOVER packet. A PXE-enabled DHCP server will then respond to the client’s request with a DHCPOFFER packet containing not only default network information (i.e. IP address), but also the TFTP server IP address and the name of the Network Bootstrap Program (NBP), in this case pxelinux.0. The client can then download the NBP from the TFTP server and start the boot process. More information about DHCP sessions here.
Once NBP is booted, it will load the configuration from pxelinux.cfg/default in order to print the installation menu or just proceed with the installation, following what has been configured. The installation menu configuration also specifies the initrd image that will handle the installation process. Check the Syslinux Project wiki for more details about the PXE boot process.
Although, I’d like to keep using my router as my default DHCP server (setup a second DHCP server just for PXE sounds like a bit of waste in my case). The problem is that my router does not have any PXE capability.
In order to address this issue, there is something called proxyDHCP, which is a second DHCP server which only provides PXE-specific information to the PXE client through DHCP.
BIOS x UEFI
The Basic Input/Output System (BIOS) is a firmware that comes pre-installed on computers, and sits between hardware and software.
Once the machine is turned on, the BIOS is responsible for initialising and configuring all the components, like CPU, memory, disks, etc. If the BIOS was not there, we’d probably be using jumpers to handle all the configuration. After the initialisation, the BIOS will perform the power-on self test (POST), to check whether all the components are working properly. Finally, the BIOS will look for bootable devices in order to hand off control to the operating system (OS).
The BIOS has been there for a long time and has some serious limitations. The BIOS can only handle Master Boot Records (MBR) until roughly 2 terabytes, for example. Slow boot time, lack of mouse support and poor graphics also figure in the list of BIOS limitations.
The Unified Extensible Firmware Interface (UEFI) is the successor of BIOS. The UEFI aims to fix BIOS limitations and add more features, like looking for malware during the boot process, adding some capabilities for remote troubleshooting and configuration, and more.
That said, it is important to know which boot mechanism is being used. A PXE server configured to work only with BIOS won’t work with UEFI, and the other way round. A netboot image is commonly prepared having one or another of these two mechanisms in mind. This article describes a PXE server configuration to be used with BIOS.
Setup a PXE server
In order to setup a PXE server, the following tools and techniques were used:
- Vagrant/VirtualBox/Ubuntu server: in order to avoid installing tools I won’t be using everyday on my machine, and to be able to re-use the same mechanism in another environment
- Dnsmasq: in order to setup a proxyDHCP and a TFTP server
- Preseeding: in order to spare myself from manually installing the OS
Setup a Vagrant box
The first step is to configure a Vagrant box running Ubuntu server over VirtualBox. At the network level, the box will be configured in bridge mode. In practice, this will leave the Virtual Machine (VM) running at the same level as the machine hosting it, allowing it to communicate directly with the DHCP server, and sparing some Network Address Translation (NAT) configuration.
Here’s the Vagrantfile that configures a box in bridge mode:
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/xenial64"
config.vm.network "public_network"
end
Based on the Vagrantfile, the command bellow will download an Ubuntu box from a HashiCorp’s catalogue, apply some configuration (if any) and start the VM.
$ vagrant up
Since this configuration is using bridge networking through Vagrant public_network, it is necessary to inform Vagrant which network it should use in the bridge. This will be the network interface the host machine uses to connect to the DHCP server and, consequently, the internet (in this case en0).
default: Available bridged network interfaces: 1) en0: Wi-Fi (AirPort) 2) en1: Thunderbolt 1 3) en2: Thunderbolt 2 4) bridge0 5) p2p0 6) awdl0 default: When choosing an interface, it is usually the one that is default: being used to connect to the internet. default: Which interface should the network bridge to?
Download the Ubuntu Netboot Image
A netboot image is a bootable image optimised to perform an installation via network (as opposed to an ISO image). It generally contains:
- a small network boot manager program like pxelinux.0 (or NBP), which is used to deploy the installation menu and load a full installation image;
- and a second-stage bootable full image (typically initrd), used to perform the installation
Download an Ubuntu netboot image that will handle the installation process over the internet:
$ wget http://archive.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64/current/images/netboot/netboot.tar.gz \ -O netboot.tar.gz \ --quiet $ mkdir -p /tmp/tftp $ tar xf netboot.tar.gz -C /tmp/tftp
Install and Configure Dnsmasq
The dnsmasq is a convenient tool that brings together many network services, like DNS, DHCP, TFTP, and more. Dnsmasq is used here to setup a proxyDHCP and a TFTP server.
Ensure dnsmask is installed in the Vagrant box:
$ sudo apt-get install dnsmasq
Finally, create the file /etc/dnsmasq.d/proxydhcp.conf, configure the TFTP server and enable the proxyDHCP:
# Set the boot filename for netboot/PXE. You will only need # this is you want to boot machines over the network and you will need # a TFTP server; either dnsmasq's built in TFTP server or an # external one. (See below for how to enable the TFTP server.) dhcp-boot=pxelinux.0 # Loads pxelinux.0 from dnsmasq TFTP server. pxe-service=x86PC, "Install Linux", pxelinux # Set the root directory for files available via FTP. tftp-root=/tmp/netboot # Log lots of extra information about DHCP transactions. log-dhcp # This range(s) is for the public interface, where dnsmasq functions # as a proxy DHCP server providing boot information but no IP leases. # Any ip in the subnet will do, so you may just put your server NIC ip here. dhcp-range=REPLACE_THIS_WITH_THE_IP_ADDRESS_OF_INTERFACE_USED_IN_THE_BRIDGE,proxy
Preseed the Installation
Preseeding is a method for automating the installation of the Debian operating system and its derivatives. Answers to installation questions, which would normally be answered interactively by an operator, are predetermined and supplied via a configuration file (and sometimes boot parameters). – Wikipedia
This process is sometimes called unattended installation.
In order to activate the unattended installation, append the preseed configuration to the boot command line in the installation menu (preseed/url=tftp://…/preseed.cfg) and activate the automation of the installation process (auto=true):
$ cat > /tmp/tftp/pxelinux.cfg/default << EOF default install label install kernel ubuntu-installer/amd64/linux append vga=788 initrd=ubuntu-installer/amd64/initrd.gz auto=true priority=critical preseed/url=tftp://REPLACE_THIS_WITH_THE_IP_ADDRESS_OF_INTERFACE_USED_IN_THE_BRIDGE/preseed.cfg EOF
Finally, create a preseed.cfg file with the desired parameters for the installation, and place it under the TFTP root directory (in this case /tmp/tftp). Here‘s an example of preseed configuration.
Troubleshooting
How to check if the TFTP service is running?
$ netstat -tul
A TFTP service listening for connections should appear as follows:
udp 0 0 *:tftp *:* udp6 0 0 [::]:tftp [::]:*
Check that I can get files from the TFTP server
Install the TFTP client and try to get files from the server.
$ sudo apt-get install tftp $ sudo tftp tftp: connect 192.168.1.106 tftp: get pxelinux.0 Received 43114 bytes in 0.0 seconds tftp: quit
How to check if the proxyDHCP server is receiving the DHCPDISCOVERY packet with PXE-specific options?
$ sudo tcpdump -i enp0s8 udp -s0 -vvv
When the PXE client is started, UDP packets like the following one should be intercepted by tcpdump (note the NBP pxelinux.0 in the response):
21:03:00.491030 IP (tos 0x0, ttl 20, id 1, offset 0, flags [none], proto UDP (17), length 576) 0.0.0.0.bootpc - 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:26:b9:8d:de:4d (oui Unknown), length 548, xid 0xbb8dde4d, secs 8, Flags [Broadcast] (0x8000) Client-Ethernet-Address 00:26:b9:8d:de:4d (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Parameter-Request Option 55, length 24: Subnet-Mask, Time-Zone, Default-Gateway, IEN-Name-Server Domain-Name-Server, RL, Hostname, BS Domain-Name, SS, RP, EP Vendor-Option, Server-ID, Vendor-Class, BF Option 128, Option 129, Option 130, Option 131 Option 132, Option 133, Option 134, Option 135 MSZ Option 57, length 2: 1260 GUID Option 97, length 17: 0.68.69.76.76.49.0.16.87.128.90.195.192.79.81.52.74 ARCH Option 93, length 2: 0 NDI Option 94, length 3: 1.2.1 Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001" END Option 255, length 0 PAD Option 0, length 0, occurs 212 21:03:00.492142 IP (tos 0xc0, ttl 64, id 52306, offset 0, flags [none], proto UDP (17), length 349) 192.168.1.113.bootps - 255.255.255.255.bootpc: [bad udp cksum 0xc373 - 0x374b!] BOOTP/DHCP, Reply, length 321, xid 0xbb8dde4d, secs 8, Flags [Broadcast] (0x8000) Client-Ethernet-Address 00:26:b9:8d:de:4d (oui Unknown) file "pxelinux.0"[|bootp]
Alternativelly, dnsmasq writes its logs in /var/log/syslog. Since we are using the log-dhcp option in the dnsmasq configuration, we should also have more information in that file.
My computer seems not to be booting in PXE mode
- Does your NIC has to be configured through BIOS to have PXE enabled?
- Are you using UEFI, and not BIOS?
- Do you need to press some key during boot time to boot in PXE mode?
To answer all these questions you might need to check your computer’s manual.
Diego's notes 