What is PXE boot? How network booting and PXE installs work

What is PXE boot? PXE, the Preboot eXecution Environment, is a network boot standard led by Intel; version 2.1, the one still in use, was published in 1999. “Preboot” means before any operating system starts: even with nothing on the disks, the NIC firmware can get onto the network by itself, download a small program from a server into memory and run it. The “PXE Boot” or “Network Boot” entry in a server’s boot menu is this feature. Its most common use is a PXE install: using the network boot to deliver an installer and an answer file to the machine so the OS installs without anyone watching.

What PXE is: a boot client in the firmware

The PXE client does not live in the operating system. It lives in firmware:

Boot mode	Where the PXE client lives	What you typically see on screen at boot
Legacy (BIOS)	The NIC’s option ROM, which contains a DHCP client, a TFTP client and the UNDI driver interface	The NIC model, its MAC address and the word PXE
UEFI	The network stack in the motherboard firmware, with a UEFI driver supplied by the NIC	Start PXE over IPv4 (or IPv6)

One point makes everything else easier to follow: PXE itself does exactly one thing, which is to download the first program into memory and execute it. That program is the Network Bootstrap Program (NBP). Common NBPs are PXELINUX, GRUB and iPXE, plus Windows Deployment Services’ wdsnbp.com (BIOS) and wdsmgfw.efi (UEFI). What menu appears next, which kernel loads and which OS gets installed is up to the NBP and the installer; PXE has already left the stage. A PXE install is really a three-stage relay: the PXE client in firmware fetches the NBP, the NBP fetches the kernel and installer, and the installer follows the answer file to put the OS on disk.

A few terms that often get mixed up:

• PXE boot: a boot method, alongside booting from a local disk or a USB stick.
• PXE server: the machine (or machines) providing DHCP, TFTP and the installation source. It is a role, not a particular piece of software.
• PXE install: the whole process of installing an operating system through PXE boot, also called a network install.

How PXE boot works, step by step

1. The firmware initializes the NIC. After POST, when the boot order reaches the network card, the firmware loads the PXE client and starts sending requests once the link is up.
2. The client broadcasts a DHCP Discover. Unlike an ordinary DHCP request, it carries a few special options. Option 60 (vendor class) holds a string such as PXEClient:Arch:00007:UNDI:003016, which announces “I am here to network boot”. Option 93 is the client architecture: 0 for legacy BIOS, usually 7 for x86-64 UEFI (some firmware reports 9) and 11 for ARM64 UEFI. Option 97 is the client identifier, normally the motherboard UUID.
3. The DHCP server answers. Besides an IP address, the reply needs two things: the TFTP server address (the next-server field in the packet header, or option 66) and the boot file name (the file field in the header, or option 67). The DHCP server reads option 93 to tell BIOS from UEFI and hands out a different file name for each. If the production DHCP server is not yours to change, run a proxyDHCP alongside it: the existing server still assigns addresses, the proxyDHCP adds only the boot information, and UDP port 4011 is used along the way.
4. The client downloads the NBP over TFTP. It sends a read request to UDP port 69 on the TFTP server, downloads the NBP into memory and runs it. Classic TFTP sends 512-byte blocks and waits for an acknowledgment after each one, which is why it is slow; when both sides support the blksize option (RFC 2348), they can negotiate larger blocks.
5. The NBP reads its configuration and loads the kernel and initrd. PXELINUX, for example, looks for its configuration file in a fixed order: a file named after the motherboard UUID, then pxelinux.cfg/01- followed by the MAC address in dash-separated form, then the IP address in hexadecimal, dropping one digit at a time, and finally pxelinux.cfg/default. That order means you can give one machine its own configuration without affecting any other.
6. The installer fetches the installation source and the answer file. Once the kernel is up it starts the installer, which downloads packages and the answer file over HTTP, NFS or SMB from the addresses given in the boot parameters. A multi-gigabyte installation source does not go over TFTP, which is why a PXE setup needs an HTTP server as well.
7. The install runs unattended and the server reboots into the new system. The installer partitions, installs packages, runs post-install scripts and reboots. The classic trap is here: if the NIC is first in the boot order, the server goes straight back into PXE after the reboot and installs itself all over again. The usual fixes are to set “PXE on next boot only” through the BMC, or to make the menu default to booting from local disk and delete that machine’s own configuration once the install is done.

Steps 1 to 4 usually take somewhere between ten-odd seconds and a minute. Most of the time goes into downloading packages in step 6 and installing them in step 7, which depends on the bandwidth of the installation source and how many packages you install.

Watching a PXE boot with tcpdump

To troubleshoot, or just to learn, capture traffic on the PXE server’s install-network interface and match it against the steps above. Replace ens19 with your interface name and the MAC with the target server’s NIC:

tcpdump -ni ens19 -vv 'port 67 or port 68 or port 4011 or ether host 3c:ec:ef:12:34:56'

Order	What you see	What to check
1	DHCP Discover (client broadcast)	Option 60 starts with PXEClient; option 93 is 0 or 7
2	DHCP Offer	It carries next-server and a boot file name that matches the architecture; two servers answering means a rogue or duplicate DHCP server
3	DHCP Request / ACK	The address is confirmed
4	TFTP read request to UDP 69	The requested file name matches the actual path on the server
5	A long run of TFTP data and ACK packets	The NBP, kernel and initrd are downloading
6	HTTP requests to TCP 80	The installer is pulling the installation source and answer file; the PXE stage is over

Whichever step is followed by silence is where the problem is.

What you need for a PXE server

Component	Job	Common implementations	Ports
DHCP	Assigns addresses and tells the client where to get which boot file	ISC DHCP, Kea, dnsmasq, Windows DHCP	UDP 67, 68
proxyDHCP (optional)	Adds only the boot information when the existing DHCP server cannot be changed	dnsmasq, WDS	UDP 67, 4011
TFTP	Serves the NBP, menus, kernel and initrd	tftp-hpa, the TFTP server built into dnsmasq	UDP 69, with data on high ports
Network bootstrap program	Shows a menu and loads a kernel or WinPE	PXELINUX, GRUB (usually with shim under UEFI), iPXE	—
Installation source	OS packages or the image contents	nginx, Apache, NFS, SMB (for Windows)	TCP 80, 2049, 445
Answer file	Answers every installer question for you	Kickstart, preseed, autoinstall, autounattend.xml, ESXi ks.cfg	Served with the installation source
Network	Lets the DHCP broadcast reach the server	A dedicated install VLAN; a DHCP relay on the gateway across subnets	—
Trigger	Makes the server boot from the network this one time	One-time boot through the BMC, F12 at boot	UDP 623 (IPMI)

The first three (DHCP, TFTP and the NBP) are PXE proper; the rest belong to the install. Tools such as Cobbler, Foreman, MAAS, iVentoy and WDS bundle several of these components and manage them together, but the mechanism underneath is the same.

Two network details that are easy to overlook:

• Enable fast forwarding on the switch ports. If a port facing a server runs classic spanning tree, it spends about 30 seconds in listening and learning states after the link comes up before it forwards traffic, and the PXE client’s DHCP requests can time out in the meantime. Set those ports as edge ports (PortFast on Cisco).
• UEFI Secure Boot needs a signed chain all the way through. With Secure Boot on, the NBP and the kernel must carry valid signatures. A distribution-signed shim plus GRUB passes; an iPXE you compiled yourself generally does not, so either turn off Secure Boot or switch to a signed bootloader.

PXE has also moved on. iPXE is an open-source NBP that PXE can load and then hand over to: it downloads over HTTP instead of TFTP and runs scripts, which makes it much faster and more flexible. UEFI HTTP Boot, available from UEFI 2.5, lets the firmware itself download the boot file over HTTP, with its own client architecture value (16 for x86-64 UEFI HTTP). Both still start with DHCP, so think of them as the PXE approach carried forward.

Answer files: what makes a PXE install unattended

PXE only solves booting. What lets an install finish without anyone at the console is the answer file. Each OS uses a different format and a different way of pointing the installer to it:

OS	Answer file	How the installer finds it
Rocky Linux, AlmaLinux, RHEL 8 / 9	Kickstart	Kernel parameters inst.repo=http://10.30.0.2/rocky9 inst.ks=http://10.30.0.2/ks/web.cfg
Debian	preseed	Kernel parameters auto=true priority=critical url=http://10.30.0.2/preseed/web.cfg
Ubuntu Server 20.04 and later	autoinstall (cloud-init format)	Add autoinstall to the kernel parameters and point to a nocloud data source; the syntax has changed between releases, so follow the Ubuntu documentation
Windows Server	autounattend.xml	After WinPE starts, run setup.exe /unattend: followed by the answer file path
VMware ESXi	ks.cfg	Put ks=http://10.30.0.2/esxi/ks.cfg on the kernelopt line of boot.cfg

The riskiest line in any answer file is which disk to install to. An answer file that says “wipe all disks” will take the data disks with it on a machine that has them. In Kickstart, name the target disk explicitly:

ignoredisk --only-use=sda
clearpart --all --initlabel --drives=sda
autopart --type=lvm

Device names are not stable on machines that mix SATA, NVMe and RAID controllers, and sda may not be the disk you think it is. Before a bulk install, confirm the device names in the installer on one sample machine, keep separate answer files for different models, or use paths under /dev/disk/by-path/, which are named by physical location.

PXE vs USB vs virtual media installs

	PXE network install	USB install	Physical disc or BMC virtual media
On-site visit	Not needed when triggered through the BMC	Needed	Needed for a physical disc; not for virtual media
Up-front work	Set up services, write menus and answer files; days for the first build	Make a bootable stick in 10 to 20 minutes	Have the ISO ready
Hands-on time per server	A few minutes to set a network boot	Plug in, pick the boot device, babysit the installer	Mount the image, babysit the installer
Many servers at once	Yes, limited by the installation source’s bandwidth	One stick per server	Each server mounts its own image, one at a time
Consistent results	Same answer file, same partitions and packages	Depends on who does it	Depends on who does it
Typical failures	Competing DHCP servers, VLAN not reachable, BIOS/UEFI boot file mismatch	Stick written in the wrong mode, USB port not recognized	Slow or interrupted image transfer
Best for	Bulk racking, rental servers that are reinstalled again and again	A few machines, or a new site whose network is not up yet	One-off remote recovery, unusual operating systems

Whether PXE is worth setting up comes down to arithmetic. Suppose, as an example, a USB install takes 30 minutes of hands-on time per server (walking to the rack, plugging in, choosing the boot device, answering the installer), building the PXE environment takes 8 hours the first time, and after that each server takes 2 minutes. Then 8 × 60 ÷ (30 − 2) ≈ 17.1: by the 18th server, PXE has saved more time than it cost. If you are racking a dozen servers once, a USB stick or virtual media is the better deal. If you rent out servers, every customer change, return or OS switch means another reinstall of the same machine, several times a year, and an environment you build once pays for itself quickly.

Hold off on PXE when: the network has a DHCP server you do not control and proxyDHCP is not an option; you install only a handful of machines a year; Secure Boot must stay on and you have no signed boot chain; or the OS needs so many manual choices that it cannot be scripted into an answer file.

Scheduling PXE installs from a management system

PXE answers how a machine pulls an installer off the network. In a data center, someone still has to decide which machine goes through PXE and when, confirm the install finished, and know which image went on. With Toplink DCIM’s IPMI remote management you set a server’s next boot to PXE, an online ISO or the local disk through its BMC as part of a reinstall. For bulk installs, set a batch of servers to boot from PXE at once and hand them to the task queue for reinstalling, with each server’s progress visible in the admin panel, while agent nodes deployed in each data center run the reinstall jobs locally. Once a server is assigned to a customer, the customer picks an image and reinstalls it from the self-service portal; because a reinstall wipes the disk, it requires a one-time code to confirm.