On this page:

PXE sever

Ubuntu server autoinstallation can be done through network using PXE (Preboot eXecution Environment). x86-64 systems boot in either UEFI or legacy (“BIOS”) mode (many systems can be configured to boot in either mode). The precise details depend on the system firmware, but both modes supports the PXE specification, which allows the provisioning of a bootloader over the network.

See details in Ubuntu’s documentation: “Ubuntu Advance Installation - Netbooting the server installer in amd64”

A PXE server will be deployed in the Cluster for automatically autoinstall Ubuntu 22.04 in x86 nodes. This PXE server will be deployed in gateway node, leveraging DHCP and TFTP server dnsmasq can provide, and adding a Kick-start web service for serving cloud-init files and ISO live image.

Install process is like this

The to-be-installed machine boots, and is directed to network boot.
The DHCP server tells the machine its network configuration and where to get the bootloader.
The machine’s firmware downloads the bootloader over TFTP and executes it.
The bootloader downloads configuration, also over TFTP, telling it where to download the kernel (vmlinuz), RAM Disk (initrd) and kernel command line to use (grub.cfg or pxelinux.cfg).
The RAM Disk looks at the kernel command line to learn how to configure the network and where to download the server ISO from.
The RAM Disk downloads the ISO and mounts it as a loop device.
From this point on the install follows the same path as if the ISO was on a local block device

pxe-flow

PXE server installation

PXE server will have installed all required components for enabling net boot and autoinstall of Ubuntu OS.

The components required are:

DHCP server:
TFTP server
Web server

Kick-start server installation (Webserver)

Webserver will be used as kick-start server providing to the netboot installer, via HTTP, Ubuntu’s live ISO and the cloud-init files required for launching the installation.

Step 1. Install apache2
```
sudo apt install apache2
```
Step 2. Created a new file ks-server.conf under /etc/apache2/sites-available/ with the following content

<VirtualHost 10.0.0.10:80>
    ServerAdmin root@server1.example.com
    DocumentRoot /
    ServerName server.example.com
    ErrorLog ${APACHE_LOG_DIR}/ks-server.example.com-error_log
    CustomLog ${APACHE_LOG_DIR}/ks-server.example.com-access_log common
    <Directory /ks>
        Options Indexes MultiViews
        AllowOverride All
        Require all granted
    </Directory>
    <Directory /images>
        Options Indexes MultiViews
        AllowOverride All
        Require all granted
    </Directory>
</VirtualHost>

Step 3. Create HTTP serving directories

ks which will contain cloud-init files and images where we will place our ubuntu live-server iso image.
```
sudo mkdir -p /var/www/html/ks
sudo mkdir -p /var/www/html/images
```
Step 4. Enable apache2
```
sudo systemctl enable apache2 --now
```
Step 4. Check apache status
```
sudo systemctl status apache2
```

Serving ubuntu live ISO file via HTTP

Ubuntu live ISO need to be served by the HTTP server

Step 1. Download Ubuntu 22.04 server live ISO

wget http://cdimage.ubuntu.com/ubuntu-server/jammy/daily-live/current/jammy-live-server-amd64.iso

Step 2. Copy to images directory

cp jammy-live-server-amd64.iso /var/www/html/images/.

Serving cloud-init files via HTTP

auto-install cloud-init files are served by HTTP server

Step 1. Create a directory with name within /var/www/html/ks
```
mkdir -p /var/www/html/ks/<server-macaddress>
```
/var/www/html/ks/aa:bb:cc:dd:ee:00/meta-data /var/www/html/ks/aa:bb:cc:dd:ee:00/user-data

Step 2. Create user-data file in /var/www/html/ks/

This must be a cloud-init ubuntu auto-install file

Minimal config

#cloud-config
autoinstall:
  identity:
    hostname: jammy-minimal
    password: $6$gnqbMUzHhQzpDEw.$.cCNVVDsDfj5Feebh.5O4VbOmib7tyjmeI2ZsFP7VK2kWwgJFbfjvXo3chpeAqCgXWVIW9oNQ/Ag85PR0IsKD/
    username: ubuntu
  version: 1

The above cloud-init user-data file creates a minimum installation, setting server hostname, and ubuntu default password (ubuntu)

Step 3. Create meta-data file in /var/www/html/ks/

Create cloud-init meta-data file containing the hostname of the server or a empty file.
```
cat > /var/www/html/ks/<mac-address>/meta-data <<EOF
instance-id: ubuntu-server
EOF
```

Note:

These files must be placed under /var/www/html/ks/ if different configurations are desired for different servers.

DHCP and TFTP configuration

Assuming that DHCP is already installed and configured as part of gateway deployment, only additional configuration need to be added for PXE boot. TFTP server that is part of dnsmasq need to be also configured.

See Gateway DHCP/DNS Configuration

Step 1. Create TFTP server directory

sudo mkdir /srv/tftp
sudo mkdir /srv/tftp/grub
sudo mkdir /srv/tftp/pxelinux.cfg

Step 2. Configure dnsmasq

Edit file /etc/dnsmasq.d/dnsmasq.conf adding the following lines

# Enabling logs
log-queries
log-dhcp
log-facility=/var/log/dnsmasq.log

# Enabling PXE boot x86 servers
# Legacy BIOS boot
dhcp-boot=pxelinux.0
# UEFI boot
dhcp-match=set:efi-x86_64,option:client-arch,7
dhcp-boot=tag:efi-x86_64,bootx64.efi

# Enable-tftp
enable-tftp
tftp-root=/srv/tftp

Step 3. Restart dnsmasq service
```
sudo systemctl restart dnsmasq
```

Serving boot loading files via TFTP

Copying kernel and initrd files

Step 1. Download Ubuntu 22.04 server live ISO

wget http://cdimage.ubuntu.com/ubuntu-server/jammy/daily-live/current/jammy-live-server-amd64.iso

Step 2. Mount the ISO file
```
mount jammy-live-server-amd64.iso /mnt
```
Step 3. Copy linux kernel and initrd files to TFTP server root
```
cp /mnt/casper/{vmlinuz,initrd} /srv/tftp/
```

Copying files for UEFI boot

Step 1. Copy the signed shim binary into place:

```shell apt download shim-signed dpkg-deb –fsys-tarfile shim-signed*deb | tar x ./usr/lib/shim/shimx64.efi -O > /srv/tftp/bootx64.efi

Step 2. Copy the signed GRUB binary into place:

apt download grub-efi-amd64-signed
dpkg-deb --fsys-tarfile grub-efi-amd64-signed*deb | tar x ./usr/lib/grub/x86_64-efi-signed/grubnetx64.efi.signed -O > /srv/tftp/grubx64.efi  

Step 3. Copy unicode.pf2

apt download grub-common
dpkg-deb --fsys-tarfile grub-common*deb | tar x ./usr/share/grub/unicode.pf2 -O > /srv/tftp/unicode.pf2

Step 4. Prepare grub.cfg file and copy to /srv/tftp/grub

set default="0"
set timeout=5

if loadfont unicode ; then
  set gfxmode=auto
  set locale_dir=$prefix/locale
  set lang=en_US
fi
terminal_output gfxterm

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
if background_color 44,0,30; then
  clear
fi

function gfxmode {
        set gfxpayload="${1}"
        if [ "${1}" = "keep" ]; then
                set vt_handoff=vt.handoff=7
        else
                set vt_handoff=
        fi
}

set linux_gfx_mode=keep

export linux_gfx_mode

menuentry 'Install Ubuntu 22.04' {
        gfxmode $linux_gfx_mode
        linux vmlinuz ip=dhcp url=http://10.0.0.1/images/jammy-live-server-amd64.iso autoinstall ds=nocloud-net\;s=http://10.0.0.1/ks/${net_default_mac}/ cloud-config-url=/dev/null
        initrd initrd
}

This configuration launch live ISO in autoinstall mode using cloud-init files downloaded from Kick-start web server under ks// (`${net_default_mac}`).

Copying files for legacy boot

Step 1. Copy the pxelinux.0 binary:

apt download pxelinux
dpkg-deb --fsys-tarfile pxelinux*deb | tar x ./usr/lib/PXELINUX/pxelinux.0 -O > /srv/tftp/pxelinux.0

Step 2. Copy syslinux-common packages:

apt download syslinux-common
dpkg-deb --fsys-tarfile pxelinux*deb | tar x ./usr/lib/PXELINUX/pxelinux.0 -O > /srv/tftp/pxelinux.0
dpkg-deb --fsys-tarfile syslinux-common*deb | tar x ./usr/lib/syslinux/modules/bios/ldlinux.c32 -O > /srv/tftp/ldlinux.c32
dpkg-deb --fsys-tarfile syslinux-common*deb | tar x ./usr/lib/syslinux/modules/bios/menu.c32 -O > /build/menu.c32
dpkg-deb --fsys-tarfile syslinux-common*deb | tar x ./usr/lib/syslinux/modules/bios/libutil.c32 -O > /srv/tftp/libutil.c32 

Step 4. Prepare pxe.conf file and copy to /srv/tftp/pxelinux.cfg

PXE looks for a file containing in the name the MAC address, using as separator ‘-‘

01-, ie: 01-10-e7-c6-16-54-10 for MAC address 10:e7:c6:16:54:10

default menu.c32
menu title Ubuntu installer

label jammy
        menu label Install Ubuntu J^ammy (22.04)
        menu default
        kernel vmlinuz
        initrd initrd
        append ip=dhcp url=http://10.0.0.1/images/jammy-live-server-amd64.iso autoinstall ds=nocloud-net;s=http://10.0.0.1/ks/10:e7:c6:16:54:10/ cloud-config-url=/dev/null
prompt 0
timeout 300

This configuration launch live ISO in autoinstall mode using cloud-init files downloaded from Kick-start web server under ks//.

Alternative booting ISO contents via nfsroot.

Netboot installation requires to download the ISO and keep it in RAM, which is not possible if the server RAM is not > 4GB. Testing with servers with less than 5 GB of memory, for example for testing PXE server in virtualized environment like Virtualbox, installation hangs with initdram message “not space left”.

NOTE: Idea from this POST: https://discourse.ubuntu.com/t/netbooting-the-live-server-installer/14510/184

“Extracting or mounting ISO on NFS server, then serve the contents itself, which goes around the requirement of keeping whole ISO in RAM plus additional RAM to do the booting and installation procedure”

Step 1: Intall NFS server
```
sudo apt install nfs-kernel-server
```

Step 2: Make shared NFS directory

sudo mkdir -p /mnt/jammy-live-server-amd64-iso-nfs/

Step 3: Mount ubuntu ISO file

sudo mount /var/www/html/images/jammy-live-server-amd64.iso /mnt/jammy-live-server-amd64-iso-nfs/ 

Configure mount on start

Add to /etc/fstab file the following line

/var/www/html/images/jammy-live-server-amd64.iso /mnt/jammy-live-server-amd64-iso-nfs iso9660 loop 0 0

Step 4: Configure NFS

Edit /etc/exports file adding the following line:

/mnt/jammy-live-server-amd64-iso-nfs 10.0.0.0/24(ro,sync,no_subtree_check)

Step 5: export NFS directory
```
sudo exportfs -a
```

Step 6: Restart NFS service

sudo systemctl restart nfs-kernel-server

Step 7: Show NFS directories
```
sudo exportfs -v
```

Step 8: Update /srv/tftp/grub/grub.cfg file

set default="0"
set timeout=-30

if loadfont unicode ; then
  set gfxmode=auto
  set locale_dir=$prefix/locale
  set lang=en_US
fi
terminal_output gfxterm

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
if background_color 44,0,30; then
  clear
fi

function gfxmode {
        set gfxpayload="${1}"
        if [ "${1}" = "keep" ]; then
                set vt_handoff=vt.handoff=7
        else
                set vt_handoff=
        fi
}

set linux_gfx_mode=keep

export linux_gfx_mode

menuentry 'Install Ubuntu 22.04' {
        gfxmode $linux_gfx_mode
        linux vmlinuz netboot=nfs nfsroot=10.0.0.10:/mnt/jammy-live-server-amd64-iso-nfs ip=dhcp  autoinstall ds=nocloud-net\;s=http://10.0.0.10/ks/ cloud-config-url=/dev/null
        initrd initrd
}

References

Last Update: Jun 24, 2023

Comments:

On this page: