Introduction
In my last post, I explored running the Xen hypervisor on two deprecated versions of CentOS. In this post, I return to the present, setting up Xen on Debian 12 and 13, while using some of the techniques learned in the previous post. As is the case with a lot of my posts, this will involve a variety of topics, including Linux, networking, Perl, cloud-init, and more.
Installing Xen
All of the hosts used for this exercise started off with clean installations of Debian 12 or 13, running in command line mode (no GUI). From what I can tell, the differences between 12 and 13 are few, at least for non-GUI systems. I used a variety of hosts for this exercise, the newest being 10 years old.
Installing Xen itself is straightforward. Simply run: sudo apt install ‐‐no-install-recommends xen-hypervisor xen-hypervisor-common xen-utils bridge-utils (following this guide). The ‐‐no-install-recommends ensures that apt will not try to install a GUI or any other unnecessary packages. I also installed the packages vlan and qemu-utils to add VLAN capabilities to the host and utilize QCOW2 disks, respectively. Before rebooting, I also edited /etc/default/grub.d/xen.cfg, uncommented the parameter GRUB_CMDLINE_XEN_DEFAULT, and set it to “dom0_mem=1024M,max:1024M loglvl=all guest_loglvl=all” (following the “best practices” guide here). This sets a limit on how much RAM is allocated to the host hypervisor (you could probably get away with less). Run sudo update-grub to apply the changes and reboot the system to enable the Xen kernel.
Finally, in addition to the above, I installed Nginx to serve up kickstart and OS installation files for Enterprise Linux.
The below Ansible playbook can also be used to configure many aspects of a system used for this exercise. Before running it, please review it and remove any tasks that are irrelevant to your use (you may not want to use dnsmasq, for example).
---
- hosts: all
vars:
xen_mem: 1024
tasks:
- name: Add the OS repo if needed
ansible.builtin.lineinfile:
path: /etc/apt/sources.list
regexp: 'contrib main non-free-firmware'
line: "deb http://deb.debian.org/debian {{ ansible_facts['distribution_release'] }} contrib main non-free-firmware"
- name: Install required packages
ansible.builtin.apt:
name:
- bridge-utils
- dnsmasq
- genisoimage
- libconfig-tiny-perl
- libtemplate-perl
- nginx
- qemu-utils
- xen-hypervisor
- xen-hypervisor-common
- xen-utils
- vlan
state: present
update_cache: true
install_recommends: false
- name: Set RAM for Xen Dom0
ansible.builtin.lineinfile:
path: /etc/default/grub.d/xen.cfg
regexp: '^(#)?GRUB_CMDLINE_XEN_DEFAULT'
line: "GRUB_CMDLINE_XEN_DEFAULT=\"dom0_mem={{ xen_mem }}M,max:{{ xen_mem }}M loglvl=all guest_loglvl=all\""
notify: Run update-grub
- name: Create /srv/xen
ansible.builtin.file:
path: /srv/xen
state: directory
- name: Set /etc/dnsmasq.conf to include .conf files in /etc/dnsmasq.d
ansible.builtin.lineinfile:
path: /etc/dnsmasq.conf
regexp: '^(#)?conf-dir=/etc/dnsmasq.d/,\*.conf'
line: 'conf-dir=/etc/dnsmasq.d/,*.conf'
notify: Restart dnsmasq
handlers:
- name: Run update-grub
ansible.builtin.command: /usr/sbin/update-grub
- name: Restart dnsmasq
ansible.builtin.systemd:
name: dnsmasq
state: restarted
Bridged network configuration
In order for Xen virtual machines to have networking, a network bridge on the host will need to be set up. The steps for this are the same as they would be for KVM. Edit /etc/network/interfaces and change the below (change eno1 to your interface identifier):
# The primary network interface allow-hotplug eno1 iface eno1 inet manual auto br0 iface br0 inet static address 192.168.2.44/24 gateway 192.168.2.1 bridge_ports eno1 bridge_stp off bridge_waitport 0 bridge_fd 0
Set the inet to dhcp instead of static and omit the address and gateway lines if you would prefer to use DHCP instead. After making these changes, restart the networking service with sudo systemctl restart networking. Xen is now ready for use.
Running Enterprise Linux guests
For CentOS 6 and 7, I was able to use the same Perl kickstart script I wrote for my last post. Below are the steps for setting this up, if you are so inclined to run CentOS 6 or 7:
- Install the packages libconfig-tiny-perl and libtemplate-perl (sudo apt install libconfig-tiny-perl libtemplate-perl).
- Install a web server (Nginx or Apache) and create the directory /var/www/html/kickstart. If hosting the OS files on the Xen host, also create /var/www/html/el/6 and/or /var/www/html/el/7
- Mount the CentOS 6 or 7 DVD (sudo mount -o loop CentOS-7-x86_64-DVD-2009.iso /mnt) and copy the contents to /var/www/html/el/7 on the Xen host or another web server hosting the OS files.
- If your Xen server isn’t hosting the OS files, it will at least need to host the image files for bootstrapping the initial installation, initrd.img and vmlinuz. These can be copied from images/pxeboot on the CentOS 6 or 7 DVDs, or from vault.centos.org. Instead of placing them in /var/www/html/el, you could create a directory like /srv/xen/el/7/images/pxeboot and place them here. You will need to set the os_path parameter in prov_xen.cfg to /srv/xen/el.
- Copy prov_xen.pl, prov_xen.cfg, and ks_el6.cfg/ks_el7.cfg to the Xen host. Modify prov_xen.cfg for your environment
- Run the script with something like sudo ./prov_xen.pl -v 7 -r 2048 -p wget,telnet -f -i 192.168.2.12 cent7test
os_url=http://192.168.1.2/el os_path=/srv/xen/el ks_url=http://192.168.1.2/kickstart domain=ridpath.lab root_pw_hash=$6$9sVgwAX.dI5hhZZm$o5gF1BBqumGFEHns8nZVWGBQVAJgCK7HCctxTueHO.8zT0LnHAGVwLnxI1N1Uuavo6y.hPhf7e5Y2Nbk.PNrB1
#!/usr/bin/perl -w
use strict;
use Config::Tiny;
use Getopt::Long;
use Template;
# Subroutine for generating MAC addresses
sub mac_gen {
my @m;
my $x = 0;
while ($x < 3) {
$m[$x] = int(rand(256));
$x++;
}
my $mac = sprintf("00:16:3E:%02X:%02X:%02X", @m);
return $mac;
}
my $config_file = 'prov_xen.cfg';
my $config = Config::Tiny->read($config_file) || die "Unable to open $config_file\n";
# Parse config file
my $image_path = $config->{_}->{image_path} || '/srv/xen';
my $ks_path = $config->{_}->{ks_path} || '/var/www/html/kickstart';
my $os_path = $config->{_}->{os_path} || '/var/www/html/centos';
my $os_url = $config->{_}->{os_url} || die "os_url is undefined\n";
my $ks_url = $config->{_}->{ks_url} || die "ks_url is undefined\n";
my $root_pw_hash = $config->{_}->{root_pw_hash} || die "root_pw_hash is undefined\n";
my $netmask = $config->{_}->{netmask} || '255.255.255.0';
# If gateway is undefined, it will be set to xxx.xxx.xxx.1
my $gateway = $config->{_}->{gateway} || '';
my $dns_server = $config->{_}->{dns_server} || '1.1.1.1';
my $domain = $config->{_}->{domain} || '';
my $bridge = $config->{_}->{bridge} || 'br0';
# Parse command line options
my($ip, $pkgs, $autostart, $force, $help);
# RAM is in MB
my $ram = 2048;
my $vcpu = 1;
# Disk is in GB
my $disk_size = 10;
# Default to CentOS 7
my $os_ver = 7;
GetOptions ("ip=s" => \$ip,
"ram=i" => \$ram,
"cpu=i" => \$vcpu,
"disk=i" => \$disk_size,
"pkgs=s" => \$pkgs,
"autostart" => \$autostart,
"force" => \$force,
"version=i" => \$os_ver,
'help|?' => \$help,
);
if (defined($help)) {
print "prov_xen.pl usage: [-f (force)] [-i <ip_address>] [-r <ram_mb>] [-d <disk_gb>] [-c <cpus>] [-p <comma-separated-list-of-packages>] [-v <os_version>] [--autostart] <vm_name>";
exit 0;
}
# Check for a valid EL major release number
my @os_vers = (6..7);
unless (grep { $_ == $os_ver } @os_vers) {
die "$os_ver is not a valid OS version. It must be " . join(', ', @os_vers) . ".\n";
}
my $name = $ARGV[0] || die "You must specify a name for the virtual machine.\n";
my $kernel_path = "${os_path}/${os_ver}/images/pxeboot";
my $ks = "${ks_path}/${name}.cfg";
my $mac = &mac_gen();
$os_url .= "/${os_ver}";
my $xen_cmd = '/usr/sbin/xl';
my $xen_create_cmd = "$xen_cmd create -c /etc/xen/${name}.cfg";
unless (-d $image_path) {
system("mkdir -p $image_path");
}
# Checks if the VM already exists.
# If -f is specified, the VM is stopped and deleted.
# Otherwise, fail the script.
if (-f "/etc/xen/${name}.cfg") {
if (defined($force)) {
open(XM_LIST, "$xen_cmd list |") || die "$xen_cmd list failed!\n";
while (my $line = <XM_LIST>) {
if ($line =~ /^$name/) {
system("$xen_cmd destroy $name");
}
}
close(XM_LIST);
unlink("/etc/xen/auto/${name}.cfg") if (-f "/etc/xen/auto/${name}.cfg");
} else {
die "A virtual machine with the name $name already exists.\n";
}
}
my $dhcp = 1;
if (defined($ip)) {
$dhcp = 0;
unless ($ip =~ /^(\d{1,3}\.){3}\d{1,3}$/) {
die "$ip failed regex IP test. Exiting.\n";
}
# Set gateway to xxx.xxx.xxx.1 if not set.
if ($gateway eq '') {
$gateway = $ip;
$gateway =~ s/\.\d+$/\.1/;
}
}
# Additional package specified with -p pkg1,pkg2,etc
my @extra_pkgs = qw(sudo vim-enhanced);
if (defined($pkgs)) {
foreach my $pkg (split(/,/, $pkgs)) {
push(@extra_pkgs, $pkg);
}
}
my $hostname = $name;
unless ($domain eq '') {
$hostname .= ".${domain}";
}
# Variables for the kickstart template.
my $ks_vars = {
os_url => $os_url,
root_pw_hash => $root_pw_hash,
dhcp => $dhcp,
ip => $ip,
netmask => $netmask,
gateway => $gateway,
dns_server => $dns_server,
hostname => $hostname,
mac => $mac,
extra_pkgs => \@extra_pkgs,
};
# Parse kickstart template.
my $ks_tt = Template->new();
$ks_tt->process("ks_el${os_ver}.cfg.tt", $ks_vars, $ks) || die $ks_tt->error;
# Variables for Xen config template.
my $xen_vars = {
name => $name,
kernel_path => $kernel_path,
ks_url => $ks_url,
bridge => $bridge,
ram => $ram,
vcpu => $vcpu,
image_path => $image_path,
mac => $mac,
install => 1,
};
# Parse initial Xen config template.
my $ks_xen_tt = Template->new();
$ks_xen_tt->process('xen.tt', $xen_vars, "/etc/xen/${name}.cfg") || die $ks_xen_tt->error;
print "Starting kickstart install of Xen VM ${name}\n.";
system("/usr/bin/qemu-img create -f raw ${image_path}/${name}.img ${disk_size}G") == 0 || die "qemu-img create -f raw ${image_path}/${name}.img ${disk_size}G failed!\n";
system($xen_create_cmd) == 0 || die "$xen_create_cmd failed to run. Check output above.\n";
print "Installation finished. VM can be started with $xen_create_cmd\n";
# Re-parse Xen config template with post-install settings.
$xen_vars->{'install'} = 0;
my $xen_tt = Template->new();
$xen_tt->process('xen.tt', $xen_vars, "/etc/xen/${name}.cfg") || die $xen_tt->error;
if (defined($autostart)) {
print "Setting VM to be autostarted as requested.\n";
symlink("/etc/xen/${name}.cfg", "/etc/xen/auto/${name}.cfg");
}
# Delete kickstart template.
unlink($ks);
install url --url [% os_url %] repo --name=epel --baseurl=http://archives.fedoraproject.org/pub/archive/epel/6/x86_64/ lang en_US keyboard us [% IF dhcp == 1 -%] network --bootproto=dhcp --device=[% mac %] --hostname=[% hostname %] [% ELSE -%] network --bootproto=static --device=[% mac %] --gateway=[% gateway %] --ip=[% ip %] --nameserver=[% dns_server %] --netmask=[% netmask %] --hostname=[% hostname %] [% END -%] rootpw --iscrypted [% root_pw_hash %] firewall --enabled --ssh authconfig selinux --permissive timezone --utc Etc/UTC bootloader --location=mbr text skipx poweroff # Partitioning zerombr clearpart --all autopart %packages --nobase [% FOREACH package = extra_pkgs -%] [% package %] [% END %] %post cat << 'EOF' > /etc/yum.repos.d/CentOS-Base.repo [base] name=CentOS-$releasever - Base baseurl=[% os_url %] gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 EOF cat << 'EOF' > /etc/yum.repos.d/epel.repo [epel] name=epel baseurl=http://archives.fedoraproject.org/pub/archive/epel/6/x86_64/ gpgcheck=0 EOF
install url --url [% os_url %] repo --name=epel --baseurl=http://archives.fedoraproject.org/pub/archive/epel/7/x86_64/ lang en_US keyboard --vckeymap=us [% IF dhcp == 1 -%] network --bootproto=dhcp --device=[% mac %] --hostname=[% hostname %] [% ELSE -%] network --bootproto=static --device=[% mac %] --gateway=[% gateway %] --ip=[% ip %] --nameserver=[% dns_server %] --netmask=[% netmask %] --hostname=[% hostname %] [% END -%] rootpw --iscrypted [% root_pw_hash %] firewall --enabled --ssh authconfig selinux --permissive timezone --ntpservers=0.pool.ntp.org Etc/UTC bootloader --location=mbr text skipx poweroff # Partitioning zerombr clearpart --all autopart --nolvm --nohome --fstype=ext4 %packages [% FOREACH package = extra_pkgs -%] [% package %] [% END %] %end %post cat << 'EOF' > /etc/yum.repos.d/CentOS-Base.repo [base] name=CentOS-$releasever - Base baseurl=[% os_url %] gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 EOF cat << 'EOF' > /etc/yum.repos.d/epel.repo [epel] name=epel baseurl=http://archives.fedoraproject.org/pub/archive/epel/7/x86_64/ gpgcheck=0 EOF %end
For the Xen configuration template, I changed to the newer format for specifying disks (discussed in this page):
name = "[% name %]" memory = "[% ram %]" vcpus = [% vcpu %] disk = [ '[% image_path %]/[% name %].img,,xvda,w', ] vif = [ 'mac=[% mac %],bridge=[% bridge %]', ] [% IF install == 1 -%] kernel = "[% kernel_path %]/vmlinuz" ramdisk = "[% kernel_path %]/initrd.img" extra = "text ks=[% ks_url %]/[% name %].cfg" on_reboot = 'destroy' on_crash = 'destroy' [% ELSE -%] bootloader = "pygrub" on_reboot = 'restart' on_crash = 'restart' [% END -%]
Enterprise Linux (Red Hat/Oracle/AlmaLinux/Rocky, etc.) 8 and 9 lack the Xen paravirtualization drivers in the kernel, so you must run these OS’s in full virtualization mode. In past years there may have been a performance penalty for this, but I don’t know if this is the case anymore on modern hardware (it is much slower on my test systems). To do a kickstart installation on Xen, you can do something similar to the below for AlmaLinux 9:
- Create a disk image with something like: sudo qemu-img -f qcow2 /srv/xen/alma9.qcow2
- Install a web server (Nginx or Apache) and create the directory /var/www/html/kickstart. If hosting the OS files, also create /var/www/html/el/9.
- Mount the AlmaLinux 9 DVD (sudo mount -o loop AlmaLinux-9.6-x86_64-dvd.iso /mnt) and copy the contents to /var/www/html/el/9 on the Xen host or another web server hosting the OS files.
- If your Xen server isn’t hosting the OS files, it will at least need to host the image files for bootstrapping the initial installation, initrd.img and vmlinuz. These can be copied from images/pxeboot on the AlmaLinux DVD. Instead of placing them in /var/www/html/el, you could create a directory like /srv/xen/el/9/images/pxeboot and place them here. You will need to set the os_path parameter in prov_xen.cfg to /srv/xen/el.
- Generate a random MAC address. There are tools out there for doing this.
- Create a kickstart file similar to the ks_el9.cfg code snippet below. This can go in in /var/www/html/kickstart.
- Create a /etc/xen/vm_name.cfg file similar to the hvm.cfg code snippet below.
- Start the installation with sudo xl create -c /etc/xen/vm_name.cfg.
- After the installation completes, comment out or remove the kernel, ramdisk, and extra lines from /etc/xen/vm_name.cfg. Start the VM with sudo xl create /etc/xen/vm_name.cfg. Include -c to start the VM with the console or do sudo xl console vm_name to attach the console later.
url --url http://192.168.1.2/el/9/ lang en_US keyboard --vckeymap=us network --bootproto=dhcp --device=88:99:aa:bb:cc:dd --hostname=alma9.example.com rootpw --iscrypted $6$3hueGOH4Bi3kb3G4$uTygCBj0f8gjlUeFuMLPIQetbhPAleosozQNFiQMnJO68kMsi3Z/0PU8DY.iKlRQaR185M9XBVKQ98Pow/opg1 firewall --enabled --ssh selinux --disabled timezone US/Eastern timesource --ntp-server 0.pool.ntp.org bootloader --location=mbr text skipx poweroff # Partitioning zerombr clearpart --all --initlabel part / --fstype xfs --size=1 --grow --asprimary part swap --recommended %packages telnet psmisc -cockpit %end
name = "alma9" type = "hvm" memory = "4096" vcpus = 2 disk = [ '/srv/xen/alma9.qcow2,qcow2,xvda,w', ] vif = [ 'mac=88:99:aa:bb:cc:dd,bridge=br0', ] kernel = "/srv/xen/el/9/vmlinuz" ramdisk = "/srv/xen/el/9/initrd.img" extra = "ip=dhcp inst.ks=http://192.168.1.2/kickstart/ks_el9.cfg inst.text console=hvc0" on_reboot = 'restart' on_crash = 'restart'
But of course this prompted yet another revision of my Perl provisioning script. The modifications were few: the OS versions were limited to EL8-9 (10 does not appear to work, at least on my system) and the Xen template was modified to be HVM instead of PV. qemu-img is used for disk image creation instead of dd. Since there is only a single difference between the EL8 and 9 templates (how the NTP server is specified), I used a single template for both versions. Below is the script and the template files.
os_url=http://192.168.1.2/el os_path=/srv/xen/el ks_url=http://192.168.1.2/kickstart domain=ridpath.lab root_pw_hash=$6$9sVgwAX.dI5hhZZm$o5gF1BBqumGFEHns8nZVWGBQVAJgCK7HCctxTueHO.8zT0LnHAGVwLnxI1N1Uuavo6y.hPhf7e5Y2Nbk.PNrB1
#!/usr/bin/perl -w
use strict;
use Config::Tiny;
use Getopt::Long;
use Template;
# Subroutine for generating MAC addresses
sub mac_gen {
my @m;
my $x = 0;
while ($x < 3) {
$m[$x] = int(rand(256));
$x++;
}
my $mac = sprintf("00:16:3E:%02X:%02X:%02X", @m);
return $mac;
}
my $config_file = 'prov_xen.cfg';
my $config = Config::Tiny->read($config_file) || die "Unable to open $config_file\n";
# Parse config file
my $image_path = $config->{_}->{image_path} || '/srv/xen';
my $ks_path = $config->{_}->{ks_path} || '/var/www/html/kickstart';
my $os_path = $config->{_}->{os_path} || '/var/www/html/el';
my $os_url = $config->{_}->{os_url} || die "os_url is undefined\n";
my $ks_url = $config->{_}->{ks_url} || die "ks_url is undefined\n";
my $root_pw_hash = $config->{_}->{root_pw_hash} || die "root_pw_hash is undefined\n";
my $netmask = $config->{_}->{netmask} || '255.255.255.0';
# If gateway is undefined, it will be set to xxx.xxx.xxx.1
my $gateway = $config->{_}->{gateway} || '';
my $dns_server = $config->{_}->{dns_server} || '1.1.1.1';
my $domain = $config->{_}->{domain} || '';
my $bridge = $config->{_}->{bridge} || 'br0';
# Parse command line options
my($ip, $pkgs, $autostart, $force, $help);
# RAM is in MB
my $ram = 4096;
my $vcpu = 1;
# Disk is in GB
my $disk = 10;
# Default to CentOS 9
my $os_ver = 9;
GetOptions ("ip=s" => \$ip,
"ram=i" => \$ram,
"cpu=i" => \$vcpu,
"disk=i" => \$disk,
"pkgs=s" => \$pkgs,
"autostart" => \$autostart,
"force" => \$force,
"version=i" => \$os_ver,
'help|?' => \$help,
);
if (defined($help)) {
print "ks_el_xen.pl usage: [-f (force)] [-i <ip_address>] [-r <ram_mb>] [-d <disk_gb>] [-c <cpus>] [-p <comma-separated-list-of-packages>] [-v <os_version>] [--autostart] <vm_name>";
exit 0;
}
# Check for a valid EL major release number
my @os_vers = (8..9);
unless (grep { $_ == $os_ver } @os_vers) {
die "$os_ver is not a valid OS version. It must be " . join(', ', @os_vers) . ".\n";
}
my $name = $ARGV[0] || die "You must specify a name for the virtual machine.\n";
my $kernel_path = "${os_path}/${os_ver}";
my $ks = "${ks_path}/${name}.cfg";
if ($disk < 10) {
die "You must specify at least 10 (GB) for the disk.\n";
}
if ($ram < 4096) {
die "You must specify at least 4096 (MB) of RAM.\n";
}
my $mac = &mac_gen();
$os_url .= "/${os_ver}";
my $xen_cmd = '/usr/sbin/xl';
my $xen_create_cmd = "$xen_cmd create -c /etc/xen/${name}.cfg";
unless (-d $image_path) {
system("mkdir -p $image_path");
}
# Checks if the VM already exists.
# If -f is specified, the VM is stopped and deleted.
# Otherwise, fail the script.
if (-f "/etc/xen/${name}.cfg") {
if (defined($force)) {
open(XM_LIST, "$xen_cmd list |") || die "$xen_cmd list failed!\n";
while (my $line = <XM_LIST>) {
if ($line =~ /^$name/) {
system("$xen_cmd destroy $name");
}
}
close(XM_LIST);
unlink("/etc/xen/auto/${name}.cfg") if (-f "/etc/xen/auto/${name}.cfg");
unlink("${image_path}/${name}") if (-f "${image_path}/${name}");
} else {
die "A virtual machine with the name $name already exists.\n";
}
}
my $dhcp = 1;
if (defined($ip)) {
$dhcp = 0;
unless ($ip =~ /^(\d{1,3}\.){3}\d{1,3}$/) {
die "$ip failed regex IP test. Exiting.\n";
}
# Set gateway to xxx.xxx.xxx.1 if not set.
if ($gateway eq '') {
$gateway = $ip;
$gateway =~ s/\.\d+$/\.1/;
}
}
# Additional package specified with -p pkg1,pkg2,etc
my @extra_pkgs = qw(sudo vim-enhanced);
if (defined($pkgs)) {
foreach my $pkg (split(/,/, $pkgs)) {
push(@extra_pkgs, $pkg);
}
}
my $hostname = $name;
unless ($domain eq '') {
$hostname .= ".${domain}";
}
# Variables for the kickstart template.
my $ks_vars = {
os_url => $os_url,
root_pw_hash => $root_pw_hash,
dhcp => $dhcp,
ip => $ip,
netmask => $netmask,
gateway => $gateway,
dns_server => $dns_server,
hostname => $hostname,
mac => $mac,
os_ver => $os_ver,
extra_pkgs => \@extra_pkgs,
};
# Parse kickstart template.
my $ks_tt = Template->new();
$ks_tt->process("ks_el.cfg.tt", $ks_vars, $ks) || die $ks_tt->error;
# Variables for Xen config template.
my $xen_vars = {
name => $name,
kernel_path => $kernel_path,
ks_url => $ks_url,
bridge => $bridge,
ram => $ram,
vcpu => $vcpu,
image_path => $image_path,
mac => $mac,
install => 1,
};
# Parse initial Xen config template.
my $ks_xen_tt = Template->new();
$ks_xen_tt->process('xen_hvm.tt', $xen_vars, "/etc/xen/${name}.cfg") || die $ks_xen_tt->error;
print "Starting kickstart install of Xen VM ${name}\n.";
system("/usr/bin/qemu-img create -f qcow2 ${image_path}/${name}.qcow2 ${disk}G") == 0 || die qemu-img create -f qcow2 ${image_path}/${name}.qcow2 ${disk}G failed!\n";
system($xen_create_cmd) == 0 || die "$xen_create_cmd failed to run. Check output above.\n";
print "Installation finished. VM can be started with $xen_create_cmd\n";
# Re-parse Xen config template with post-install settings.
$xen_vars->{'install'} = 0;
my $xen_tt = Template->new();
$xen_tt->process('xen_hvm.tt', $xen_vars, "/etc/xen/${name}.cfg") || die $xen_tt->error;
if (defined($autostart)) {
print "Setting VM to be autostarted as requested.\n";
symlink("/etc/xen/${name}.cfg", "/etc/xen/auto/${name}.cfg");
}
# Delete kickstart template.
unlink($ks);
url --url [% os_url %] lang en_US keyboard --vckeymap=us [% IF dhcp == 1 -%] network --bootproto=dhcp --device=[% mac %] --hostname=[% hostname %] [% ELSE -%] network --bootproto=static --device=[% mac %] --gateway=[% gateway %] --ip=[% ip %] --nameserver=[% dns_server %] --netmask=[% netmask %] --hostname=[% hostname %] [% END %] rootpw --iscrypted [% root_pw_hash %] firewall --enabled --ssh selinux --disabled [% IF os_ver == 8 -%] timezone US/Eastern --ntpservers=0.pool.ntp.org [% ELSE -%] timezone US/Eastern timesource --ntp-server 0.pool.ntp.org [% END %] bootloader --location=mbr text skipx poweroff # Partitioning zerombr clearpart --all --initlabel autopart --noboot --nohome --nolvm --fstype=xfs %packages [% FOREACH package = extra_pkgs -%] [% package %] [% END %] -cockpit %end
name = "[% name %]" type = "hvm" memory = "[% ram %]" vcpus = [% vcpu %] disk = [ '[% image_path %]/[% name %].qcow2,qcow2,xvda,w', ] vif = [ 'mac=[% mac %],bridge=[% bridge %]' ] [% IF install == 1 -%] kernel = "[% kernel_path %]/vmlinuz" ramdisk = "[% kernel_path %]/initrd.img" extra = "ip=dhcp inst.ks=[% ks_url %]/[% name %].cfg inst.text console=hvc0" [% END -%] on_reboot = 'restart' on_crash = 'restart'
Moving on to cloud images
While kickstarting an CentOS 5 system on Xen used to take under two minutes, this is certainly no longer the case. Whether you’re installing Debian or Enterprise Linux from scratch, it can take 10-15 minutes for the installation to complete. Who really has time to wait around for such a thing, when most Linux distributions publish pre-installed cloud images? These are images that already contain a running OS, and can be further configured with cloud-init. Below are some examples:
Of note, the Debian images support modes paravirtualization, full virtualization, and the hybrid PVHVM. I found that PVHVM mode would not run cloud-init properly for some reason, so I skipped it and just did paravirtualization and full virtualization.
Provisioning a cloud image manually is relatively straightforward. First, you’ll need to create a cloud-init ISO. This will consist of two files, meta-data and user-data. meta-data contains the parameters instance-id and local-hostname, both of which I usually set to the hostname of the instance:
instance-id: mydeb13vm local-hostname: mydeb13vm
user-data, on the other hand, is somewhat more complicated and contains tasks that you want to run when the instance is provisioned. For example, on EL you might use it to disable SELinux or install packages. In this example I kept it relatively simple, only adding a user account and setting the root password. Per the cloud-init reference, the password hashes can be generated with: mkpasswd –method=SHA-512 –rounds=500000 (the one-liner python3 -c ‘import crypt,getpass;pw=getpass.getpass();print(crypt.crypt(pw) if (pw==getpass.getpass(“Confirm: “)) else exit())’ works also). More information on the different modules can be found in the cloud-init module reference.
#cloud-config
users:
- name: ansible_user
passwd: $6$7JnhIhkmDNu4rkr8$1NPhvMbqW.dsPmQBgQ6fIbptd2mqw49byvMdjIldnlg.AW44PR7YNa0esI9lXCS2PY8XIIEqdY4.kBmyvQUuJ.
ssh_authorized_keys:
- ssh-ed25519 pubkey1 mattpubkey1
- ssh-ed25519 pubkey1 mattpubkey1
groups: sudo
shell: /bin/bash
lock_passwd: false
ssh_pwauth: true
ssh_deletekeys: true
chpasswd:
expire: false
users:
- name: root
password: $6$7JnhIhkmDNu4rkr8$1NPhvMbqW.dsPmQBgQ6fIbptd2mqw49byvMdjIldnlg.AW44PR7YNa0esI9lXCS2PY8XIIEqdY4.kBmyvQUuJ.
Next, generate an ISO with the command genisoimage -output /path/to/iso.iso -V cidata -r -J user-data meta-data. After this, download the appropriate cloud image. If provisioning a paravirtualized Debian instance, you will need the debian-ver-generic-amd64.raw file instead of the .qcow2 file. Fully-virtualized guests can use the .qcow2 file, which is smaller. Copy this to wherever you store your Xen images, such as /srv/xen. You can also resize it with sudo qemu-img resize -f raw/qcow2 image_file sizeG.
Finally, create the Xen configuration file, in /etc/xen/instance.cfg. Below is one I created for a Debian 13 paravirtualized guest:
name = "debian13" memory = "2048" type = "pv" vcpus = 1 disk = [ '/srv/xen/debian13.img,,xvda,w', '/tmp/debian_ci.iso,,xvdb,cdrom' ] vif = [ 'mac=88:99:aa:bb:cc:dd,bridge=br0' ] bootloader = 'pygrub' on_reboot = 'restart' on_crash = 'restart'
And then boot it with sudo xl create -c /etc/xen/instance.cfg. After booting, you should be able to log in at the console with the password you set. Setting up an Enterprise Linux image is similar, except that the instance will be fully-virtualized and will use a qcow2 image. In my example, I also disabled SELinux on provisioning the instance in the user-data file:
#cloud-config
users:
- name: ansible_user
passwd: $6$7JnhIhkmDNu4rkr8$1NPhvMbqW.dsPmQBgQ6fIbptd2mqw49byvMdjIldnlg.AW44PR7YNa0esI9lXCS2PY8XIIEqdY4.kBmyvQUuJ.
ssh_authorized_keys:
- ssh-ed25519 pubkey1 mattpubkey1
- ssh-ed25519 pubkey1 mattpubkey1
sudo: ALL=(ALL) ALL
shell: /bin/bash
lock_passwd: false
ssh_pwauth: true
ssh_deletekeys: true
chpasswd:
expire: false
users:
- name: root
password: $6$7JnhIhkmDNu4rkr8$1NPhvMbqW.dsPmQBgQ6fIbptd2mqw49byvMdjIldnlg.AW44PR7YNa0esI9lXCS2PY8XIIEqdY4.kBmyvQUuJ.
runcmd:
- [ sed, -i, s/^SELINUX=.*$/SELINUX=disabled/, /etc/selinux/config ]
- [ setenforce, 0 ]
Then generate the ISO with: genisoimage -output /path/to/iso.iso -V cidata -r -J user-data meta-data. After that, create a /etc/xen/instance.cfg file that looks similar to below:
name = "alma9test" type = "hvm" memory = "4096" vcpus = 1 disk = [ '/srv/xen/alma9test.qcow2,qcow2,xvda,w', '/tmp/alma9test.iso,,xvdb,cdrom' ] vif = [ '88:99:aa:bb:cc:dd,bridge=br0' ] on_reboot = 'restart' on_crash = 'restart'
Then start the instance with sudo xl create -c /etc/xen/instance.cfg. On my particular system, HVM instances take longer to boot than PV instances.
I also turned the above steps into a Perl script (of course), modifying the kickstart provisioning script for this purpose. Enterprise Linux and Debian-like systems use distinct cloud-init user-data files, as you will probably want to configure these differently. I used the user-data_el and user-data_deb files from above. I also gave the user the option of specifying an alternative user-data file and a different network bridge. Below are the steps for setting up the script:
- Install the packages libconfig-tiny-perl, libtemplate-perl, and genisoimage (sudo apt install libconfig-tiny-perl libtemplate-perl genisoimage).
- Copy the cloud image to /srv/xen (for example: debian-12-generic-amd64.raw).
- Copy prov_xen_cloud.pl and user-data_el/user-data_deb to the Xen host.
- Run the script with something like sudo ./prov_xen_cloud.pl -r 2048 -d 20 -o debian12 deb12test
#!/usr/bin/perl -w
use strict;
use Config::Tiny;
use Getopt::Long;
use Template;
sub mac_gen {
my @m;
my $x = 0;
while ($x < 3) {
$m[$x] = int(rand(256));
$x++;
}
my $mac = sprintf("00:16:3E:%02X:%02X:%02X", @m);
return $mac;
}
my $config_file = 'prov_xen.cfg';
my $config;
if (-f $config_file) {
$config = Config::Tiny->read($config_file) || die "Unable to open $config_file\n";
}
my $image_path = $config->{_}->{image_path} || '/srv/xen';
my $bridge = $config->{_}->{bridge} || 'br0';
# Parse command line options
my($autostart, $force, $help);
# RAM is in MB
my $ram = 2048;
my $vcpu = 1;
# Disk is in GB
my $disk = 10;
# Default to Debian 13
my $os = 'debian13';
my $mac = &mac_gen();
my $user_data_file = '';
GetOptions ("ram=i" => \$ram,
"bridge=s" => \$bridge,
"cpu=i" => \$vcpu,
"disk=i" => \$disk,
"autostart" => \$autostart,
"force" => \$force,
"os=s" => \$os,
"udfile=s" => \$user_data_file,
'help|?' => \$help,
);
if (defined($help)) {
print "prov_xen_cloud.pl usage: [-f (force)] [-b <bridge>] [-r <ram_mb>] [-d <disk_gb>] [-c <cpus>] [-o <os>] [--autostart] [-u <user-data-file>] <vm_name>";
exit 0;
}
my $name = $ARGV[0] || die "You must specify a name for the virtual machine.\n";
my %os_hash = (
'alma9' => {
'family' => 'el',
'image' => 'AlmaLinux-9-GenericCloud-latest.x86_64.qcow2',
'type' => 'hvm'
},
'debian12' => {
'family' => 'deb',
'image' => 'debian-12-generic-amd64.raw',
'type' => 'pv'
},
'debian13' => {
'family' => 'deb',
'image' => 'debian-13-generic-amd64.raw',
'type' => 'pv'
}
);
# Check for a valid os
unless (grep { $_ eq $os } keys %os_hash) {
die "$os is not a valid OS. It must be " . join(', ', keys %os_hash) . ".\n";
}
my $cloud_image = $image_path . '/' . $os_hash{$os}{'image'};
my $vm_type = $os_hash{$os}{'type'};
my $os_family = $os_hash{$os}{'family'};
if ($user_data_file eq '') {
$user_data_file = "user-data_${os_family}";
}
unless (-f $user_data_file) {
die "user-data file $user_data_file not found!\n";
}
my $xen_cmd = '/usr/sbin/xl';
my $xen_create_cmd = "$xen_cmd create /etc/xen/${name}.cfg";
# Check if bridge interface is valid and present on the system.
unless ($bridge =~ /^br\d{1,4}$/) {
die "Bridge interface identifier invalid. It must be like br0, etc.\n";
}
unless (-f '/usr/sbin/brctl') {
die "bridge-utils are missing!\n";
}
my $bridge_found = 0;
open(BRCTL, '/usr/sbin/brctl show |') || die "brctl show failed!\n";
while (my $line = <BRCTL>) {
if ($line =~ /^$bridge\s+/) {
$bridge_found = 1;
last;
}
}
close(BRCTL);
if ($bridge_found == 0) {
die "Bridge interface $bridge not present!\n";
}
# Checks if the VM already exists.
# If -f is specified, the VM is stopped and deleted.
# Otherwise, fail the script.
if (-f "/etc/xen/${name}.cfg") {
if (defined($force)) {
open(XM_LIST, "$xen_cmd list |") || die "$xen_cmd list failed!\n";
while (my $line = <XM_LIST>) {
if ($line =~ /^$name/) {
system("$xen_cmd destroy $name");
}
}
close(XM_LIST);
unlink("/etc/xen/auto/${name}.cfg") if (-f "/etc/xen/auto/${name}.cfg");
} else {
die "A virtual machine with the name $name already exists.\n";
}
}
# Run qemu-image info on the base image and check if the size is sufficient.
open(QEMU_IMG, "/usr/bin/qemu-img info $cloud_image |") || die "qemu-img info $cloud_image failed!\n";
my($cloud_img_size, $img_fmt);
while (my $line = <QEMU_IMG>) {
if ($line =~ /file format:\s+(qcow2|raw)/) {
$img_fmt = $1;
}
if ($line =~ /virtual size:\s+(\d+)/) {
$cloud_img_size = $1;
}
}
close(QEMU_IMG);
if ($cloud_img_size > $disk) {
die "Disk size ${disk}GB is smaller than the virtual size of disk image ${cloud_image}. It must be at least ${cloud_img_size}GB."
}
# Create the disk image
my $disk_img = "${image_path}/${name}.${img_fmt}";
system("cp $cloud_image $disk_img");
system("/usr/bin/qemu-img resize -f $img_fmt $disk_img ${disk}G") == 0 || die "qemu-img resize -f $img_fmt $disk_img ${disk}G failed!\n";
# Create the cloud-init ISO
mkdir("/tmp/${name}_ci");
system("cp $user_data_file /tmp/${name}_ci/user-data");
open(META_DATA, '>', "/tmp/${name}_ci/meta-data");
print META_DATA "instance-id: ${name}\n";
print META_DATA "local-hostname: ${name}\n";
close(META_DATA);
system("genisoimage -quiet -output /tmp/${name}_ci.iso -V cidata -r -J /tmp/${name}_ci/user-data /tmp/${name}_ci/meta-data") == 0 || die "genisoimage failed!\n";
# Create Xen configuration file
my $xen_vars = {
name => $name,
type => $vm_type,
bridge => $bridge,
ram => $ram,
vcpu => $vcpu,
disk_img => $disk_img,
img_fmt => $img_fmt,
mac => $mac,
install => 1,
};
# Parse initial Xen config template.
my $xen_prov_tt = Template->new();
$xen_prov_tt->process('xen_cloud.tt', $xen_vars, "/etc/xen/${name}.cfg") || die $xen_prov_tt->error;
# Start VM and wait 5 seconds to re-parse config
system($xen_create_cmd) == 0 || die "$xen_create_cmd failed to run. Check output above.\n";
sleep(5);
print "VM $name has been started. You can connect to it with $xen_cmd console ${name}.\n";
print "MAC address is $mac.\n";
# Re-parse Xen config template with post-install settings.
$xen_vars->{'install'} = 0;
my $xen_tt = Template->new();
$xen_tt->process('xen_cloud.tt', $xen_vars, "/etc/xen/${name}.cfg") || die $xen_tt->error;
# Set VM to be autostarted if requested.
if (defined($autostart)) {
print "Setting VM to be autostarted as requested.\n";
symlink("/etc/xen/${name}.cfg", "/etc/xen/auto/${name}.cfg");
}
name = '[% name %]' type = '[% type %]' memory = '[% ram %]' vcpus = [% vcpu %] [% IF install == 1 -%] disk = [ '[% disk_img %],[% img_fmt %],xvda,w', '/tmp/[% name %]_ci.iso,,xvdb,cdrom' ] [% ELSE -%] disk = [ '[% disk_img %],[% image_fmt %],xvda,w' ] [% END -%] vif = [ 'mac=[% mac %],bridge=[% bridge %]' ] [% IF type == 'pv' -%] bootloader = 'pygrub' [% END -%] on_reboot = 'restart' on_crash = 'restart'
Configuring VLAN bridged interfaces
On my home network I have configured three VLANs: a guest/IoT network, the main VLAN with Internet access, and a lab network without Internet access. Initially I configured my Xen systems to use the main VLAN. However, I thought it might be of interest to demonstrate configuring multiple VLANs, with the DHCP server on the lab network placed under the management of the provisioning script.
First, you will need a managed switch and a router configured to host multiple VLANs. In an earlier post of mine I discussed doing this with a Debian or Ubuntu computer, but there are off-the-shelf routers that can be configured to host VLANs as well. This article will focus on configuring a Xen host for VLANs and not on the router or switch configuration.
After hooking the NIC of the Xen host up to a managed switch with the appropriate VLANs, I edited /etc/network/interfaces and set the following. VLAN 20 is the main LAN, while VLAN 40 is the lab LAN:
auto enp2s0 iface enp2s0 inet manual auto enp2s0.20 iface enp2s0.20 inet manual auto enp2s0.40 iface enp2s0.40 inet manual auto br20 iface br20 inet static address 10.0.20.2/24 gateway 10.0.20.1 bridge_ports enp2s0.20 bridge_stp off bridge_waitport 0 bridge_fd 0 auto br40 iface br40 inet static address 192.168.40.2/24 bridge_ports enp2s0.40 bridge_stp off bridge_waitport 0 bridge_fd 0
After making these changes, restart the the networking service with sudo systemctl restart networking.
Configuring dnsmasq to assign consistent IPs
One problem with using the cloud images is that the instances will use DHCP by default when provisioned. Of course you can log into the instance via the console and set a static IP, but what’s the fun in that? You can grab the MAC address and set a reservation on your router, but that is still a manual process. cloud-init has the ability to set static IPs in Debian and Ubuntu, but I’m not clear how this works for EL systems, and I wanted my script to work with both. The solution I came up with for this exercise was to run a dnsmasq DNS/DHCP server on the hypervisor for the lab VLAN instead, similar to what libvirt does for its NAT networks. The provisioning script writes the reservations to a configuration file (/etc/dnsmasq.d/xen.conf) and restarts the dnsmasq service. The solution is a little ugly and it has the limitation of running on one server. I have some thoughts on a multi-server setup, but I will look into that at a later date.
To set up dnsmasq, I first installed it with sudo apt install dnsmasq. I made only one modification to the main configuration file, uncommenting this line: conf-dir=/etc/dnsmasq.d/,*.conf. I then created my own configuration file, /etc/dnsmasq.d/xen.conf that looks like below:
listen-address=::1,127.0.0.1,192.168.40.2 no-resolv no-hosts server=192.168.87.1 domain=ridpath.lab dhcp-range=192.168.40.100,192.168.40.200,12h dhcp-option=3,192.168.40.1 dhcp-option=6,192.168.40.2 # Reservations
An explanation of each option:
- listen-address tells dnsmasq to listen on localhost and the lab IP of the system, set in the previous section. I didn’t want it to listen on my main VLAN, as that already has a DHCP server.
- no-resolv means don’t load the DNS servers out of /etc/resolv.conf.
- no-hosts means don’t load the entries in /etc/hosts.
- server specifies the DNS server to forward requests to. Multiple server lines can be used to specify additional servers.
- domain sets the DHCP domain. This doesn’t work for me yet, but I may need to do some tweaking with cloud-init
- dhcp-range sets the range of IPs to assign
- dhcp-option=3 sets the default gateway
- dhcp-option=6 sets the DNS server for DNS
For the final time in this post, I created yet another version of the Perl provisioning script. This time I moved a few of the steps to subroutines, with the eventual plan of moving them to a separate module for reuse in other scripts. In order to set DNS reservations, the parameter dhcp_net must be set in prov_xen.cfg; it contains the first three octets of the subnet (such as 192.168.40). The script does not check if the bridge identifier matches up with the DHCP subnet; it assumes that you know what you’re doing (which isn’t always a good assumption). To set a DHCP reservation for the instance, include the option -i IP_Address.
domain=ridpath.lab bridge=br40 dhcp_net=192.168.40
#!/usr/bin/perl -w
use strict;
use Config::Tiny;
use Getopt::Long;
use Template;
# Generate a random MAC address.
sub mac_gen {
my @m;
my $x = 0;
while ($x < 3) {
$m[$x] = int(rand(256));
$x++;
}
my $mac = sprintf("00:16:3E:%02X:%02X:%02X", @m);
return $mac;
}
# Regex-validate an IP address.
sub validate_ip {
my $ip = $_[0];
unless ($ip =~ /^(\d{1,3}\.){3}\d{1,3}$/) {
die "$ip failed regex IP test. Exiting.\n";
}
}
# Check if bridge interface is valid.
sub check_bridge {
my $bridge = $_[0];
unless ($bridge =~ /^br\d{1,4}$/) {
die "Bridge interface identifier invalid. It must be like br0, etc.\n";
}
unless (-f '/usr/sbin/brctl') {
die "bridge-utils are missing!\n";
}
my $bridge_found = 0;
open(BRCTL, '/usr/sbin/brctl show |') || die "brctl show failed!\n";
while (my $line = <BRCTL>) {
if ($line =~ /^$bridge\s+/) {
$bridge_found = 1;
last;
}
}
close(BRCTL);
if ($bridge_found == 0) {
die "Bridge interface $bridge not present!\n";
}
}
# Run qemu-image info on the base image and check if the size is sufficient.
sub check_image {
my($cloud_image, $disk_size) = @_;
unless (-f '/usr/bin/qemu-img') {
die "qemu-utils are missing!\n";
}
my($cloud_img_size, $img_fmt);
open(QEMU_IMG, "/usr/bin/qemu-img info $cloud_image |") || die "qemu-img info $cloud_image failed!\n";
while (my $line = <QEMU_IMG>) {
if ($line =~ /file format:\s+(qcow2|raw)/) {
$img_fmt = $1;
}
if ($line =~ /virtual size:\s+(\d+)/) {
$cloud_img_size = $1;
}
}
close(QEMU_IMG);
if ($cloud_img_size > $disk_size) {
die "Disk size ${disk_size}GB is smaller than the virtual size of disk image ${cloud_image}. It must be at least ${cloud_img_size}GB.\n";
}
return($img_fmt);
}
# Create the cloud-init ISO
sub create_ci_iso {
my($name, $user_data_file) = @_;
mkdir("/tmp/${name}_ci");
system("cp $user_data_file /tmp/${name}_ci/user-data");
open(META_DATA, '>', "/tmp/${name}_ci/meta-data");
print META_DATA "instance-id: ${name}\n";
print META_DATA "local-hostname: ${name}\n";
close(META_DATA);
system("genisoimage -quiet -output /tmp/${name}_ci.iso -V cidata -r -J /tmp/${name}_ci/user-data /tmp/${name}_ci/meta-data") == 0 || die "genisoimage failed!\n";
}
# Parse /etc/xen/hostname.cfg
sub parse_xen_conf {
my $name = $_[0];
my $xen_vars = {
name => $name,
type => $_[1],
bridge => $_[2],
ram => $_[3],
vcpu => $_[4],
disk_img => $_[5],
img_fmt => $_[6],
mac => $_[7],
install => $_[8]
};
my $template = $_[9];
my $xen_tt = Template->new();
$xen_tt->process($template, $xen_vars, "/etc/xen/${name}.cfg") || die $xen_tt->error;
}
# Large subroutine to parse a dnsmasq config.
sub dnsmasq_conf {
my($name, $domain, $ip, $mac, $dhcp_net) = @_;
my $fqdn = $name;
unless ($domain eq '') {
$fqdn .= ".${domain}";
}
if ($dhcp_net eq '') {
die "dhcp_net=xxx.xxx.xxx is missing from the config! Unable to assign IP reservation.\n";
}
unless ($ip =~ /^$dhcp_net\.\d{1,3}$/) {
die "IP $ip does not match DHCP subnet ${dhcp_net}.0!\n";
}
unless (-f '/etc/dnsmasq.d/xen.conf') {
die "/etc/dnsmasq.d/xen.conf is missing! Exiting.\n";
}
open(DNSMASQ_IN, '<', '/etc/dnsmasq.d/xen.conf') || die "Unable to open /etc/dnsmasq.d/xen.conf.\n";
open(DNSMASQ_OUT, '>', '/tmp/dnsmasq_xen.conf') || die "Unable to open /tmp/dnsmasq_xen.conf.\n";
while (my $line = <DNSMASQ_IN>) {
if ($line =~ /^dhcp-host=/) {
chomp($line);
my @line_split = split(/,/, $line);
if ($line_split[1] eq $ip) {
if ($line_split[2] eq $name) {
next;
} else {
die "IP $ip already appears to be in use. Check /etc/dnsmasq.d/xen.conf.\n";
}
} else {
print DNSMASQ_OUT "$line\n";
}
} elsif ($line =~ /^address=\/$fqdn\//) {
next;
} else {
print DNSMASQ_OUT $line;
}
}
close(DNSMASQ_IN);
print DNSMASQ_OUT "dhcp-host=${mac},${ip},${name}\n";
print DNSMASQ_OUT "address=/${fqdn}/${ip}\n";
close(DNSMASQ_OUT);
system('cp /tmp/dnsmasq_xen.conf /etc/dnsmasq.d/xen.conf');
open(LEASES_IN, '<', '/var/lib/misc/dnsmasq.leases') || die "Unable to open /var/lib/misc/dnsmasq.leases.\n";
open(LEASES_OUT, '>', '/tmp/dnsmasq.leases') || die "Unable to open /tmp/dnsmasq.leases.\n";
while (my $line = <LEASES_IN>) {
if ($line =~ /\s+$ip\s+/) {
next;
} else {
print LEASES_OUT $line;
}
}
close(LEASES_IN);
close(LEASES_OUT);
system('cp /tmp/dnsmasq.leases /var/lib/misc/dnsmasq.leases');
system('/usr/bin/systemctl restart dnsmasq') == 0 || die "dnsmasq failed to restart!\n";
}
# Read in the config.
sub read_config {
my $config_file = $_[0];
my $config = Config::Tiny->read($config_file) || die "Unable to open $config_file\n";
my $image_path = $config->{_}->{image_path} || '/srv/xen';
my $bridge = $config->{_}->{bridge} || 'br0';
my $dhcp_net = $config->{_}->{dhcp_net} || '';
my $domain = $config->{_}->{domain} || '';
if (($dhcp_net ne '') && !($dhcp_net =~ /^(\d{1,3}\.){2}\d{1,3}$/)) {
die "Invalid format for dhcp_net in config. It must be xxx.xxx.xxx\n";
}
return($image_path, $bridge, $dhcp_net, $domain);
}
my($image_path, $bridge, $dhcp_net, $domain) = &read_config('prov_xen.cfg');
# Parse command line options
my($autostart, $force, $help);
my $ip = '';
# RAM is in MB
my $ram = 2048;
my $vcpu = 1;
# Disk is in GB
my $disk_size = 10;
# Default to Debian 13
my $os = 'debian13';
my $mac = &mac_gen();
my $user_data_file = '';
GetOptions ("ram=i" => \$ram,
"bridge=s" => \$bridge,
"cpu=i" => \$vcpu,
"disk=i" => \$disk_size,
"ip=s" => \$ip,
"autostart" => \$autostart,
"force" => \$force,
"os=s" => \$os,
"udfile=s" => \$user_data_file,
'help|?' => \$help,
);
if (defined($help)) {
print "prov_xen_cloud.pl usage: [-f (force)] [-b <bridge>] [-r <ram_mb>] [-d <disk_gb>] [-c <cpus>] [-o <os>] [--autostart] [-u <user-data-file>] <vm_name>";
exit 0;
}
my $name = $ARGV[0] || die "You must specify a name for the virtual machine.\n";
my %os_hash = (
'alma9' => {
'family' => 'el',
'image' => 'AlmaLinux-9-GenericCloud-latest.x86_64.qcow2',
'type' => 'hvm'
},
'debian12' => {
'family' => 'deb',
'image' => 'debian-12-generic-amd64.raw',
'type' => 'pv'
},
'debian13' => {
'family' => 'deb',
'image' => 'debian-13-generic-amd64.raw',
'type' => 'pv'
}
);
# Check for a valid os
unless (grep { $_ eq $os } keys %os_hash) {
die "$os is not a valid OS. It must be " . join(', ', keys %os_hash) . ".\n";
}
my $cloud_image = $image_path . '/' . $os_hash{$os}{'image'};
my $vm_type = $os_hash{$os}{'type'};
my $os_family = $os_hash{$os}{'family'};
if ($user_data_file eq '') {
$user_data_file = "user-data_${os_family}";
}
unless (-f $user_data_file) {
die "user-data file $user_data_file not found!\n";
}
my $xen_cmd = '/usr/sbin/xl';
my $xen_create_cmd = "$xen_cmd create /etc/xen/${name}.cfg";
&check_bridge($bridge);
# Checks if the VM already exists.
# If -f is specified, the VM is stopped and deleted.
# Otherwise, fail the script.
if (-f "/etc/xen/${name}.cfg") {
if (defined($force)) {
open(XM_LIST, "$xen_cmd list |") || die "$xen_cmd list failed!\n";
while (my $line = <XM_LIST>) {
if ($line =~ /^$name/) {
system("$xen_cmd destroy $name");
}
}
close(XM_LIST);
unlink("/etc/xen/auto/${name}.cfg") if (-f "/etc/xen/auto/${name}.cfg");
} else {
die "A virtual machine with the name $name already exists.\n";
}
}
my($img_fmt) = &check_image($cloud_image, $disk_size);
# Configure dnsmasq for a DHCP reservation
unless ($ip eq '') {
&validate_ip($ip);
&dnsmasq_conf($name, $domain, $ip, $mac, $dhcp_net);
}
# Create the disk image
my $disk_img = "${image_path}/${name}.${img_fmt}";
system("cp $cloud_image $disk_img");
system("/usr/bin/qemu-img resize -f $img_fmt $disk_img ${disk_size}G") == 0 || die "qemu-img resize -f $img_fmt $disk_img ${disk_size}G failed!\n";
# Create cloud-init ISO
&create_ci_iso($name, $user_data_file);
# Parse initial Xen config template.
&parse_xen_conf($name, $vm_type, $bridge, $ram, $vcpu, $disk_img, $img_fmt, $mac, 1, 'xen_cloud.tt');
# Start VM and wait 5 seconds to re-parse config
system($xen_create_cmd) == 0 || die "$xen_create_cmd failed to run. Check output above.\n";
sleep(5);
print "VM $name has been started. You can connect to it with $xen_cmd console ${name}.\n";
if ($ip eq '') {
print "MAC address is $mac.\n";
}
&parse_xen_conf($name, $vm_type, $bridge, $ram, $vcpu, $disk_img, $img_fmt, $mac, 0, 'xen_cloud.tt');
# Set VM to be autostarted if requested.
if (defined($autostart)) {
print "Setting VM to be autostarted as requested.\n";
symlink("/etc/xen/${name}.cfg", "/etc/xen/auto/${name}.cfg");
}
“Bonus” section: paravirtualized guests on a CPU without virtualization extensions
One thing I really wanted to try was running Xen guests on a CPU without virtualization extensions, to prove that it can be done. And yes, it does work, with Debian 13. The system I attempted this on has an Intel Atom D2550 running at 1.86GHz and 4GB of RAM. I was able to install Xen on this system and do the following:
- Provision a CentOS 6 instance using the prov_xen.pl kickstarting script.
- Provision a Debian 12 instance using the prov_xen_cloud.pl script.
And yes, it was slow, but not unusable. It demonstrates how Xen can function on low-spec hardware.
Conclusion
Is there any point to anything I experimented with in this post? Probably not. Xen, Perl scripts, kickstart files, etc. are all somewhat “retro” at this point. There are more modern and well-charted ways of spinning up virtual machines, with tools like Terraform. But for me it was a fun exercise, giving me a chance to dive into how things work and write scripts to make tasks repeatable. Ultimately, my goal is to have fun in experimenting with this stuff, though if it is of use to someone, that is awesome. In the future, my goal is to write a sort of “Perl Terraform-ish” kind of thing for this. As always, thanks for reading!