With the release of XenServer 5.6 Feature Pack 1 it was due time to upgrade my Homelab to the latest version. So eagerly I started to burn the downloaded iso files for the install and linux supplement pack CDs and burned the ISO files to CDs. With a quick scan through the Admin Guide, I figured I understood the process and made my first mistake.
I performed the upgrade in the wrong order, so had to do some troubleshooting.
Prepping the XenServer Homelab
After a quick scan of the upgrade steps from the XenServer 5.6 FP1 Admin Guide, I started with an upgrade of XenCenter. Installing the new msi to upgrade my XenCenter 5.5 to the 5.6 FP1 version was easy.
Then I decided to get rid of some old VMs (I had no idea what VMs were still current, with a couple of months spent on the vShpere exam preps). Unfortunately I also deleted the DC for my newly created domain, but that’s a different post.
So with only the required VMs, all shut down, I was ready to start. And that’s when I made my first (rookie) mistake.
Upgrading a XenServer Pool the wrong way
I started with my Pool Master XenServer, making sure no VMs were attached to the server, I set the XenServer into Maintenance Mode and answered the question that popped up, by transfering the Pool Master role to the second XenServer. And after that started with an upgrade of my XenServer (now a member server of the pool). Big mistake, as this ensured my upgraded Server could nog longer communicate with the Pool Master, as the new Pool Master still had a lower version. Therefore no Pool information was found, no NICs were discovered and trying to make changes resulted in some undefined errors.
Lucky for my I only started with a Homelab upgrade, so I could easily fix my problems, by upgrading the second (new Pool Master) XenServer and afterwards, reboot my first upgraded server, to ensure it did find the Pool Master this time and got the right information to start up with NIC functionality and config info. All worked perfectly after that action.
Upgrading a XenServer Pool the right way
To upgrade a XenServer Pool the right way, you start with the Pool Master. First you XenMotion all running VMs to a different XenServer, ensure no VMs are attached to the Pool Master. Do not enter any Maintenance Mode, just reboot your Pool Master from the install media and upgrade your XenServer. Afterwards you can XenMotion the running VMs back to the Pool Master and start upgrading the other servers. Make sure you have HA disabled during the upgrade process, to keep control of the VMs and their running states.
Troubleshooting a failed upgrade
Thanks to the Citrix Forum, clear directives are available to troubleshoot a failed upgrade. The following commands can be useful to correct NIC settings on your XenServer.
* List the current interfaces:
[root@<name-xs> ~]# xe pif-list
uuid ( RO) : [uuid-nic]
device ( RO): eth0
currently-attached ( RO): true
VLAN ( RO): -1
network-uuid ( RO): [uuid-network]
uuid ( RO) : [uuid-nic]
device ( RO): eth0
currently-attached ( RO): true
VLAN ( RO): -1
network-uuid ( RO): [uuid-network]
Which uses the following additional syntax:
- uuid-nic: A unique identifier for the NIC.
- uuid-network: A unique identifier for the network.
In this case I get two different uuid-nic values for the pysical eth0 as it shows me the pool-wide settings for all XenServer eth0 physical NICs. In order to find the right uuid-nic, the following command specifies more details about each entry.
* List detailed info for each interface:
[root@<name-xs> ~]# xe pif-param-list uuid=[uuid-nic-eth0]
uuid ( RO) : [uuid-nic-eth0]
device ( RO): eth0
MAC ( RO): [mac-eth0]
physical ( RO): true
currently-attached ( RO): true
MTU ( RO): 1500
VLAN ( RO): -1
bond-master-of ( RO):
bond-slave-of ( RO): <not in database>
tunnel-access-PIF-of ( RO):
tunnel-transport-PIF-of ( RO):
management ( RO): true
network-uuid ( RO): [uuid-network]
network-name-label ( RO): Pool-wide network associated with eth0
host-uuid ( RO): [uuid-host]
host-name-label ( RO): [host-name]
IP-configuration-mode ( RO): Static
IP ( RO): [ip-address]
netmask ( RO): [subnetmask]
gateway ( RO): [gateway]
DNS ( RO): [dns-server]
io_read_kbs ( RO): 0.062
io_write_kbs ( RO): 0.082
carrier ( RO): true
vendor-id ( RO): 8086
vendor-name ( RO): Intel Corporation
device-id ( RO): 105e
device-name ( RO): 82571EB Gigabit Ethernet Controller
speed ( RO): 1000 Mbit/s
duplex ( RO): full
disallow-unplug ( RW): false
pci-bus-path ( RO): 0000:01:00.0
other-config (MRW):
Which uses the following additional syntax:
- uuid-nic-eth0: A unique identifier for the first physical NIC in the XenServer.
- uuid-network: A unique identifier for the network.
- mac-eth0: The MAC address of the physical NIC.
- uuid-host: A unique identifier for the XenServer host that contains the physical NIC.
- host-name: Hostname of the XenServer that contains the physical NIC.
- ip-address: IP-address for eth0.
- subnetmask: subnet mask for eth0.
- gateway: gateway for eth0.
- dns-server: DNS server for eth0.
With the additional information for the XenServer name, ip-address and MAC-address of the NIC, you can ensure you have the right uuid for the NIC.
* Remove an interface:
[root@<name-xs> ~]# xe pif-forget uuid=[uuid-nic]
Which uses the following additional syntax:
- uuid-nic: A unique identifier for the NIC.
* Rescan XenServer interfaces:
[root@<name-xs> ~]# xe pif-scan host-uuid=[uuid-host]
Which uses the following additional syntax:
- uuid-host: A unique identifier for the XenServer to be scanned.
Results from the scan are not shown, they can be viewd with the xe pif-list command. Which can be used to check the correct uuids for your NICs.
* Mount interface to your XenServer:
[root@<name-xs> ~]# xe pif-plug uuid=[uuid-nic]
Which uses the following additional syntax:
- uuid-nic: A unique identifier for the NIC.
* Reconfigure the interface:
[root@<name-xs> ~]# xe pif-reconfigure-ip uuid=[uuid-nic] mode=static IP=[ip-address] netmask=[subnetmask] gateway=[gateway] DNS=[dns-server]
Which uses the following additional syntax:
- uuid-nic: A unique identifier for the NIC.
- ip-address: IP-address for NIC.
- subnetmask: subnet mask for NIC.
- gateway: default gateway for NIC.
- dns-server: DNS server for NIC.
* Assign management to the interface:
[root@<name-xs> ~]# xe host-management-reconfigure pif-uuid=[uuid-nic]
Which uses the following additional syntax:
- uuid-nic: A unique identifier for the NIC.
The following sources have been used to create this post:
Citrix Forums – 5.5 to 5.6 Upgrade Fails with NO NICS found
Pingback: VirtuEs.IT
Pingback: virtues it
Pingback: Esther
Pingback: XenServer: management network interface weg na upgrade - webhostingtalk.nl
Nicely done. I’ve made the same mistake in the past of upgrading the slave before upgrading the master server. It’s a bit of a PITA (pain in the ass) but still recoverable.