> For the complete documentation index, see [llms.txt](https://docs.vergeos-demo.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.vergeos-demo.com/knowledge-base/backup-dr/rhel9-boot-failure-clone-snapshot-restore.md).

# RHEL 9 Family VMs Failing to Boot After Clone or Snapshot Restore

This guide covers a specific boot failure: a RHEL 9 family VM (RHEL, AlmaLinux 9, Rocky Linux 9, or CentOS Stream 9) that boots fine on the source, then fails on cloned or restored snapshot instance because LVM's system.devices file no longer matches the VM's disk serial numbers. The steps below repair the VM from rescue mode. if you haven't performed the clone or restore yet, see the "*Preventing this on future clones, snapshots, or exports*" note further down.

## Symptoms

* After restoring a VM from a snapshot, or cloning a VM, the RHEL 9 VM fails to boot (drops to emergency mode or hangs at boot)
* In rescue mode, vgchange -ay produces no output and lvdisplay shows nothing

## Overview

RHEL 9 and its derivatives (AlmaLinux 9, Rocky Linux 9, CentOS Stream 9) use an LVM devices file at `/etc/lvm/devices/system.devices` that explicitly whitelists block devices by their unique identifiers — most notably disk serial numbers. At boot, LVM only considers devices listed in this file when assembling volume groups. If the serial numbers it sees don't match the ones recorded when the file was created, LVM won't activate the root volume group and the VM drops to a dracut emergency shell.

This means any operation that gives a VM new disk serial numbers — cloning a VM with new hardware serial numbers generated, or restoring a snapshot in a way that changes serial numbers — will cause the boot process to fail.

{% hint style="info" %}
**VergeOS 26.1.5 and Later**

Starting in VergeOS 26.1.5, VM clones and snapshot restores **preserve the original drive serial numbers by default**, eliminating the serial number mismatch for most clone and restore operations. This issue most commonly affects:

* Clones or restores where the option to generate new hardware serial numbers was explicitly selected
* Environments running VergeOS versions earlier than 26.1.5
  {% endhint %}

{% hint style="info" %}
**Imports from another hypervisor**

The same boot failure can occur when importing a VM from another hypervisor *if* the VM was exported without preserving its original device serial numbers. Additionally, Imported VMs may fail to boot due to missing virtio drivers for VergeOS hardware. See [Importing a Linux VM Guide](/knowledge-base/virtual-machines/import-rhel-centos-vm.md) for instructions that include adjusting VM configuration and regenerating the initramfs with virtio drivers.
{% endhint %}

{% hint style="info" %}
**Preventing this on future clones, snapshots, or exports**

If you still have access to a working source VM, you can avoid the boot failure on future clones, snapshots, or exports from another hypervisor by disabling the LVM devices file *before* the operation. Set `use_devicesfile = 0` in `/etc/lvm/lvm.conf` on the guest. LVM will fall back to scanning all available block devices regardless of serial number. The LVM devices file was designed primarily for hosts attached to a SAN fabric, where a single machine can see many block devices that don't belong to it; in a typical VergeOS environment a VM only sees the virtual disks explicitly assigned to it, so the protection it provides is rarely needed. See Red Hat's [Limiting LVM device visibility and usage](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_logical_volumes/limiting-lvm-device-visibility-and-usage_configuring-and-managing-logical-volumes) for more on the devices file.
{% endhint %}

## Prerequisites

* Access to the VergeOS UI and VM console
* A RHEL 9 family installation ISO available in VergeOS media (required only if the VM cannot reach the GRUB menu)

## Steps

### 1. Boot into a Rescue Environment

The VM will likely fail to boot normally. Use one of the following methods to reach a working shell.

#### Option A — GRUB Rescue Kernel (preferred)

* Power on the VM and open the console in VergeOS.
* During boot, press and hold **Shift** (BIOS) or **Esc** (UEFI) to access the GRUB menu.
* Select **Advanced options** and choose the **rescue** kernel entry.
* Follow any prompts — if asked whether to mount the root filesystem, select **Continue**.

#### Option B — Boot from Installation ISO

If the GRUB menu is unreachable or the VM won't get that far:

* In VergeOS, attach a RHEL 9 family installation ISO to the VM as a CD/DVD drive.
* Adjust the VM's boot order to boot from the ISO first, or use a one-time boot option.
* At the installer menu, navigate to **Troubleshooting → Rescue a \[distro] system**.
* When prompted about mounting the installed system, choose option **1** (mount under `/mnt/sysimage`).
* Select **Shell** to open a command prompt.

{% hint style="warning" %}
**ISO rescue uses a different mount path**

When using installer rescue mode, your installed system is at `/mnt/sysimage` rather than `/mnt`. Adjust next steps accordingly.
{% endhint %}

### 2. Confirm the disk is visible, rescan, and verify volume groups

* **Display Block devices:**

  `lsblk`

  Note the device name (e.g., /dev/sda) and the LVM partition (e.g., /dev/sda3)
* **Force LVM to re-scan all block devices (bypasses the devices file):**

  `pvscan --cache`\
  `vgchange -ay`
* **If rescan still shows nothing, register the LVM partition explicitly:**

  `lvmdevices --adddev /dev/sda3` Replace with your actual LVM partition from lsblk\
  `vgchange -ay`

  <div data-gb-custom-block data-tag="hint" data-style="success" class="hint hint-success"><p><strong>RHEL 9 installations typically use LVM, with the root logical volume at <code>/dev/mapper/rhel-root</code> (or <code>almalinux-root</code>, <code>rocky-root</code>).</strong></p></div>
* Verify volume groups are now visible

  `lvdisplay`

### 3. Mount the root partition and bind the filesystems

* `mount /dev/<device_name> /mnt`\
  Replace \<device\_name> with your root partition (e.g., sda2, mapper/vg0-root).
* **Verify the mount:**

  `ls /mnt`

  You should see directories like /root, /boot, /home, /etc, and /var.
* **Bind the Filesystems:**

  **Use the following for-loop to bind the necessary virtual filesystems:**

  `for i in proc sys dev run; do mount --rbind /$i /mnt/$i; done`

  **OR mount them individually:**

  `mount --rbind /proc /mnt/proc`\
  `mount --rbind /sys /mnt/sys`\
  `mount --rbind /dev /mnt/dev`\
  `mount --rbind /run /mnt/run`

  For EFI/UEFI systems, also mount the EFI partition:

  ```bash
  mount /dev/sdX1 /mnt/boot/efi
  ```

  Replace /dev/sdX1 with your actual EFI partition (check with lsblk or fdisk -l).

### 4. Chroot into the Installed System and mount additional filesystems

* **Change the root directory**

  `chroot /mnt`
* After chrooting, **mount any additional partitions** defined in fstab:

  `mount -a`

### 5. Regenerate the Initramfs

* **Rebuild the initramfs** so it picks up the cleaned-up device configuration:

  `dracut -f --regenerate-all`

### 6. Reboot and Verify

* **Exit the chroot environment:**

  `exit`
* **Reboot the VM:**

  `reboot`
* \*\*Verify Boot:\*\*Confirm that the VM boots successfully, and all drives appear as expected.

## Troubleshooting

{% hint style="warning" %}
**Can't reach GRUB menu**

If the VM doesn't display GRUB at all, use Option B from Step 1 to boot from an installation ISO into rescue mode.
{% endhint %}

{% hint style="warning" %}
**No network after boot**

* Confirm the NIC is set to **virtio** in the VergeOS VM settings.
* Check that no persistent network rules (`70-persistent-net.rules`) exist that would block the new interface from being named correctly.
* Run `ip link show` to see whether a NIC is present but not configured, then check `/etc/sysconfig/network-scripts/` or NetworkManager for the interface configuration.
  {% endhint %}

## Additional Resources

* [Limiting LVM device visibility and usage (Red Hat)](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_logical_volumes/limiting-lvm-device-visibility-and-usage_configuring-and-managing-logical-volumes)
* [Configuring and managing logical volumes (Red Hat)](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_logical_volumes/index)
* [Importing Linux VMs — General Guide](/knowledge-base/virtual-machines/import-rhel-centos-vm.md)
* [VM Best Practices](/run-the-platform/virtual-machines/vm-best-practices.md)

{% hint style="info" %}
**Need Help?**

If you continue to experience boot issues after following this guide, contact VergeOS support for assistance.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.vergeos-demo.com/knowledge-base/backup-dr/rhel9-boot-failure-clone-snapshot-restore.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.