Qubes OS: Rescue

It finally happened...I jacked up my Qubes OS so bad it was in an unusable state. Through the use of some aggressive udev rules I created situation were my machine would immediately shut down after unlocking the LUKS partition and getting to the login screen...

Apologies in advance for the lack of screenshots. This scenario will assume that you've messed up something with the disposable vm template that generates your sys-usb. This could place the system in to an unusable state. Fortunately, you can still access the entire file system via Qubes usb and hopefully fix what you broke.

Step 1: Create a Qubes installation USB

You've now doubt done this before, but in case you need a refresher:

Installation guide

Welcome to the Qubes OS installation guide! This guide will walk you through the process of installing Qubes. Please read it carefully and thoroughly, as it contains important information for ensuring that your Qubes OS installation is functional and secure. Pre-installation Hardware re…

Qubes OS

Step 2: Boot up your system with the installation USB

On the bootloader screen you will have the option to rescue an existing Qubes installation. Choose this.

Step 3: Choose the option that gives you a shell and unlock your drive

You now have a root shell. From here, you are going to use cryptsetup to unlock the encrypted partition holding the contents of your system.

Use lsblk to list the block devices on your system. Find the device and partition containing your encrypted data and open with cryptsetup. Assuming your LUKS parition was found at /dev/sda3 you would do cryptsetup open /dev/sda3 unlocked and will be prompted for your password to unlock the drive.

Step 4: Mount the file that correlates to the qube you need to make a change to

Start by creating somewhere to mount to: mkdir /mnt/unlocked
cd /dev/qubes_dom0 and find the file that correlates to the qube you want to mount.
In my case I jacked up my DVM that handles sys-usb. In this case you would mount /dev/qubes_0/vm-debian11-dvm-private /mnt/unlocked
Navigate to /mnt/unlocked and you can view the entire filesystem for that Qube. From here, you can use vim to make edits or changes that will hopefully undo whatever you broke. In my case, I was able to rollback the udev rules that got me in to this mess.

Step 5: Unmount, Restart and learn from your mistakes

Like the title says. Hopefully you've fixed what you broke and were able to learn something.

I found a lot of value from https://forum.qubes-os.org/ when troubleshooting this.