I have a proxmox (v5.4 I think) installation on top of ZFS in a server with 6 disks. There are 2 pools: rpool in a mirror of two SSDs that has the proxmox root filesystem and some containers and zvols. And the other 4 disks are in another pool (HDD) in raidz-1.
Two months ago I was doing some optimization work in the ZFS parameters and I activated
dnodesize = auto in all pools, including rpool pool and children pools. Now I have rebooted the server and the following message said hello:
error: no such device: 40d7d14f38cc... error: unknown filesystem Entering rescue mode... grub rescue>
With the research I have done I think it is related with the
dnodesize parameter (check here, here and here). Directly quoting from the first link:
Some file must have triggered a non-legacy (512 bytes) dnode size in the dataset, which meant that GRUB could no longer read the drive.
Now I have some ideas on where to start but also a lot of doubts (I’m a not very experimented admin), so I wanted to ask you first, before reaching a dead end and/or break something definitely.
In the first place I could create a live usb with a ZFS capable OS. Probably the best option would be to use the same 5.4 proxmox ISO. Once there, could it be possible to load and manipulate the old proxmox pools from the live USB? How?
The problem is that isn’t enough to set
dnodesize = legacy as changes are not applied retroactively. So I don’t know exactly how to proceed here. The options I have though are the following:
- If there is a way to detect which files are breaking the standard dnode limit of 512 bytes. Then cut/move them to another location and paste/move them back. Problem here, how to detect those files?
- Cut all the rpool content to an external disk and paste them again into rpool. Problem here is that I’m don’t feel safe moving the proxmox root filesystem forward and backward and expect it to work again. Maybe with
- Seems that the only content that needs to be readable by grub is the content inside /boot. But /boot folder is inside the rpool pool, and zfs pools aren’t shrinkable. So I can’t shrink any of the current pools to create a new one for boot, and I can’t add any extra disk to the server. The only option would be to move the /boot to a pen drive and tell grub to search there. But it doesn’t seem very robust or definitive solution.
Thanks for your help.