Crash Recovery

Strato Root Server

Login to Remote-Console first, so you can see boot messages, … and kernel panics, if unlucky

Re-Install takes 1h

Recovery Console:

Set Boot mode to recovery in “RecoveryManager” at https://config.stratoserver.net/ Wait for 15-20min!! until “ServerData” Serverstatus shows: Installation finished

Strato sets RAIDs to /md/126 and md127 - that sucks! - via grub install, …,.. you can mess up the system

log in to recovery via

ssh root@IP   # recovery root passwd is shown in "Serverdata"

mount /dev/md127 /mnt/    # system partition
mount /dev/md126 /mnt/boot/    # boot partition

to re-install grub, you need to chroot

mount -o bind /dev /mnt/dev
mount -t proc none  /mnt/proc/
mount -t sysfs none /mnt/sys
chroot /mnt
update-grub

( grub-mkdevicemap  #  In case of error “no such disc” )
grub-install /dev/sda
grub-install /dev/sdb

To exit the recovery console you need to set boot mode to normal and wait 10min until “ServerData” Serverstatus shows: Installation finished

SoYouStart / OVH Root Server

You need to setup your ssh key in the webmanager to enter in rescue mode without password! the password will be sent by mail as well.

- Change netboot mode in web-manager: set to “rescue” - Click “reboot”

Mount partitions:

mkdir /mnt/md2
mount /dev/md2 /mnt/md2/

After restore, click “netboot” in webmanager again and reset it to “harddisc”

Full Disc Encrypted System Rescue

If the system does not boot anymore, you can open the encrypted device anyway with a live system on USB stick or CD.

# find the correct partitions:
fdisk -l 

# open the container:
cryptsetup luksOpen /dev/sdXY sdXY_crypt

# activate the LVM volume group:
vgchange -ay

# find new volume groups in /dev/mapper/ and mount them (maybe it's just root or home, var, ...might be in, too):
mount /dev/mapper/ubuntu--vg-root /mnt

# mount the unencrypted boot parti:
mount /dev/sdXZ /mnt/boot

# bind mount some important dirs:
mount -t proc proc /mnt/proc
mount -o bind /dev /mnt/dev
mount -t sysfs none /mnt/sys

Now you can take your data back already.

To repair what is broken, you probably need to change root into that system:

chroot /mnt

All following commands are executed inside the broken system. Some hints for repairs:

apt install lvm2     # debian uninstalled lvm during an upgrade :(

If your root system is BTRFS, you need btrfs-progs and the kernel module during the early initramfs boot stage:

apt install btrfs-progs

Add a missing module to initramfs - inside the chroot:

mcedit /etc/initramfs-tools/modules

Write one module name like “btrfs” per line.

Then regenerate the initramfs for all installed kernels:

update-initramfs -u -k all

Also check the UUID entries in /etc/crypttab compared to ls -la /dev/disk/by-uuid/

TbC.. There may be various other reasons for bootproblems…

 
Back to top
linux/crash_recovery.txt · Last modified: 2018/10/08 11:09 by tkilla