Upgrading from SLES15 SP7 to SLES 16.0 failed!

Please share a copy of /var/log/distro_migration.log

@n-michalko have you seen https://forums.suse.com/t/upgrade-from-sles-15-sp7-to-16-doesnt-work/ ?

Yes, of course - but it did not help me ….

It looks to me that the SUSE developers took a clean server and didn’t test it once on a server with a “history”. With “history” I mean, that the server has been upgraded from SLES 12 step by step, so it got some old data because of previous upgrade-processes which seem to be “unclean”.

Also, the documentation how to upgrade is simply wrong. I mean the SSH part. Without the help of @aalzayed I would be stuck.

I wrote my own step-by-step guide which I will publish within the next couple of days.

2 Likes

There is no indicator why system canceled the migration process. If you can share a supportconfig, this would be good.

1 Like

Thanks for sharing supportconfig.

I think migration is failing because /var is on separate disk:

# /bin/df -Th
Dateisystem    Typ      Gr?sse Benutzt Verf. Verw% Eingeh?ngt auf
/dev/sda1      xfs         40G    2.7G   38G    7% /
/dev/sda3      xfs         40G     13G   28G   31% /usr
...
/dev/sda7      xfs         71G    2.3G   69G    4% /home
/dev/sda8      xfs         40G    887M   40G    3% /tmp
/dev/sda6      xfs         40G    4.6G   36G   12% /opt
/dev/sda5      xfs         40G    2.0G   39G    5% /var

From: https://documentation.suse.com/sles-sap/16.0/html/SAP-upgrading/index.html#task-sle16-upgrade-requirements :

The operating system needs to reside on a single partition. Support for multiple partitions, such as LVM, is limited to configurations where the separated partitions do not contain OS-critical data or processes. For example, a separate /home partition will function whereas a separate /var will not.

Let us try again, please try again the migration and once it failed share:

cat /etc/issue

Also, you can add:

debug: true
verbose_migration: true

to: /etc/sle-migration-service.yml file and then trigger the migration. Let me know the result.

1 Like

I created the /etc/sle-migration-service.yml file (it didn’t exist) with the proposed content, also added “PermitRootLogin yes” to /etc/ssh/sshd_config.d/50-permit-root-login.conf and started migration again. After reboot I can not login into the server with ssh though - now I must wait until friday to log in locally on the server …

Thanks!

This is what I meant that there is an error in the documentation regarding SSH. It doens’t work but I was a lucky guy because I had from another VM an existing connection, so I could log in and install the package which @aalzayed suggested in my thread.

The correct way is to check before you run the migration if you can log in via ssh root@hostname to the server which is going to be migrated.

1 Like

Here it is: https://documentation.suse.com/sles/16.0/html/SLE-differences-faq/index.html#sle16-differences-faq-security-root

however passwored-less ssh login should still works. But I’m with you, the upgrade doc should mention it. By the way, suse-migration-pre-checks command print a hint :wink:

Is it pingable? Have you tried: ssh migration@IP_OF_INSTANCE

No, it is not pingable …

And yes, i have tried to connect with “ssh migration@…” - No route to host

:unamused_face:

If it is not reachable, then you need to check the console.

Of course, but today it is not possible - tomorrow!

Thanks!

So, after reboot I ‘ve tried (as suggested in above link) :

zypper in openssh-server-config-rootlogin

“Paket ‘openssh-server-config-rootlogin’ nicht gefunden.”

I mean the upgrade doc should be definitely corrected!

Server ist still on SLE15 SP7 and I can log in as root with ssh. Here is the last /var/log/distro_migration.log:

“Running Pre-migration btrfs snapshot creation
Running ssh keys service
Calling: [‘chroot’, ‘/system-root’, ‘snapper’, ‘–no-dbus’, ‘get-config’]
Getting keys from /system-root/root/.ssh/authorized_keys
Save keys to /home/migration/.ssh/authorized_keys
Copying host ssh keys
Restarting sshd
Calling: [‘systemctl’, ‘restart’, ‘sshd’]
Running post mount actions
Copy file: /system-root/etc/udev/rules.d/70-persistent-net.rules to: /etc/udev/rules.d
Copy file: /system-root/etc/sysctl.conf to: /etc
Copy file: /system-root/etc/sysctl.d/70-yast.conf to: /etc/sysctl.d
Calling: [‘udevadm’, ‘control’, ‘–reload’]
Calling: [‘udevadm’, ‘trigger’, ‘–type=subsystems’, ‘–action=add’]
Calling: [‘udevadm’, ‘trigger’, ‘–type=devices’, ‘–action=add’]
Running suse-migration-pre-checks with options: fix: True
Using migration_system mode
Checking harmful migration conditions
→ Checking system architecture version…
→ Checking for local private repos…
Done
→ Checking for encrypted rootfs…
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/4cc6ed31-d861-42c1-80e2-4fffec2ffe2c’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/a5a89a78-e8bc-445c-b6a3-1e1f51a7e38e’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/9aad7fb4-5cbe-4c6e-8b45-4b7eb4f00bbf’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/695b5516-9d51-416b-a438-888e0eef3b3d’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/ace03825-f7d2-458a-bc2f-a743dd56eaed’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/eb1121a8-ecce-44fb-85ef-bb1a7f196bc8’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/490F-236C’]
Calling: [‘sysctl’, ‘–system’]
Calling: [‘blkid’, ‘-s’, ‘TYPE’, ‘-o’, ‘value’, ‘/dev/disk/by-uuid/910ef6db-e9f6-4794-b34d-0a7120877ab8’]
Done
→ Checking for latest kernel in multiversion kernel system…
The config option ‘multiversion’ in /etc/zypp/zypp.conf includes the keyword ‘kernel.’ The current value is set as
‘multiversion = provides:multiversion(kernel)’.
Checking the config option ‘multiversion.kernels’ to see if multiple kernels are also enabled
Calling: [‘chroot’, ‘/system-root’, ‘rpm’, ‘-qa’, ‘kernel-default’]
Running setup host network service
Calling: [‘mount’, ‘–bind’, ‘/system-root/etc/sysconfig/network/providers’, ‘/etc/sysconfig/network/providers’]
Calling: [‘systemctl’, ‘reload’, ‘network’]
Running wicked2nm host network migration
Calling: [‘rpm’, ‘–query’, ‘–quiet’, ‘wicked2nm’]
Calling: [‘rpm’, ‘–query’, ‘–quiet’, ‘NetworkManager-config-server’]
Calling: [‘wicked2nm’, ‘migrate’, ‘–netconfig-base-dir’, ‘/system-root/etc/sysconfig/network’, ‘–activate-connections’, ‘/system-root/var/cache/wicked_config/config.xml’]
Calling: [‘nm-online’, ‘-q’]
Running setup resolver service
Copying /system-root/etc/resolv.conf
Running prepare for migration
Running grub setup service
Calling: [‘chroot’, ‘/system-root’, ‘zypper’, ‘–non-interactive’, ‘–no-gpg-checks’, ‘remove’, ‘SLEMigration’, 'suse-migration--activation’]
Done
→ Checking LSM migration…
Skipped LSM migration checks.
Done
→ Checking upgrade path against registration server…
Done
→ Checking high availability extension…
corosync.conf not found. Skipped checks for high availablity extension.
Done
→ Checking wicked to NetworkManager migration…
Done
→ Checking sshd configuration…
Done
Uninstalling migration:
Reading installed packages…
Resolving package dependencies…

The following 2 packages are going to be REMOVED:

SLES16-Migration suse-migration-sle16-activation

2 packages to remove.

Package install size change:
| 0 B required by packages that will be installed
-319.2 MiB | - 319.2 MiB released by packages that will be removed

Backend: classic_rpmtrans
Continue? [y/n/v/…? shows all options] (y): y
(1/2) Removing: suse-migration-sle16-activation-2.1.30-150700.15.13.1.x86_64 [..
warning: waiting for exclusive lock on /usr/lib/sysimage/rpm/Packages
Generating grub configuration file …
Found theme: /boot/grub2/themes/SLE/theme.txt
Found linux image: /boot/vmlinuz-6.4.0-150700.53.31-default
Found initrd image: /boot/initrd-6.4.0-150700.53.31-default
Adding boot menu entry for UEFI Firmware Settings …
done
.done]
(2/2) Removing: SLES16-Migration-2.1.30-15.26.4.x86_64 […done]

Calling: [‘chroot’, ‘/system-root’, ‘grub2-mkconfig’, ‘-o’, ‘/boot/grub2/grub.cfg’]
Creating new grub menu:
Generating grub configuration file …
Found theme: /boot/grub2/themes/SLE/theme.txt
Found linux image: /boot/vmlinuz-6.4.0-150700.53.31-default
Found initrd image: /boot/initrd-6.4.0-150700.53.31-default
Adding boot menu entry for UEFI Firmware Settings …
done

Running update bootloader service
Installing the shim package
Calling: [‘bash’, ‘-c’, ‘zypper --no-cd --non-interactive --gpg-auto-import-keys --root /system-root install --auto-agree-with-licenses --allow-vendor-change --download in-advance --replacefiles --allow-downgrade shim &>> /system-root/var/log/distro_migration.log’]
Running initrd creation service
No action needed, dracut was called during installation
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
Refreshing service ‘Basesystem_Module_15_SP7_x86_64’.
Refreshing service ‘Certifications_Module_15_SP7_x86_64’.
Refreshing service ‘Desktop_Applications_Module_15_SP7_x86_64’.
Refreshing service ‘Development_Tools_Module_15_SP7_x86_64’.
Refreshing service ‘Legacy_Module_15_SP7_x86_64’.
Refreshing service ‘Python_3_Module_15_SP7_x86_64’.
Refreshing service ‘SUSE_Linux_Enterprise_Server_15_SP7_x86_64’.
Refreshing service ‘SUSE_Package_Hub_15_SP7_x86_64’.
Refreshing service ‘Server_Applications_Module_15_SP7_x86_64’.
Refreshing service ‘Web_and_Scripting_Module_15_SP7_x86_64’.
Loading repository data…
Reading installed packages…
warning: Found bdb_ro Packages database while attempting ndb backend: using bdb_ro backend.
‘shim’ is already installed.
No update candidate for ‘shim-16.1-150300.4.31.3.x86_64’. The highest available version is already installed.
Resolving package dependencies…
Nothing to do.
Updating the shimbootloader
Calling: [‘chroot’, ‘/system-root’, ‘shim-install’, ‘–removable’]

Updating the bootloader
Calling: [‘chroot’, ‘/system-root’, ‘/sbin/update-bootloader’, ‘–reinit’]
Running reboot service
The migration file ‘/etc/migration-config.yml’ has been updated with ‘/system-root/etc/sle-migration-service.yml’ info

Config file content:
debug: true
preserve:
rules:

  • /etc/systemd/network/*.link
  • /etc/udev/rules.d/70-persistent-net.rules
    sysctl:
  • /etc/sysctl.conf
  • /etc/sysctl.d/*.conf
    soft_reboot: false
    verbose_migration: true

Calling: [‘systemctl’, ‘stop’, ‘suse-migration-console-log’]
Reboot skipped due to debug flag set

And here is the output of cat /etc/issue

Welcome to SUSE Linux Enterprise Server 15 SP7 (x86_64) - Kernel \r (\l).

eth0: \4{eth0} \6{eth0}
eth1: \4{eth1} \6{eth1}

I’m not sure why the migration is not working.

But can you give it another try? remove: kernel-default-devel package

rpm -qa | grep kernel-default , on my system show only kerne-default.

Let me know the result.

If this didn’t help. Please remove the log file and then: add the file /etc/sle-migration-service.yml and add

debug: true

With the debug switch set the system will not reboot in case of an error and you can check what was wrong inside of the live migration system by ssh via migration user. Check the failed services: systemctl –failed

let me know the result.