Our recent kernel problem

Q&A

What actually happened?

An update to the MX-18.3 default kernel (4.19) was released that lacked the ability to rebuild various significant kernel modules such as nvidia, virtualbox, broadcom, and a few others. We normally push upgrades within a kernel series in order to keep pace with security measures, as we did here. The upgraded kernel was tested before release but the problem did not show up. In general, the problem has been very irregular in users affected and in effects it causes.

How did we respond?

It took us a while to figure out exactly what was happening, since the early reports differed greatly. Our first step was to try to get information to at least some users through announcements on the Forum, Facebook and Twitter. We know those do not reach everybody, of course, but it’s a start. We then took down the kernel update, which will disappear from repos as soon as they are synced up. Once that was done, we began to explore solutions, and have done a great deal of testing.

Are there solutions available for people who were affected?

We have developed and tested three different solutions that are are described in detail further down this page:
1) force the rebuilding of the kernel modules
2) install a different kernel
3) LiveUSB only: launch a recovery command sequence

Is it safe to return to normal update/upgrade??

As soon as the corrupt update is gone from the repo being used, normal update/upgrade can be reinstated. This should be completed in all repos by two days after the date of this post. (EDIT: the update is completely gone now). If you want to test whether it is still there, right-click the MX Updater icon > Check for Updates and look for the presence of linux-image-4.19. If you see it there, then do not upgrade yet.

RECOVERY PROCEDURES

1) Force the rebuilding of the kernel modules

If you can get to a desktop, press F4 to open a terminal, and enter as normal user this command:

sudo /usr/share/mx-packageinstaller-pkglist/rebuild_dkms_packages.sh

If you can’t get to a desktop and end up on a black screen with a blinking cursor, press Alt-F1 to get to a prompt. Log in as your regular user and then run that code.

2) Switch to a different kernel

This is part of 3) for LiveUSB, but may be relevant on an installed MX if procedure 1) did not work for you. If you already have another kernel installed, then reboot and select it by clicking on the “Advanced options” link on the boot screen. If you do not already have one, do one of the following:

  • if you can get to a desktop, press F4 followed by Alt-F1
  • if you can’t, press the letter “e” when the boot screen comes up, then look for the line that begins with “linux” and add a “3” (without the quotes) at the end of it

Once you arrive at a command prompt, become root and enter the command

cli-aptiX

Choose to Ignore updates if you see that screen first. You will be presented with a menu, from which you should select “Search for all kernels.” Choose a new one that suits your system; if you have no idea what to do, the Debian Stretch default 4.9 would be a safe choice.

3) LiveUSB only

Launch a recovery command sequence as laid out in this post: https://forum.mxlinux.org/viewtopic.php?p=510505#p510505.

.

20 thoughts on “Our recent kernel problem”

  1. Nice one, quickly sorted in my view, even if it didn’t affect me, (I don’t have nVidia, Broadcom and don’t use VirtualBox). I’d already updated before I saw the warning message.

  2. I noticed the notice not to install updates when I got onto the Forum a couple of days ago, but even though the text was in red, it was still pretty small and might have been easily overlooked. In the event that something happens where it becomes necessary to publish a notice like this in the future, I’d suggest that you use the color red and use VERY LARGE text so that it can’t be overlooked.

  3. Thank you…i’m running MX 18.3 on a laptop with hdmi out, and intel hd integrated graphics…And it is awesome. Everything works flawless except for Bluetooth playback and output device switching. And if you guys will by default include the video=1024×768 command line in the Grub by default, instead of quiet splash………You can just sit back and watch the fan base grow EXPONENTIALLY.

  4. “If you want to test whether it is still there, right-click the MX Updater icon > Check for Updates and look for the presence of linux-image-4.19. If you see it there, then do not upgrade yet.”

    The problem with this advice is the text scrolls by quickly and then the window closes automatically almost instantly. I can’t read that fast (with confidence).

    • It does scroll by quickly, but when it stops and you can read the last couple of lines to see what updates are available.

  5. Why do you make it so difficult to change the default webpage in Firefox? It defaults to mxlinux.org. You have to go into /opt/firefox/distribution/distribution.ini in order to change the default webpage. This makes me ask a lot of questions.

  6. @Zanaris
    just drag the current link onto the little “house” (Home page button) to the left and done…
    That has everything to do with firefox and nothing with the distribution (here MX Linux)

  7. The instructions above said: “If you want to test whether it is still there, right-click the MX Updater icon > Check for Updates and look for the presence of linux-image-4.19. If you see it there, then do not upgrade yet.”

    When I did this, my system suddenly updated and I didn’t have a chance to look at anything. So I just corrupted my system by CHECKING to see if it was corrupted. Right clicking that icon resulted in immediate automatic update, zoom, done!

    Could you post clear, adequate instructions on how to now un-corrupt the kernel on my system? You say “rebuild the kernel”, and I don’t know if this means trashing every setting on my system, including the hard-won firewall settings. Sorry but many of us users don’t know if ‘rebuilding the kernel’ is going to wipe the system clean and leave us dead in the water. For whatever reason the instructions on CHECKING that icon by right clicking it just put that thing straight into my version of MX Linux. Perhaps there is an automatic setting there I don’t know about. Please list clear instructions with warnings regarding un-corrupting all the systems which got corrupted by loading the corrupt kernel in this fashion. By the way, other than this fiasco, MX Linux is about as good as it gets – it’s truly a magnificent product. Anyone contributing to this, well, as a team you’re just awesome. But now, please give us users out here the simple instructions – with warnings – how to fix our systems and ‘rebuilding the kernel’ doesn’t sound very attractive for someone who just got infected with a corrupt kernel. Is this safe? (remember Marathon Man????) Sounds like a very dangerous thing like driving off a cliff…

    • I don’t know how you did that, since the instructions are correct and well tested and what you report has never happened to any other user that we know of. Do you in fact have that kernel, and has any effect shown up?

      If you are concerned, follow the directions for installing and using a different kernel. If you update the kernel that is all that will be changed. If doing that threatened any other settings, we would have stated that.

      Thanks.

  8. Pues yo tengo este KERNEL:

    Kernel: 5.1.2-antix.1-amd64-smp x86_64 bits: 64 compiler: gcc
    v: 6.3.0 Desktop: Gnome 3.22.3 Distro: MX-18.3_x64 Continuum May 26 2019
    base: Debian GNU/Linux 9 (stretch)

    Alguno tiene problemas con éste kernel?
    En mi caso funciona bien.
    Rápido y limpio.

    Saludos

  9. Is this issue fixed? The post says, “As soon as the corrupt update is gone from the repo being used, normal update/upgrade can be reinstated. This should be completed in all repos by two days after the date of this post.” but I don’t see a date for the post.

  10. Why not put dates on the post and have date/times with the comments too?

    It’s hard to know if this issue is relevant for me at all with new newish install.

    I am having problems with audio, but I don’t think this relates to the post, does it?
    – my audio is very subject to interference which presents as static.

  11. I’ve not been able to download the OS. Do I need bit torrent?
    I think I have several downloads 386 and 64 bit but can’t do anything with them. Any help out there for a newbie?’

  12. Installing a different kernal is not accurate (as of this date of post) – there is no listing of a 4.9 kernal as suggested in the list that is presented (mentioning “Debian Stretch default 4.9”) – could someone update these instructions so I don’t break my system?
    I am interested in fixing all kinds of recent issues, some with Virtual Box and others with my accounting program “Manager” which was working well a few months ago – and now freezes and crashes often (never happened earlier – ever) so I hope once I update the kernal – it could be back to normal?

Leave a Comment

Do NOT follow this link or you will be banned from the site!