Systemd 256 may break sleep

Looks like an issue with the recent update to systemd 256 is that it may break sleep (and do so particularly bad on some hosts). See Since 256, suspend fails over 50% of the time: Freezing user space processes failed · Issue #33626 · systemd/systemd · GitHub for details. In a nutshell, this systemd version has introduced freezing of user sessions when entering the various sleep modes. According to the linked issue, this does not work well with the current kernels. Furthermore, according to user session fails to resume from suspend when user is using NFS or KVM · Issue #33083 · systemd/systemd · GitHub the potential advantages for systems without encrypted homes are modest.

Because not sleeping properly is not just an annoyance, but may cause data loss and even hardware damage (think of a laptop going full power while closed in a neoprene bag and reaching extreme temperatures), I would like to suggest that the freezing of user sessions practiced by systemd is at least temporarily disabled in manjaro.

This can be done by setting the SYSTEMD_SLEEP_FREEZE_USER_SESSIONS=false environment variable. It can be done by creating a drop-in conf for systemd-suspend in /etc/systemd/system/systemd-suspend.service.d/disable_freeze_user_session.conf containing

[Service]
Environment="SYSTEMD_SLEEP_FREEZE_USER_SESSIONS=false"
4 Likes

According to your linked topic - it is for linux6.7 and the other 6.10 rc kernel.

As I see that - it is not relevant for Manjaro users - Manjaro is long past that - currently 6.10.8 on unstable branch and 6.10.6 on stable.

systemd with Manjaro is at 256.5 on all branches

Yet just another reason to keep your system and kernel up-to-date.

3 Likes

I predict many potentially losing sleep over systemd.

1 Like

Actually, I posted because I am experiencing the issue in an up to date manjaro with latest stable kernel. Seems to happen on arch too, that is even more up to date, e.g. [SOLVED] Using Linux QEMU/KVM causes s2idle hard freeze on Arch - Linux 6.10.8 - #8 by David_Markey - Linux - Framework Community. The proposed solution there is the one I am indicating.

I have seen more of these lately - but the issues is not a generic issue - it seems to be very specific circumstances.

Linux 6.10.8 was build earlier today for unstable branch.

I have had issues with my network not waking up on a Thinkpad - but nothing a restart of NetworkManager couldn’t fix. And certainly no dataloss.

Also I mentioned it to @philm which had something similar - but he claimed it was fixed the current kernel versions. I also mentioned that some blame systemd 256 - I haven’t heard back on that one.

There is only a few incidents - 2 or 3 I believe - but they have no dataloss only weird behavior.

So I still think it is not a cause for alarm - I mean there is no reason to cause a stir with something which may be a storm in glas of water.

Manjaro does not provide kernels from Arch - they are maintained in-house - so is systemd for what I recall. (reference to the support message pointing to the forum in systemd journal)

The issue is very easy to detect:

  • When you try to put the laptop to sleep it freezes; or
  • When you try to hibernate the laptop it freezes; or, more subtle,
  • If you have the sleep-then-hibernate systemd feature active, when the laptop wakes up from sleep to estimate power or to hibernate, it cannot complete the action and it freezes.

The last possibility is the worst one, since it can easily result in a laptop overheating while sealed in a bag. For the rest, most filesystems should by today be robust enough to resist unclean shutdowns and people should be wise enough to save work before closing the laptop lid.

The issue does not happen all the time, it is occasional. The systemd issue says ‘50% of the time’, but this estimation is very inaccurate. In my own experience, the issue happens way more frequently on machines that are slow (weak CPU, SD based storage, etc). It is know to happen very frequently if you use KVM or the io_uring.

The issue is not actually in systemd, but in the kernel. To the best of my understanding it affects all current kernels. A fix may appear in 6.11 and then be progressively backported to other kernel versions. At present time, it is unknown if the kernel fix that is being adopted will completely clear the issue. It may be the case that the problem is triggered in more places, so that other fixes are needed.

The systemd developers are well aware of the issue. They had some internal discussion and they decided to leave it to the downstream distributions to disable the “user session freezing” feature that is problematic. They prefer to leave the feature enabled by default upstream, in order to speed up the discovery of related issues in the kernel (“We had some discussions about this, but overall the conclusion is leaving this enabled by default upstream is more favorable, which might drive kernel to eventually fixing this. But distros are always free to patch this out”). In fact, it is not a patch. It is just setting an environment variable.

So IMHO, Manjaro could either:

  • Temporarily deliver the systemd configuration drop in files setting the enviroment variable, until all the supported Manjaro kernels are known to be completely fixed;
  • Introduce this item in the “additional info” section that appears in the stable announcements, with instructions on how to create the drop-in systemd configuration files for users who encounter problems;
  • … or obviously do nothing, since the information is anyway present in the systemd bug tracker (though not totally easy to find).

Personally, I would pick at least the 2nd option.

2 Likes

From what I can gather around this issue - and possibly the reason I cannot reproduce it - it seems to be related to systems with Nvidia GPU.

https://bbs.archlinux.org/viewtopic.php?pid=2180042#p2180042

So I think it would be a good thing to add Nvidia to the tag list - perhaps mention in the OT that this is generally an issue for Nvidia users.

Or am I off track here?

Testing → [Testing Update] 2024-09-07 - Plasma 6.1.4, KDE Gear 24.08, Mauikit 4.0, Mesa 24.2.2 - #2 by philm

Stable → [Stable Update] 2024-09-02 - Kernels, Pipewire, Systemd, LibreOffice, OGUI, Inputplumber, Firefox - #2 by philm

Done. :wink:

It is not NVIDIA only, though. NVIDIA is just one of those factors that make the issue more frequent. Personally, I am experiencing the issue with Intel and AMD graphics. The machine I have on which the issue is more frequent at all is Chuwi two-in-one with a weak CPU (N4120), Intel graphics and EMMC/SD-card storage.

There is a piece of bad news, as one use is reporting that the issue is still present with the latest 6.11RC that should have a fix. Apparently, as some systemd developer suggested, there may be more pieces to fix before the freezing of user sessions is working with no hiccup.

Thank you for keeping us informed.

In that case the workaround you brought to attention is worth the effort.

Please, take the following cautiously. The first reports seem to indicate that the kernel fix proposed so far might not be sufficient to completely resolve the issue (https://github.com/systemd/systemd/issues/33626#issuecomment-2365265129). There might be more places in need of some patch. So, for the time being, the workaround above appears to remain useful.

Cross posting from the stable update, but was hoping maybe I can get some insights here

It seems like suspend is completely broken for me now after the recent stable update. It used to sometimes fail on me before, not waking up from sleep because it couldn’t detect the EDID settings of my monitor according to the logs, but now it’s completely unusable. I am using proprietary nvidia drivers, and I was on kernel 6.9, before moving on to 6.10. In an attempt to fix it, I am now on 6.11, thinking a newer kernel might’ve fixed the issue. However the issue still occurs.

I can’t really tell if this is the same issue as the one mentioned in the post, because I don’t get Failed to freeze unit 'user.slice' in the logs. Also, I get the following message towards the end:

User sessions remain unfrozen on explicit request ($SYSTEMD_SLEEP_FREEZE_USER_SESSIONS=0).
This is not recommended, and might result in unexpected behavior, particularly
in suspend-then-hibernate operations or setups with encrypted home directories.

which indicates to me that SYSTEMD_SLEEP_FREEZE_USER_SESSIONS is actually disabled. Should I still apply the fix mentioned in this thread?

If your system is not freezing user session before sleep, your issue is probably different. Setting config to ask your system not to do something that it is not doing anyway should bring no harm but no benefit either.

1 Like

To the best of my understanding there are issues with linux 6.11.2 with hibernation, at least on AMD machines. I certainly see them on an ASUS ROG14 2022. This is a different issue from that triggered by systemd 256 with the default freezing of user sessions.

Linux 6.11.2 is also the current stable kernel on manjaro. Those experiencing the issue can transiently use 6.10.13 that is available in Manjaro and should not have the issue.

1 Like