Sudden crash: almost freeze and force me to log out

Support
1

While using the computer, the mouse and the keyboard (both built-in and external) were almost frozen, and force me to log out. Though I am running a Python program, 32 GB RAM seems enough. I am not sure how to check the sys log to see how this happened, as there just be too much info even for journalctl --since "10 min ago". From what I saw, it says something about out of memory, but it is about the chorme, even though I did not open too many tabs (less than 10). Any idea what happened here? Thank you so much!

Some parameters I found that maybe helpful for debugging here:
sudo grep -R . /sys/module/zswap/parameters:

/sys/module/zswap/parameters/same_filled_pages_enabled:Y
/sys/module/zswap/parameters/enabled:Y
/sys/module/zswap/parameters/max_pool_percent:20
/sys/module/zswap/parameters/compressor:zstd
/sys/module/zswap/parameters/zpool:z3fold
/sys/module/zswap/parameters/accept_threshold_percent:90

sudo dmesg | grep zswap: nothing

sudo grep -R . /sys/kernel/debug/zswap:

/sys/kernel/debug/zswap/same_filled_pages:0
/sys/kernel/debug/zswap/stored_pages:0
/sys/kernel/debug/zswap/pool_total_size:0
/sys/kernel/debug/zswap/duplicate_entry:0
/sys/kernel/debug/zswap/written_back_pages:0
/sys/kernel/debug/zswap/reject_compress_poor:0
/sys/kernel/debug/zswap/reject_kmemcache_fail:0
/sys/kernel/debug/zswap/reject_alloc_fail:0
/sys/kernel/debug/zswap/reject_reclaim_fail:0
/sys/kernel/debug/zswap/pool_limit_hit:0

watch -n 0.1 LANG=C free -h again for what exactly I did just before the crash, which is reasonably small.

               total        used        free      shared  buff/cache   available
Mem:            31Gi        13Gi       5.4Gi       1.4Gi        12Gi        15Gi
Swap:             0B          0B          0B

It seems that what is “free” is much smaller than what is “available”. Any suggestions? Thank you!

2

Any help here? Thank you!

3

It is not clear, what kind of program.

Without information, nobody can say anything, even it is overwhelming: better more than nothing. “A Python program” is not enough.

That looks all good and since you have no swapfile, it is quite normal. zswap is used in combination with a swap device. So it holds some swap pages compressed in the RAM before it gets swapped to the disk, to reduce disk activity. It is a cache.

If you want no disk activity and have enough ram (32GB+), then use a ram disk as swap device. Have look here for zram: Improving performance - ArchWiki

4

Thank you @megavolt for your reply! The python program that I was running, at least around the crash, was heavily used in CPU.

And honestly I am still a newbie and am learning anything about the system. Just to clarify, it seems that cache occupied some places, so I wonder do you mean to swap whatever in the cache now to disk would be a solution if “out of memory”? Thank you!

              total        used        free      shared  buff/cache   available
Mem:            31Gi       3.1Gi        15Gi       1.1Gi        11Gi        26Gi
5

ok let me explain it very simple:

  1. About the swap: If you have swap device (a file or parition), then it will swap out pages to the disk. In combination with zswap, it will hold the swapped pages as long as possible at the memory compressed, then it decompress it and swap it to the disk.

As you see:

Nothing has been swapped to zswapm because there is no swap device:

Read this here: zswap — The Linux Kernel documentation

  1. Buffer/Cache is another case. It contains cache which is not needed for applications, but improves performance by caching disk activity to the ram. It can be deleted anytime without any malfunction. In fact, it does it all the time.

If your python program run in 32bit, then keep in mind that can only address 4GB of your ram. So out of memory could be also addressed to this issue.

So the solution could be installing systemd-swap which dynamically create swap files when the system thinks it needs it.

  1. Install it:
pamac install systemd-swap
  1. Edit the config:
sudo nano /etc/systemd/swap.conf
  1. Enable at least 2 variables and save it:
zswap_enabled=1
swapfc_enabled=1
  1. Activate it:
sudo systemctl enable --now systemd-swap.service
  1. Check if it is running:
sudo systemctl status systemd-swap.service

It runs immediately, no reboot needed.

6

Thanks for you detailed explanations and documentation! I followed the steps and double checked after a reboot, but it seems that it still has 0B for swap. Do I have any misunderstanding or anything that I can further check that I am doing everything correctly?

               total        used        free      shared  buff/cache   available
Mem:            31Gi       2.2Gi        26Gi       910Mi       2.7Gi        27Gi
Swap:             0B          0B          0B
7

Check if it is running:

sudo systemctl status systemd-swap.service

As said, it only creates a swap device if needed. If there is no need, then it will not create one.

So, to see a swap device, you need to fill up your ram.

8

it is activated. Thank you so much! you really save my life!

1 Like
9

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.