Very slow boot times of over 2 minutes

Si I have just started using Manjaro and I definitely enjoy it. My only gripe right now is the very very long boot times I am experiencing. They started almost immediately.

First happens when I turn on the pc. I'm requested a password before GRUB shows. Then I wait for something that feels like an eternity. Then grub shows and I have to wait again for the pc to boot to the user interface.

My output looks like:
Startup finished in 11.249s (firmware) + 42.269s (loader) + 7.652s (kernel) + 1min 865ms (userspace) = 2min 2.036s
graphical.target reached after 1min 421ms in userspace

I'm booting from a HDD.

Best regards,

Welcome to Manjaro Forum!
:bouquet:

Perhaps
systemd-analyze blame
and
systemd-analyze critical-chain
would give some more details and some ideas to some community members on your boot process.

Are you using full system encryption?

They are inherently slower to boot from.

3 Likes

Thank you for welcoming me and taking your time to answer.

I have a full system encryption as per company policy. The data has to be encrypted at rest. Windows PCs are required to use bitlocker or veracrypt.

15.706s docker.service
10.413s NetworkManager-wait-online.service
9.967s snapd.service
8.906s lvm2-monitor.service
7.414s dev-mapper-luks\x2d9c90eda1\x2d50bb\x2d4a12\x2d9c3f\x2d06296bfcf3e8.device
5.646s org.cups.cupsd.service
5.515s ufw.service
4.085s systemd-journal-flush.service
3.581s polkit.service
3.387s apparmor.service
2.588s ModemManager.service
1.993s systemd-udevd.service
1.975s systemd-tmpfiles-clean.service
1.889s avahi-daemon.service
1.876s bluetooth.service
1.814s NetworkManager.service
1.700s systemd-sysusers.service
1.661s systemd-logind.service
1.351s systemd-random-seed.service
1.245s ldconfig.service

And here for the second output. My pc has a 8th gen intel i7 and a radeon card.

graphical.target @41.961s
└─multi-user.target @41.961s
  └─docker.service @26.254s +15.706s
    └─network-online.target @26.252s
      └─NetworkManager-wait-online.service @15.837s +10.413s
        └─NetworkManager.service @14.015s +1.814s
          └─dbus.service @13.791s
            └─basic.target @13.782s
              └─sockets.target @13.782s
                └─snapd.socket @13.777s +4ms
                  └─sysinit.target @13.747s
                    └─systemd-timesyncd.service @12.956s +790ms
                      └─systemd-tmpfiles-setup.service @12.351s +371ms
                        └─local-fs.target @12.349s
                          └─boot-efi.mount @12.231s +116ms
                            └─systemd-fsck@dev-disk-by\x2duuid-0CBE\x2dE6C1.service @11.978s>
                              └─local-fs-pre.target @11.97

6s

Best regards,

Try a different kernel, such as 4.14 or 4.19 instead of 5.4. Although, any slow times before Grub appears have nothing to do with Manjaro.

1 Like

Exactly, it's the firmware loading. A different kernel than 5.4 should help but the network connection seems to be taking a long time to establish as well. Make sure with whomever is responsible for configuring the LAN that you have the right settings in place. If possible use a fixed address rather than DHCP.

I will try the 4.19 LTS kernel. Kind of bummed. I'm running the 5.5.2-1 because 5.4 seemed a bit slow.

Is there any way to improve the speed to decrypt/unlock grub ? Because it is definitely taking from the feeling around 60% of my time. Between inputting the password and getting slot0 unlocked it takes ages.

I doubt it, with an HDD you are using software encryption (unless it's a specific enterprise grade drive and not a cheap WD Blue or Seagate Barracuda Green) so you arelimited by the rest of the hardware. an 8th gen i7 means nothing really as there are a miriad of different performance options with that brand name. Please post inxi -Fxzc0 so we can see what you actually have in there.

If you had a modern SSD, most have hardware level encryption which of course is much faster for the host system to lock and unlock.

I just checked the suggested kernels and they perform the same.

Here is the output

System:    Host: xxx-pc Kernel: 5.5.2-1-MANJARO x86_64 bits: 64 compiler: gcc 
           v: 9.2.0 Desktop: KDE Plasma 5.17.5 Distro: Manjaro Linux 
Machine:   Type: Laptop System: Dell product: Vostro 5471 v: N/A serial: <filter> 
           Mobo: Dell model: 05F5VX v: A00 serial: <filter> UEFI: Dell v: 1.13.0 
           date: 11/18/2019 
Battery:   ID-1: BAT0 charge: 25.6 Wh condition: 25.6/42.0 Wh (61%) 
           model: BYD DELL FW8KR82 status: Full 
CPU:       Topology: Quad Core model: Intel Core i7-8550U bits: 64 type: MT MCP 
           arch: Kaby Lake rev: A L2 cache: 8192 KiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx 
           bogomips: 32012 
           Speed: 1531 MHz min/max: 400/4000 MHz Core speeds (MHz): 1: 1531 2: 1834 
           3: 2095 4: 1573 5: 1378 6: 2206 7: 1423 8: 1357 
Graphics:  Device-1: Intel UHD Graphics 620 vendor: Dell driver: i915 v: kernel 
           bus ID: 00:02.0 
           Display: x11 server: X.Org 1.20.7 driver: amdgpu,intel FAILED: ati 
           unloaded: modesetting resolution: 1920x1080~60Hz 
           OpenGL: renderer: Mesa DRI Intel UHD Graphics 620 (Kabylake GT2) 
           v: 4.6 Mesa 19.3.3 direct render: Yes 
Audio:     Device-1: Intel Sunrise Point-LP HD Audio vendor: Dell driver: snd_hda_intel 
           v: kernel bus ID: 00:1f.3 
           Sound Server: ALSA v: k5.5.2-1-MANJARO 
Network:   Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet vendor: Dell 
           driver: r8168 v: 8.048.00-NAPI port: d000 bus ID: 02:00.0 
           IF: enp2s0 state: down mac: <filter> 
           Device-2: Intel Wireless 3165 driver: iwlwifi v: kernel port: d000 
           bus ID: 03:00.0 
           IF: wlp3s0 state: up mac: <filter> 
           IF-ID-1: docker0 state: down mac: <filter> 
Drives:    Local Storage: total: 1.03 TiB used: 23.59 GiB (2.2%) 
           ID-1: /dev/sda vendor: Seagate model: ST1000LM035-1RK172 size: 931.51 GiB 
           ID-2: /dev/sdb vendor: SK Hynix model: SC311 SATA 128GB size: 119.24 GiB 
RAID:      Hardware-1: Intel 82801 Mobile SATA Controller [RAID mode] driver: ahci v: 3.0 
           bus ID: 00:17.0 
Partition: ID-1: / size: 450.78 GiB used: 23.53 GiB (5.2%) fs: ext4 dev: /dev/dm-0 
Sensors:   System Temperatures: cpu: 39.0 C mobo: 38.0 C sodimm: 38.0 C gpu: amdgpu 
           temp: 42 C 
           Fan Speeds (RPM): cpu: 0 
Info:      Processes: 265 Uptime: 7m Memory: 7.68 GiB used: 1.96 GiB (25.6%) 
           Init: systemd Compilers: gcc: 9.2.0 Shell: bash v: 5.0.11 inxi: 3.0.37

Well Is there a way to avoid software decryption or alternatives not requiring to change hardware? I just timed how much it took between pressing enter on the entered password pre grub and till the moment grub appears I have to wait over 30 seconds. It seems to be much longer than windows on that hdd with fastboot disabled.

I don't have much experience with encryption so this is purely a question of satisfying my curiosity. Wouldn't encrypting only the home folder speed things up a bit?

yes it probably but their employers want him to use full drive encryption for some reason. maybe they're a 00 :man_shrugging:

1 Like

I thought there might not be so much on the system itself worth encrypting, the valuable data residing in the home.

that's a password stored and controlled by the laptop firmware, nothing to do with manjaro or grub. it's also possibly to do with the fake RAID mode employed by the storage controller. as suspected though, your i7 is the ultra low power model so not actually that potent. it still wouldn't affect the boot time but for the record, unless you are running on mains and the laptop is not too hot, you will never get the 4.0 GHz boost speed from it.

I dumb question but why is GRUB encrypted in the first place ? I just tested the encrpytion decryption speed from live USB and it is instantaneous.

Hehe nothing like that. But since it's a laptop it can get stolen easily. So the attack vector is extracting the HDD and copying files and other simple data like login information as far as I understand. This is the main point of contention. If that can reside in an encrypted part then win win.

Ah, ok. I'm not entirely familiar with all the vocabulary. It's just since I installed luks that I'm required to provide it.

The only way you're going to get this cow of a laptop to become a working horse and I'm not even talking about a racing horse, just one of these:

Brabants trekpaard

is by:

  1. disabling hardware encryption and violating corporate policy
  2. changing the HDD to an SLC or eMLC SSD (not just any SSD: the expensive ones!), not like the 128 GB SK hynix 48-layer V3 3D TLC NAND that is hidden behind the 1TB drive: that's TLC, three times as slow as SLC

Case closed, your honour! :sob:

Upgrading it might not be an option if the machine is supplied as part of a contract with Dell.

If however it is, then yes a Samsung EVO 970 Plus would give it a real kick up the backside while providing hardware encryption. the Hynix drive is an SSD but standard SATA variety, not a proper 4 lane PCI-E nvme drive.

The HDD is a 5400rpm Seagate, @Nyquist from the size of the root partition it looks like you installed to the HDD. If possible it is better to install the operating system to the faster Hynix drive instead. Obviously you'd have to start again though and if you aren't booting Windows too, change the RAID controller mode to AHCI instead of RAID.

As a feedback. I had to change the luks key and reduce the number of iterations. After that without any other modification it shaved off 45 seconds and reduced the total boot time by over half.

Startup finished in 11.323s (firmware) + 15.614s (loader) + 7.602s (kernel) + 39.674s (userspace) = 1min 14.215s 
graphical.target reached after 39.343s in userspace

I maintain my case that this is linked with the encryption and decryption since otherwise there would be no reason to attain such a performance boost by just reducing the -i paramater.

Hardware upgrades are not possible sadly.

Best regards,

1 Like

:sunglasses: :exclamation:

Could you edit your last answer to include how you reduced the number of iterations while still falling inside corporate policy? That might help the next person who runs into the same problem.

After that, please mark your own answer as "Solution" like this:

Solution

:wink: :+1: :innocent:

1 Like

I proceeded the following way :slight_smile:

The bigger issue was the time taken to unlock the luks volume before loading the kernel:

sudo cryptsetup luksDump /dev/sdX

I saw that the key in slot0 required over 2 million iterations before unlocking which is absolutely insane for software decryption. Same with key in slot 1.

I applied the following command to create a new key:

sudo cryptsetup luksChangeKey /dev/sdaX

-i parameter can be whatever since it is not the main objective it allows to free slot0 key and insure that you have a functionning secondary key in slot2 and free slot0.

Then just add a new key to slot0. with -i parameter sufficiently low. Don't forget it seems that when the software estimates the number of iterations it uses full HW acceleration. Thus even on laptop CPUs it uses over 2mil iterations easily for each key. Since it iterates over all keys in succession you haven't gained anything if you don't add a key to the slot0.

you can use the following command to add back slot0

sudo cryptsetup luksAddKey /dev/sdX -S 0

And done.

1 Like

Forum kindly sponsored by