Hello,
I’ve been using Manjaro to run a 24/7 server which, among other things, does some file backup. Up until last week, it was running just fine but now that I have done the huge update, I get the following error message every few days:
rsync: [sender] readlink_stat("/files/web/backup_20180819/www/mod/comments/img/back_to_top.png") failed: Bad message (74)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=v3.2.3]
It’s always a different file when this happens, sometimes even up to 5 files at at time, but always in a subdir of the /files mount point.
Luckily, this is not on the main file system, so I can stop every service accessing /files, unmount it and run this command:
[obones@server ~]$ sudo fsck /dev/sdb1
fsck de util-linux 2.36.2
e2fsck 1.46.2 (28-Feb-2021)
/dev/sdb1 contient un système de fichiers comportant des erreurs, vérification forcée.
Passe 1 : vérification des i-noeuds, des blocs et des tailles
L'arbre d'extent de l'i-noeud 88872381 (au niveau 2) pourrait être plus étroit. Optimiser<o>? oui pour tout
Passe 1E : Optimisation des arbres d'extent
Passe 2 : vérification de la structure des répertoires
Passe 3 : vérification de la connectivité des répertoires
Passe 4 : vérification des compteurs de référence
Passe 5 : vérification de l'information du sommaire de groupe
/dev/sdb1: ***** LE SYSTÈME DE FICHIERS A ÉTÉ MODIFIÉ *****
/dev/sdb1 : 1202591/122101760 fichiers (0.2% non contigus), 241306152/488378368 blocs
Once this has been done, the problematic files are accessible again, undamaged and the next rsync command works just fine.
I thought that maybe the harddrive onto which /dev/sdb1 is based is dying, so I ran this command:
[obones@server ~]$ sudo smartctl -a /dev/sdb
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.4.105-1-MANJARO] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate IronWolf
Device Model: ST2000VN004-2E4164
Serial Number: Z527RCM2
LU WWN Device Id: 5 000c50 0b4e5a1b9
Firmware Version: SC60
User Capacity: 2 000 398 934 016 bytes [2,00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5900 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Apr 4 11:09:51 2021 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 107) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 257) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x10bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 108 099 006 Pre-fail Always - 19463616
3 Spin_Up_Time 0x0003 096 095 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 44
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 072 060 030 Pre-fail Always - 17205909
9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 16867
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 51
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0
189 High_Fly_Writes 0x003a 075 075 000 Old_age Always - 25
190 Airflow_Temperature_Cel 0x0022 064 057 045 Old_age Always - 36 (Min/Max 25/38)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 16
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 92
194 Temperature_Celsius 0x0022 036 043 000 Old_age Always - 36 (0 13 0 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 16816 -
# 2 Short offline Completed without error 00% 15643 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
But to me, it’s not showing any specific issues. Running a self test yields no error. Using the serial number at the manufacturer’s website tells me its warranty expires in 2022, so I believe it’s not that old.
I believe the update that I did on March 27th brought something strange, but I cannot figure out what it could be with the huge number of packets that were updated.
The result of uname -a is this:
Linux server 5.4.105-1-MANJARO #1 SMP PREEMPT Thu Mar 11 18:27:58 UTC 2021 x86_64 GNU/Linux
I know that I’m not on the latest 5.10 kernel but I can’t use it because the video card is a bit dated and does not have drivers available for that kernel.
Would you have any suggestion on my filesystem issue?
Thanks for your help
