Dd tool failed me


#1

Hey guys,

I wanted to mention this since I just noticed it. A while back I made manjaro usb drive with the dd cli tool but when I tried to boot into it today grub kept going into rescue mode saying unknown filesystem. I fixed it by rewriting the manjaro iso with rufus (in dd mode) on a windows machine but if all my stuff was linux I’d be stuck right now trying to make a manjaro usb so I wanted to post this. Has anyone had this issue? I couldn’t google it.

Side question: Why isn’t gparted or any partition manager aside from the one tied with the installer loaded onto the iso? I tried to download it to the live usb but it didn’t have connections to the msnjaro repos.


#2

dd is not failing you. As with any tool writing data to another disk-based media you need to wait until all data is flushed to disk before removing it.

dd does not give you any indication if the data is flushed to disk.

When dd return you to the CLI you only know that dd has delivered that data to the system. Depending on your system that data might not yet be written to the media.

When using dd always use the command sync before removing the media from the system.

If you do not - you will get the error you mentioned.


Side answer:
Depending on the edition - there will always be some kind of partition tool.

Some editions only gives you CLI tools. No edition gives you all possible tools.

Depending on your system it will always be possible to install something in the live environment and it will always be limited to the available amount of memory and an available internet connection.

The steps to install something is

sudo pacman -Syy
sudo pacman -S package

#3

since dd is a cli tool wouldn’t the indication be when the terminal prompt returned waiting for a another command? The mode I ran dd in at the time gave a progress report as it worked so if it didn’t finished I would’ve seen it.

True. pacman -Syyu may have worked. I didn’t go that far. Instead I resorted to using the windows partition tool from the windows parition. (and I’m glad I did because I discovered that sadly the windows disk management tool wouldn’t let me shrink the windows partition any further so I’m going to try to start prepping myself for telling the manjaro installer to wipe everything and install it’s partitions) To be honest though I was surprised to discovered that I’d have to run that command just to get connected to the repos.


#4

No.

The dd utility is not writing directly to the media but through the system /dev/sdX. So when dd is finished and returning you to CLI this only indicates that the dd part of the job is finished.

A crude comparison is: You take the garbage, give it to your son and tell him to dispose it in the bin.

Can you consider the garbage disposed off? From your point of view, yes.

Can you be sure the garbage is disposed off? No. not unless you verify that your son did do as you told him.

From the system’s pow, some of the data still reside in the memory cache and needs to be written to the media.

That is what the eject function is for in file managers - whether Windows or Linux file managers.

When using CLI the only way to be sure is to use the sync command.


When you are running on a live ISO system there is no package databases as those databases would be obsolete when you download the ISO - the nature of a rolling distribution. If they were existing they would not match any repo databases on any mirror and thus give you problems as packages mentioned in the databases would not be available on the mirrors.

That is why you need to syncronize to a mirror before you can install anything in a live system.


#5

It’s a good idea to use the status=progress parameter to get an idea of the progress. One could also add oflag=sync parameter.


#6

Yes I know to eject from the file manager. I’m surprised though that umount doesn’t do the same. You make a good point though that could’ve been it. There was a least one time on Manjaro when I didn’t let it finish though I doubt it was then.

Yep that’s what I used. Don’t remeber what I set oflag to but probably that. I got the instructions from the archwiki.


#7

To make sure everything got transferred correctly to the USB stick, you can also use dd to verify the checksum of the written image:

dd if=/dev/sdb iflag=count_bytes count=xxxx | sha256sum (replace /dev/sdb according to your setup)

replace xxxx with the number of bytes written, which you get from the last line after the first dd command finishes. Then just compare the checksum with the one from the ISO.

I haven’t tested this!


Error Bad source source="/run/miso/bootmnt/manjaro/x86_64/rootfs.sfs"
#8

that’s true maybe that was issue. I was also surprised to discover that rufus doesn’t have an option to check what it has written yet it does have an option to check for bad sectors of the device it’s writing to.


#9

I always use ‘sync’ from a terminal and wait for completion.
There are some good dd options mentioned above.