Transfer trueOS to another SSD


#1

Hello !

How can I transfer my complete trueOS installation to another SSD ?

(now I have trueOS on nvme1 but I will transfer the complete OS to nvme0 now)

Thanks and reegards,
Max.


#2

You can take a look at this thread.
Obviously there is not much but the dd command seems to do the job.


#3

I use dd to have a ready cloned TrueOs backup full working from an usb stick to another, needing to run TrueOs installed on an usb volume in order to keep my hd (win10 installation and boot sector) safe from accidental data deletion or alteration.

Cloning with dd is a simple and effective way to transfer a bootable system from a volume to another. You just make sure that the destination volume is at least same physical size or larger than the origin volume size, not even a single byte less.


#4

More details to be found here:

http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-091-sol-dis-recovery-489183.html


#5

We think dd is not the optimal way to transfer data from one SSD to a different one, since usually disk space is not fully used in file systems, but often completely allocated on block level.

Thus any dd copy operation would also copy data not used up by ZFS or UFS, but which was used in the past and therefore contains data on a block level. Since the SSD controller does not understand file systems like ZFS, and TRIM plus Garbage Collection data is not transferred, your new SSD probably will use more space than necessary. However, the amount depends heavily on TRIM configuration within the OS and the SSD controller hardware.

In case dd is still the desired way one should make sure TRIM is executed reliably prior to dd e.g. with fstrim or fsck_ffs or let the OS idle for a certain period of time (hours), and the old SSD stays powered on but is disconnected from the bus or the computer idles at the BIOS/EFI password prompt for a certain period of time (hours), to let the GC complete its job. After that dd can be used without/with little problems.

A better way to mirror a root ZFS device (one where the OS is installed on) can be found here: https://twitter.com/TRONDELTA/status/978611581541126145
The article “Converting a 1-disk ZFS pool into a 2-disk mirrored pool” describes the procedure. Afterwards the original/source SSD can be uninstalled from the system. The difference here is, that 1. the boot partition is very small compared to the others, and 2. ZFS datasets are copied on file system (ZFS) level; thus only in-use blocks are being transferred. Note: The manual does only work for ZFS and one SSD, with everything installed.


#6

Please: Explain, why TRIM is included in Your reasoning.

As far as I know, it’s totally unrelated to the usage and results of using of ‘dd’.

If You do a block level copy, then the blocks requested will be copied regardless of their being zeroed by TRIM or not.

Edit:
Content of copied blocks may differ, amount of blocks copied won’t.

Correct me, if I’m wrong.


#7

One more thing:

TRIM firmware is a black box. Only the manufacturer knows what it does.

Some firmware is fake, and only simulates doing anything. SSDs should be kept far away from secure environments.


#8

It is, because TRIM does have an impact on file system operations with SSDs (or most other flash/cell-based) storage. TRIM can be executed in batches or on-demand, while the OS kernel, FS and vFS implementation and the underlying hardware influence TRIM heavily. Even after a dd TRIM can be executed, but with a reduced degree of reliability for the yet copied data. That is especially the case for data written beyond OS file system boundaries, or in the accessible and/or inaccessible reserve block areas (yes, that sometimes possible to accomplish).

We think that is not the case and suggest to re-read the previous post or look up more information on TRIM, GC and file systems. All NAND Flash-based SSDs use GC. Some use foreground GC and some use background or idle-time GC. Also most OS kernels implement TRIM support for flash devices. However, the implementations of GC as well as TRIM differ significantly. If someone were to execute dd, e.g. after heavy disk activity, before TRIM execution and before GC execution that could have undesired effects.

Exactly, that is the problem. The blocks are copied regardless of their data contained, even when the data is unneeded crap. However, the GC state and TRIM states are not transferred or are non-transferable. Even zeroing the free space manually makes all those blocks appear to be in use to the drive’s firmware. They will have to be re-blanked the next time they are written. Using dd to save and restore entire partitions is therefore a bad idea with an SSD. It depletes the the supply of free blocks and can cause even worse side effects, like depleting or poisoning the hidden reserve block area.

Yes, correct (manual interventions not considered). Interventions considered though, it is twice as important to exactly specify the amount of bytes or blocks during dd or ddrescue, since overwrites may be possible, depending on the target drive logic. The amount of to-copy blocks should at least be specified, when the source device is larger than the destination. Unfortunately specifying a smaller amount of blocks to be copied is not advised, when the source drive is larger or only partially occupied, since the SSD controller stores data at arbitrary locations. Therefore dd will increment addresses, copy blocks from there and miss out something. That effectively means one has to copy the whole source drive via dd to a drive equal or larger in size, no matter the usage level of the source SSD – a thing which is simply undesired.

Yes, its a black box which makes things even worse. Within our organization we lately replaced Crucial BX 200 SSD’s with Samsung PM 863(a) on Desktops and Samsung SM 863 on Servers (OS only), which have a SC (Super Capacitor). The Samsung disks are far superior and a bit less expensive than comparable devices from Intel. And yes, SSDs should not be trusted similarly to SSHDs or HHDs and at least be mixed by brand and put into ZFS mirror (RAID1) or hardware RAID 1, with a SSD-capable RAID controller with BBU. Also SSDs must employ a so called SC (Super Capacitor) mandatorily, as mentioned before, which e.g. consumer disks like Samsung 840/850 (Pro) do not have.

Bottom line: Run TRIM prior to dd copy operations and let the drive idle over night to trigger controller GC too. Also dd to a larger destination SSD, if you can afford that, and never fill up SSDs beyond ~80% (hidden reserve areas not taken into account). Best though is to copy on file system level and thus copy only what’s needed and nothing more.


#9

Summary:
1)
Don’t use dd on SSD-partitions, because blocks are no longer ordered sequentially (one partition’s blocks all together). Therefore: Whole disk or nothing. Because only filesystem drivers find their blocks and ‘dd’ relies on legacy sequential blocks which can’t be guaranteed on SSDs.

I hope, that’s correct understanding of Your message?

You and I agree, that ‘dd’ shouldn’t be used. My more traditional reason is: ‘dd’ duplicates unique numbers/IDs. That makes them non-unique, obviously, and breaks their associations within configurations.


#10

Something like
dd if=/dev/ada1 of=/dev/ada2 bs=1M
will certainly do the job, but unoccupied blocks will be transferred. Extra wear and tear on the destination that is not needed.
Since ZFS is smart about stuff like this, use gpart to read the existing disklabel, gpart to partition the new SSD, then let ZFS do it’s thing as a mirror.


#11

Yes, we think we agree with both statements.