Best Allocation of Disk Space on New TrueOS Box


#1

I’m going to set up TrueOS on a different desktop that has a 1TB main drive and a 500Gb second hard drive, both SATA.

I can boot from either one. Right now I boot from the second drive (the 500 GB one) and that’s where I have
TrueOS (unstable 11 of 10/16/2017) installed.

I’m trying to figure out the best way to allocate the disk space so this is really a ZFS question.

Do I partition the 1TB drive into two 500Gb partitions and then set them up as two mirrors of the 500 Gb second drive ?

Can that all be done during the Install ?

Thanks !
J.


#2

personally, I would go with 2 500G partitions and mirror


#3

I’m being stupid. I should just go get a 1 TB disk and then use the other one as the mirror.

Thanks !

J.


#4

or that is also another option :wink:


#5

In general, mirroring on the same drive is not a great idea. It does improve redundancy, but if the drive fails, you lose all partitions on it, whereas if you’re mirroring across drives, then you have redundancy when a drive fails, not just when a block manages to get corrupted. So, if you can, you should definitely favor mirroring across drives.

In general, with ZFS, you want a single partition per disk, and then you either mirror them or use zraid (which is less flexible but more space efficient). Personally, I always set up my drives in mirrored pairs, and over time, as I’ve needed more space, I’ve upgraded a pair at a time. I have a separate pool for the OS install, because it’s on a pair of SSDs, but most of my data goes on a separate pool made up of mirrored pairs of spinning disks.


#6

Ya, very good points there.

I’m worried about SSD’s because I’ve heard that they are only good for so many writes and then they fail.

Way sooner than hard drive SATA’s ? Or so I’ve heard, I really don’t know.

Thanks !

J.


#7

Older SSDs definitely had problems with that, and modern SSDs do still have a limited number of writes, but it’s my understanding that with a modern SSD, it’s likely that your computer will last longer than the SSD, so I’m not all that worried about that. If you have a setup where you’re constantly writing large amounts of data to disk, then you probably shouldn’t use an SSD, but for a typical computer, as I understand it, it should be fine. The bigger problem IMHO is that their error rate is higher than that of spinning disks, so the risk of data corruption is higher, but that’s just one more reason to be using ZFS, since it can deal with that.

The big reason that I don’t use SSDs more is that they’re not cost-effective. While SSDs are much cheaper than they used to be, traditional, spinning disks are still drastically cheaper, and it seems likely that they’re going to stay that way. So, I just use the SSDs for the OS and my home directory so that the OS and config stuff has faster disk access, and most of my data sits on spinning disks. And I don’t think that that approach is all that uncommon. As I understand it, plenty of folks use an SSD for their OS, and spinning disks for the rest - though as SSDs have improved and grown in size, more folks have been using them for everything - especially if they have less than a terabyte of data to worry about.

With my last disk upgrade, my data pool exceeded 30TiB of storage, so SSDs will clearly not work for me for most of my data storage. The cost of spinning hard drives is painful enough as it is (especially since mirroring doubles the cost), and no desktop would have enough SATA slots for the SSDs required to store that much data.


#8

The number of cycles for modern SSDs winds up being years to failure (like 5-10 years).
As others have noted, mirroring to different partitions on the same physical device is a bad idea. You appear to have redundancy, but single device failure kills you and you lose any benefit of the mirror.
Partitioning: keep in mind all 1TB disks are not the same. Some mfgs may count in powers of 2 others in powers of 10. If you are going to mirror or RAID, I think partitioning the disks is a good thing, you make the parititions exactly the same size. ZFS winds up using the smaller size when you start putting drives together: a mirror of a 1TB drive and a 500GB drive will only be a 500GB mirror. If you partition the 1TB drive into 2 500GB and then mirror one 500GB parition with the 500GB partition, overall performance will suffer.

@jmdavis has a good system; it’s always the data that is important, the OS and programs can usually be reinstalled. I like to separate my OS and my home directory onto physically different devices, makes it easy to upgrade the os. I’ve “evolved” to a SSD for the OS and mirror of 1TB drives for my home directory and other data (I don’t have as much data needs).

Since you (@Jimserac) say this is a desktop, if the existing disks are relatively new (spinning disks fail too), I’d simply get another 1TB drive, use the 500GB to install the OS on, then mirror the 1TB drives. I think that can be done during install, but it’s easy enough to do after install (I’d have the 500GB plugged in as the only drive during install). Once the mirror is set up, it’s easy (not trivial) to move your existing home dataset over to the mirror, change a few ZFS properties and you’re up and running.

If you’re interested in a lot of details, look for the ZFS books by Michael W Lucas and Allan Jude plus Michael’s Storage Essentials. Awesome references to have on hand for everything BSD storage related (my opinion and no I don’t get kickbacks).

A quick internet search for “how long does an SSD last” is interesting. An article on on ExtremeTech from back in 2014 shows a Samsung 840 250GB making it to 9000TB of writes, so if you do the math, that works out to “a heck of a long time”.


#9

I have my home directory on my OS pool for the extra speed, which has meant that I have to send its dataset and any children datasets to my data pool or to another machine when reinstalling the OS (which is a bit annoying), but it would basically be fatal if my data pool were my OS pool. It’s just too much data, and having to copy it to then copy it back would take forever, even if I had enough space to back it all up elsewhere (which unfortunately, at the moment, I don’t - a lot of it yes, but not all of it). So, I completely agree that separating the OS partition makes reinstalling much saner, though my original motivation was purely because I wanted the OS and config stuff on SSDs. But even with having to send my home directory dataset away before reinstalling and then send it back afterwards, ZFS makes it amazingly easy. It still feels kind of crazy to back up what would normally be a partition and then restore it exactly as it was even though the whole filesystem was just redone. It’s definitely one of those things that makes ZFS seem amazing to me.


#10

ZFS sending and receiving datasets: eyeopener when one is used to oldfashioned ways of doing remote backups. My user home dataset has actually been migrated from FreeBSD 10.x-Release up to my TrueOS install. Started out as a singleton, separate from the OS, OS spinning drive got replaced with SSD running FBSD11 (this feels like cheating, I simply unplug old working configuration so if anything is bad in new install, I swap cables and reboot) I make sure to push working /etc and /usr/local/etc over to my other disk. Then a new SSD (prices come way down) to install TrueOS; a bit of magic with ZFS to send over datasets, properties (mountpoints) and old user home dataset is running on TrueOS. Then go through the steps of mirroring that and life is good.

It all boils down to your needs and desires. Mirrors are great for redundancy, but the only performance gain is on reading. zraid configurations give you redundancy and typically space gains, but you have to be careful because you can’t simply keep adding drives (you can actually create mirrors with 3 drives which is a quick and easy way to get something replicated) and you need to be aware of how many you can lose before you can’t recover.


#11

Yes I’ve heard of those books, I may look into that.

Well, putting the OS on a separate disk is not something I’ve been doing but it does sound like an excellent idea.

I did get a Sans Disk SSD (120 GB) about 6 months ago and experimented with it. Of course the computer
runs much faster. I had Fedora 24 on it (or 23, can’t remember). But just a tad too small with all the stuff I’d want to install to use for TrueOS.

Most importantly of all, my habit of putting two mirros on the same disk in two different partitions is not a good idea, something I will change as soon as I can.

Thanks
J.


#12