What sort of performance should I expect?
-
Very interesting thoughts from someone who obviously has a lot more RAID experience than I do. I appreciate you sharing that. Are failures during rebuilds so common that RAID admins consider them to be a real concern? I don't question the possibility of having two failures occurring close to each other in time...I'm wondering if you're suggesting that a RAID 5 config exacerbates the likelihood of multiple failures. I suppose all drives are working harder while the whole thing is being rebuilt... In a way, I'm ok(-ish) with the possible loss - the whole RAID is intended to only act as an additional backup set (not my only backup set). So long as it doesn't take a week to rebuild if I do ever encounter a failure. And since this is only for backups, I'm not terribly concerned about squeezing every last iota of performance - but again, not sacrificing performance to the extent that I'm seeing right now. As mentioned elsewhere in this thread, I thought RAID5 was a good compromise between redundancy and capacity. I'm not a huge fan of the thought of pure duplication (eg, losing 50% of total capacity). OTOH, I'd be willing to do it if it guaranteed it brought performance back to where it ought to be. But if it only made things marginally faster, I don't think I'd be in a much better place.
dandy72 wrote:
suggesting that a RAID 5 config exacerbates
IMO, it's the drive count. For RAID 5, it's just a sort of high barrier to entry where it's just not worth doing unless you're using a fairly large number of drives (9+) because fewer drives tends inherently mean the recoverability/performance benefits would be better in a different RAID mode. You're right though. Something with that enclosure or one of those drives or something about the NAS networking is not right.
-
[tl;dr]: Is RAID5 really causing such a huge performance hit? I have a system (a Hyper-V VM host) with both eSATA and USB3.0 connectors. I have a retired set of 8TB drives. I got myself a Mediasonic HFR2-SU3S2 PRORAID 4-drive enclosure, which can use either connector. I love how trivial this enclosure's RAID setup is. I chose RAID5, so I have a total of 24TB worth of storage. Performance however makes it downright unusable. I could leave my VMs powered down overnight to back them up, but what I'm currently seeing could take days. Backing up a VM while it's running is just not a good idea (I use robocopy) so the VMs have to remain down while backing them up. That's not gonna fly during my workweek. I made sure that, whether I'm using USB3 or eSATA, the "Better Performance" radio button is selected in Device Manager / Disk drives / [the RAID enclosure] / Properties / Policies. Write operations hold steady at ~2.6MB/s. Active time is flat at 100%. Same setup, but using eSATA instead, holds steady at ~5MB/s. Better, but still way below expectations. I'm questioning what my expectations should be. The OS sees the RAID, not individual drives. On top of that, I use VeraCrypt to encrypt the entire RAID. I understand RAID involves some overhead, especially for Write operations--parity calculations would be done by the enclosure hardware, not my VM host's CPU. OTOH, VeraCrypt also introduces its own overhead, and *that* would be done by the host's CPU (which holds steady at ~3-4% when copying, so that's hardly the killer). Before I got the RAID enclosure, I backed up the VMs onto a single external disk over USB3, and there was always plenty of time to do the whole thing overnight. I forget what I got in terms of transfer rate, but I'll be sure to pay attention the next time I do it - surely at least 10x the current performance. That single disk is also encrypted with VeraCrypt, so--unless I'm missing something--the only thing left that can account for the difference in transfer rate is the fact that the target drives are set up in a RAID, as opposed to transferring to a single drive. My (somewhat rhetorical) question is: Really? Does my diagnostic make sense? Is the fact that I'm backing up to a RAID the real performance killer? Everything is otherwise the same - both the RAID and my single external drive are connected via USB3, *and* using VeraCrypt. Does it make sense at all that RAID5 kills performance to the extent I'm seeing? What *woul
-
Daniel Pfeffer wrote:
Unless you are writing large amounts of tiny files, I would expect at least that sort of performance out of any NAS.
So far I've only been trying to backup VM files - so, very few, but rather large files (multiple dozen GBs in size).
Daniel Pfeffer wrote:
Even using USB 2, the transfer rates are very low
Agreed, I've been using other single drives using the same USB3 port and connector, and get much better speeds. So I have to rule out this beign stuck in USB 2 mode. Your idea of testing out each drive separately is a good one. I wonder if I could power down the RAID, take one drive, put it in an enclosure and, without reformating it, run some sort of non-destructive speed test, then put the drive back in the RAID. I'd hate to rebuild it all over again at this point.
dandy72 wrote:
I wonder if I could power down the RAID, take one drive, put it in an enclosure and, without reformating it, run some sort of non-destructive speed test
Given that you want to test WRITE speed I would say no, not unless you can find a non-destructive write test. I don't know of any test program that works like that. You may also have to take into account the behaviour of the NAS. It may detect that the drive is part of a very degraded RAID-5 set and refuse to mount it. You may have to tell the NAS that this is a JBOD and have it reconfigure the drive before it can be accessed.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
[tl;dr]: Is RAID5 really causing such a huge performance hit? I have a system (a Hyper-V VM host) with both eSATA and USB3.0 connectors. I have a retired set of 8TB drives. I got myself a Mediasonic HFR2-SU3S2 PRORAID 4-drive enclosure, which can use either connector. I love how trivial this enclosure's RAID setup is. I chose RAID5, so I have a total of 24TB worth of storage. Performance however makes it downright unusable. I could leave my VMs powered down overnight to back them up, but what I'm currently seeing could take days. Backing up a VM while it's running is just not a good idea (I use robocopy) so the VMs have to remain down while backing them up. That's not gonna fly during my workweek. I made sure that, whether I'm using USB3 or eSATA, the "Better Performance" radio button is selected in Device Manager / Disk drives / [the RAID enclosure] / Properties / Policies. Write operations hold steady at ~2.6MB/s. Active time is flat at 100%. Same setup, but using eSATA instead, holds steady at ~5MB/s. Better, but still way below expectations. I'm questioning what my expectations should be. The OS sees the RAID, not individual drives. On top of that, I use VeraCrypt to encrypt the entire RAID. I understand RAID involves some overhead, especially for Write operations--parity calculations would be done by the enclosure hardware, not my VM host's CPU. OTOH, VeraCrypt also introduces its own overhead, and *that* would be done by the host's CPU (which holds steady at ~3-4% when copying, so that's hardly the killer). Before I got the RAID enclosure, I backed up the VMs onto a single external disk over USB3, and there was always plenty of time to do the whole thing overnight. I forget what I got in terms of transfer rate, but I'll be sure to pay attention the next time I do it - surely at least 10x the current performance. That single disk is also encrypted with VeraCrypt, so--unless I'm missing something--the only thing left that can account for the difference in transfer rate is the fact that the target drives are set up in a RAID, as opposed to transferring to a single drive. My (somewhat rhetorical) question is: Really? Does my diagnostic make sense? Is the fact that I'm backing up to a RAID the real performance killer? Everything is otherwise the same - both the RAID and my single external drive are connected via USB3, *and* using VeraCrypt. Does it make sense at all that RAID5 kills performance to the extent I'm seeing? What *woul
Years ago, I lived in the land of RAID. In my case, I had the need for high throughput for industrial systems. In circa 2000 all we had were spinners, and the fastest were SCSI. So, RAID made sense for higher performance and redundancy. I forget the manufacturer but they were high end and we paid a lot of $$ for these units. Redundant power supplies, controllers, drives, etc. Then one day, one of the controllers failed, and we lost the entire RAID. Seems the controller boards were sort of redundant. As the manager for the data group, it was an interesting conversation with the tech, support and ultimately a VP. It started with "you have to be elephanting kidding me?" Where upon I replaced their product. Why do I say this? First, RAID helped but tech has passed on. If you are doing this for a science experiment, fine. For day to day, it's just not worth the hassle. Go buy some 4 tb usb drives and move on. But let's talk about specifications. It is true that USB 2.0 supports UP TO 480 Mbps. USB 3.0 supports UP TO 5.0 Gbps. Note the "UP TO." The interface may support it, but I have yet to find a device that even approaches this transfer rate. I admit I have not tried a RAM disk. I have a fairly high end laptop with USB 3.2 on it. I can plug in a USB 3.2 SSD and copy my VMs to it. The burst speed is actually quite good, but when the cache fills, the transfer rate drops to 10% of the spec rate (I'm ballparking here, it's been a while since I did the test). So, if you are moving large files - like me - 150GB for a VM via cut/paste, the cache will fill up and you will see perf degrade. This is with modern hardware. Your old spinners? I'd expect worse.
Charlie Gilley “They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759 Has never been more appropriate.
-
[tl;dr]: Is RAID5 really causing such a huge performance hit? I have a system (a Hyper-V VM host) with both eSATA and USB3.0 connectors. I have a retired set of 8TB drives. I got myself a Mediasonic HFR2-SU3S2 PRORAID 4-drive enclosure, which can use either connector. I love how trivial this enclosure's RAID setup is. I chose RAID5, so I have a total of 24TB worth of storage. Performance however makes it downright unusable. I could leave my VMs powered down overnight to back them up, but what I'm currently seeing could take days. Backing up a VM while it's running is just not a good idea (I use robocopy) so the VMs have to remain down while backing them up. That's not gonna fly during my workweek. I made sure that, whether I'm using USB3 or eSATA, the "Better Performance" radio button is selected in Device Manager / Disk drives / [the RAID enclosure] / Properties / Policies. Write operations hold steady at ~2.6MB/s. Active time is flat at 100%. Same setup, but using eSATA instead, holds steady at ~5MB/s. Better, but still way below expectations. I'm questioning what my expectations should be. The OS sees the RAID, not individual drives. On top of that, I use VeraCrypt to encrypt the entire RAID. I understand RAID involves some overhead, especially for Write operations--parity calculations would be done by the enclosure hardware, not my VM host's CPU. OTOH, VeraCrypt also introduces its own overhead, and *that* would be done by the host's CPU (which holds steady at ~3-4% when copying, so that's hardly the killer). Before I got the RAID enclosure, I backed up the VMs onto a single external disk over USB3, and there was always plenty of time to do the whole thing overnight. I forget what I got in terms of transfer rate, but I'll be sure to pay attention the next time I do it - surely at least 10x the current performance. That single disk is also encrypted with VeraCrypt, so--unless I'm missing something--the only thing left that can account for the difference in transfer rate is the fact that the target drives are set up in a RAID, as opposed to transferring to a single drive. My (somewhat rhetorical) question is: Really? Does my diagnostic make sense? Is the fact that I'm backing up to a RAID the real performance killer? Everything is otherwise the same - both the RAID and my single external drive are connected via USB3, *and* using VeraCrypt. Does it make sense at all that RAID5 kills performance to the extent I'm seeing? What *woul
Just to confirm, the USB3 connection is USB-C? I have a NVME enclosure which had full USB3 speeds in one cable orientation, but if flipped 180 it would be USB2. I would hope it's not something as simple as that. Can't hurt to try. I know that's the whole point of USB-C, however, that doesn't mean everyone handles things correctly
-
Just a thought, have you tried the enclosure on a USB 2 socket?
So old that I did my first coding in octal via switches on a DEC PDP 8
-
dandy72 wrote:
I wonder if I could power down the RAID, take one drive, put it in an enclosure and, without reformating it, run some sort of non-destructive speed test
Given that you want to test WRITE speed I would say no, not unless you can find a non-destructive write test. I don't know of any test program that works like that. You may also have to take into account the behaviour of the NAS. It may detect that the drive is part of a very degraded RAID-5 set and refuse to mount it. You may have to tell the NAS that this is a JBOD and have it reconfigure the drive before it can be accessed.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
I know Steve Gibson's Spin-Rite performs non-destructive read/write operations - it'll read what's on a sector and hang onto that data before performing a write test, and then write back what was originally there, regardless of file system, OS, encryption, etc. I suppose that would do it. But I don't think Spin-Rite has anything about reporting read/write speeds. Might still be worth a shot. Also - strictly speaking - this is a RAID enclosure, not a NAS... I believe it's supposed to turn on some red LED if it detects any sort of problem, but that's not the case here.
-
Years ago, I lived in the land of RAID. In my case, I had the need for high throughput for industrial systems. In circa 2000 all we had were spinners, and the fastest were SCSI. So, RAID made sense for higher performance and redundancy. I forget the manufacturer but they were high end and we paid a lot of $$ for these units. Redundant power supplies, controllers, drives, etc. Then one day, one of the controllers failed, and we lost the entire RAID. Seems the controller boards were sort of redundant. As the manager for the data group, it was an interesting conversation with the tech, support and ultimately a VP. It started with "you have to be elephanting kidding me?" Where upon I replaced their product. Why do I say this? First, RAID helped but tech has passed on. If you are doing this for a science experiment, fine. For day to day, it's just not worth the hassle. Go buy some 4 tb usb drives and move on. But let's talk about specifications. It is true that USB 2.0 supports UP TO 480 Mbps. USB 3.0 supports UP TO 5.0 Gbps. Note the "UP TO." The interface may support it, but I have yet to find a device that even approaches this transfer rate. I admit I have not tried a RAM disk. I have a fairly high end laptop with USB 3.2 on it. I can plug in a USB 3.2 SSD and copy my VMs to it. The burst speed is actually quite good, but when the cache fills, the transfer rate drops to 10% of the spec rate (I'm ballparking here, it's been a while since I did the test). So, if you are moving large files - like me - 150GB for a VM via cut/paste, the cache will fill up and you will see perf degrade. This is with modern hardware. Your old spinners? I'd expect worse.
Charlie Gilley “They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759 Has never been more appropriate.
I had a similar experience decades ago with a RAID controller that failed. RAID sounds great in theory, but when it's the controller that dies...you're no better off, and you now have more data that's unrecoverable than you would had you been using a single, smaller drive... I reluctantly gave up on the idea for the longest time, concluding that a good RAID just isn't available at consumer prices. This enclosure was under $200, so I figured why not give it another shot - especially since I had four of these 8TB drives not doing anything anymore.
charlieg wrote:
Go buy some 4 tb usb drives and move on.
Actually I have more than enough larger drives already. They're sitting here doing nothing, which is the whole reason I decided to try to put them to some use in a RAID setup.
charlieg wrote:
So, if you are moving large files - like me - 150GB for a VM via cut/paste, the cache will fill up and you will see perf degrade
I've definitely seen this. Reading the first couple of GBs with robocopy is fast. Watching Task Manager's memory usage is rather interesting during that time period. Then it flattens out, and performance starts to crawl once it reaches a certain point.
-
Just to confirm, the USB3 connection is USB-C? I have a NVME enclosure which had full USB3 speeds in one cable orientation, but if flipped 180 it would be USB2. I would hope it's not something as simple as that. Can't hurt to try. I know that's the whole point of USB-C, however, that doesn't mean everyone handles things correctly
The docs say the enclosure is USB 3.0 - not 3.1, not USB-C (which, as I understand it, can be faster still). The cable that came with it is the plain ol', standard USB-A (rectangular) at one end, and USB-B (commonly used for printers) at the other, so I can't get them wrong. The connectors are blue, which suggests USB 3 (and not 2).
-
I had a similar experience decades ago with a RAID controller that failed. RAID sounds great in theory, but when it's the controller that dies...you're no better off, and you now have more data that's unrecoverable than you would had you been using a single, smaller drive... I reluctantly gave up on the idea for the longest time, concluding that a good RAID just isn't available at consumer prices. This enclosure was under $200, so I figured why not give it another shot - especially since I had four of these 8TB drives not doing anything anymore.
charlieg wrote:
Go buy some 4 tb usb drives and move on.
Actually I have more than enough larger drives already. They're sitting here doing nothing, which is the whole reason I decided to try to put them to some use in a RAID setup.
charlieg wrote:
So, if you are moving large files - like me - 150GB for a VM via cut/paste, the cache will fill up and you will see perf degrade
I've definitely seen this. Reading the first couple of GBs with robocopy is fast. Watching Task Manager's memory usage is rather interesting during that time period. Then it flattens out, and performance starts to crawl once it reaches a certain point.
"concluding that a good RAID just isn't available at consumer prices." we ended up removing the raid from the system overall, and this was a pure hi-end industrial unit. Typing this reminds me I need to do backups :)
Charlie Gilley “They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759 Has never been more appropriate.