I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
Echo the point made that RAID is NOT a backup. I once had a RAID
controller die on me and there were 4 drives attached to it. Lost ALL the >data on those drives even through all of teh drives themseves were not >faulty.
On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
wrote:
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
Not necessarily the system, just that the four drives are all being used
as one concatenated big one. If quarter of your big drive dies, you do
indeed lose everything.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
Yep.
If there is going to be data on here that is *only* on here, remember
that RAID is not anything remotely like backup, so you should plan to
have another 4tb somewhere (or larger) that you can run backups of this
array off to, to prevent issues with accidental deletion or the chassis
dying or whatever.
You'll need another 4TB anyway to copy everything off and back on again.
Cheers - Jaimie
On 08/07/2023 in message <u8baov$1mvcr$1@dont-email.me> SH wrote:
Echo the point made that RAID is NOT a backup. I once had a RAID
controller die on me and there were 4 drives attached to it. Lost ALL
the data on those drives even through all of teh drives themseves were
not faulty.
Many thanks Jaimie & SH, food for thought as always :-)
I went ahead with RAID 6 earlier today and now the data is kept as
follows (or will be when it's all in place):
Original : Z620 NVMe
Backup 1: Z620 SSD
Backup 2: QNAP TS451 (RAID 6) also stream media from this
Backup 3: QNAP TS431 (RAID 6)
Backup4: USB connected 4 TB HD (laptop size) - my "grab it and run" drive.
No external backup but I am wary of the "Cloud" since Dropbox was
hacked, I probably ought to try and do something about this.
I have to admit using exFAT on the Z620, it's refreshing that it no
longer whines about permissions, I will re-think that once all is
working though!
Thanks again for input.
No external backup but I am wary of the "Cloud" since Dropbox was hacked,
I probably ought to try and do something about this.
one further point...
Are the original and the 4 backups geolocated in the same location?
What happens if there is a gas blast, flooding, house fire etc.... all
your original and 4 backups are GONE!
Have some of them OFF-SITE and connected via the internet so you can do >backups online to save you tripping between the two locations
On 8 Jul 2023 at 11:09:18 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
wrote:
No external backup but I am wary of the "Cloud" since Dropbox was hacked,
I probably ought to try and do something about this.
There's no the cloud, just other people's computers. Dropbox fucked
up.
Backblaze encrypts at your end and only passes encrypted backups to
their datacentre. They're more tuned to local storage than remote
though; their cheap product is "all you can eat *on one personal
computer*" which doesn't include NASes :) For that you need to check out >volume-based pricing.
On 09/07/2023 in message <kgub5pFe6orU1@mid.individual.net> Jaimie Vandenbergh wrote:
On 8 Jul 2023 at 11:09:18 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
wrote:
No external backup but I am wary of the "Cloud" since Dropbox was
hacked,
I probably ought to try and do something about this.
There's no the cloud, just other people's computers. Dropbox fucked
up.
Yes but the Dropbox screw up shows the vulnerability of the "Cloud" generally.
Backblaze encrypts at your end and only passes encrypted backups to
their datacentre. They're more tuned to local storage than remote
though; their cheap product is "all you can eat *on one personal
computer*" which doesn't include NASes :) For that you need to check out
volume-based pricing.
$7 a month and their speed test shows it would take 79.54 days to upload 3.5TB.
I have a Giganet fibre manhole right outside my house, been there about
3 weeks, don't know when it will go live. Could offer better upload speeds.
I used to be with Virginmedia..... I had 300 Mb/s download and just 30
Mb/s upload.
Mow switched to Vodafone FTTH, now get 500 Mb/s BOTH ways and I can
upgrade to 900 Mb/s BOTH ways if I wish.
Its worth checking what the cloud bandwidth is like as Microsfot Onedrive >does not use the full FTTH fibre bandwidth :-( so when opening Camera
roll, it takes ages to update between the cloud copy and the local copy
Jeff Gaines wrote:
Its worth checking what the cloud bandwidth is like as Microsfot Onedrive >>does not use the full FTTH fibre bandwidth
When copying one humungous file, or thousands of tiddlers?
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
Its worth checking what the cloud bandwidth is like as Microsfot
Onedrive does not use the full FTTH fibre bandwidth
In article <xn0o42lmh25zx53004@news.individual.net>, Jeff Gaines wrote...
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost, >>can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is >>fine for me. does that sound like a good plan?
It seems to me the RAID is often adopted for reasons that don't quite fit
the
situation. People (reasonably) want to "protect" their data, but the
various
flavours of RAID are really to enhance performance and/or resilience. >Resilience is about keeping a system going (or restoring it) with minimal >downtime. For many of us here, a bit of downtime isn't as big a deal as it >would be if a business had to suspend transaction processing, for example.
And
as has been pointed out, RAID has its own risks, and costs. Mirroring (if >monitored) can give you early warning of disk failure, but a decent SMART >monitor can give you much earlier warning that this might happen. And a >backup
strategy matched sensibly to the significant risks (including, say, fire,
but
perhaps not comet impact)is worth more, if losing data altogether (rather >than
time) is your main concern.
Microsfot Onedrive does not use the full FTTH fibre bandwidth 🙁 so when opening Camera roll, it takes ages to update between the cloud copy and
the local copy
On 09/07/2023 in message <MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
wrote:
In article <xn0o42lmh25zx53004@news.individual.net>, Jeff Gaines wrote... >>>
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
It seems to me the RAID is often adopted for reasons that don't quite
fit the
situation. People (reasonably) want to "protect" their data, but the
various
flavours of RAID are really to enhance performance and/or resilience.
Resilience is about keeping a system going (or restoring it) with minimal
downtime. For many of us here, a bit of downtime isn't as big a deal
as it
would be if a business had to suspend transaction processing, for
example. And
as has been pointed out, RAID has its own risks, and costs. Mirroring
(if
monitored) can give you early warning of disk failure, but a decent SMART
monitor can give you much earlier warning that this might happen. And
a backup
strategy matched sensibly to the significant risks (including, say,
fire, but
perhaps not comet impact)is worth more, if losing data altogether
(rather than
time) is your main concern.
I tend to agree. Many moons ago I had a small PC that looked like a NAS
in that it had a row of drive carriers on the front. I used it as a home server but it had power supply problems and so wasn't reliable, can't remember the make. I just used NTFS on the drives which were standalone.
I use RAID6 on the NAS because it means two drives can fail before I
lose data, I appreciate RAID isn't a back up but my 2 x NAS are!
On 09/07/2023 09:53, SH wrote:
Microsfot Onedrive does not use the full FTTH fibre bandwidth 🙁 so
when opening Camera roll, it takes ages to update between the cloud
copy and the local copy
can you please explain what you mean by "Microsfot Onedrive does not use
the full FTTH fibre bandwidth"
and tell me how MS can tell what the delivery medium is at the end of
the line ?
On 09/07/2023 14:46, Jeff Gaines wrote:
On 09/07/2023 in message
<MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
wrote:
I tend to agree. Many moons ago I had a small PC that looked like a NAS
in that it had a row of drive carriers on the front. I used it as a home
server but it had power supply problems and so wasn't reliable, can't
remember the make. I just used NTFS on the drives which were standalone.
I use RAID6 on the NAS because it means two drives can fail before I
lose data, I appreciate RAID isn't a back up but my 2 x NAS are!
For 4 drives, both RAID 6 and RAID 10 means you have half the total
drive capacity. but there is no parity calculation overheard with 10
like there is with 6.
When syncing a 1TB drive to Onedrive, the upload speed is LOWER than my
500 Mb/s fibre service
Ditto when Syncing onedrive's contents to a brand new SSD, the download
speed is lower than my 500 Mb's service.
Testing mt fibre with both Speedtest and fast.com shows that my fibre
really can do 500 Mb/s BOTH ways.
On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
wrote:
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
Not necessarily the system, just that the four drives are all being used
as one concatenated big one. If quarter of your big drive dies, you do
indeed lose everything.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
Yep.
If there is going to be data on here that is *only* on here, remember
that RAID is not anything remotely like backup, so you should plan to
have another 4tb somewhere (or larger) that you can run backups of this
array off to, to prevent issues with accidental deletion or the chassis
dying or whatever.
You'll need another 4TB anyway to copy everything off and back on again.
Cheers - Jaimie
On 2023-07-08, Jaimie Vandenbergh <jaimie@usually.sessile.org> wrote:
On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
wrote:
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
Not necessarily the system, just that the four drives are all being used
as one concatenated big one. If quarter of your big drive dies, you do
indeed lose everything.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
Yep.
If there is going to be data on here that is *only* on here, remember
that RAID is not anything remotely like backup, so you should plan to
have another 4tb somewhere (or larger) that you can run backups of this
array off to, to prevent issues with accidental deletion or the chassis
dying or whatever.
You'll need another 4TB anyway to copy everything off and back on again.
Cheers - Jaimie
There is also the point to remember that is one HD fails the chances are
high that another will fail sooner rather than later. Rebuilding the array places a stress on the remaining disks.
As you need a backup of the data at any rate the RAID number will only come into play if down time is of importance, cost or time.
As pointed out RAID is not a backup.
On 9 Jul 2023 at 17:59:19 BST, "SH" <i.love@spam.com> wrote:
On 09/07/2023 14:46, Jeff Gaines wrote:
On 09/07/2023 in message
<MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
wrote:
I tend to agree. Many moons ago I had a small PC that looked like a NAS
in that it had a row of drive carriers on the front. I used it as a home >>> server but it had power supply problems and so wasn't reliable, can't
remember the make. I just used NTFS on the drives which were standalone. >>>
I use RAID6 on the NAS because it means two drives can fail before I
lose data, I appreciate RAID isn't a back up but my 2 x NAS are!
For 4 drives, both RAID 6 and RAID 10 means you have half the total
drive capacity. but there is no parity calculation overheard with 10
like there is with 6.
While true, parity is basically negligible unless you're running on a 20
year old CPU or minimal (half gig) RAM in the NAS. Even if it is, it
really only affects rebuild speed after a drive fail.
A RAID10 can only safely have one drive die; a second failed drive might
be the mirror of the first, which would kill the array - you'd need do
to an offline restore from backup after replacing the deads. Very very
slow, no availability.
I wouldn't generally bother with RAID10 unless performance was more
important than redundancy and resilience and uptime - which it often is
for commercial use. For home though... well, since my own NAS with four drives in RAID5 (or the ZFS equivalent, RAIDZ1) can fill a 10gigE
network pipe, turns out you can get plenty enough performance from
RAID5. It's a 2014 Dell server chassis (R420) that you can buy for about £300 plus drives.
Cheers - Jaimie
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost,
can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
fine for me. does that sound like a good plan?
On 07/07/2023 in message <xn0o42lmh25zx53004@news.individual.net> Jeff
Gaines wrote:
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is lost, >can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is >fine for me. does that sound like a good plan?
Can I come back on this as it there have been some interesting comments
and I'd like to try and be clear.
I think I understand the statement "RAID is not a substitute for a back
up" - don't use RAID on a PC instead of making proper back ups. Is that correct?
Assuming it is then having original data plus a back up on the computer
then 2 x back ups to 2 separate NAS's plus 1 x backup to an external USB connected (spinning) drive seems to overcome that objection.
When it comes to the NAS I can:
Run it as 4 x separate drives (I'm not sure this is JBOD as they are
entirely independent of each other) when I will lose data if the system
drive fails. This means the drives will be formatted to whatever the NAS
uses so I can't pull them out of the NAS and read them by putting them in
a Windows PC.
Run some version of RAID. RAID 6 having the advantage that it can tolerate
2 x drives failing. Does that depend on which 2 drives or is it any two drives? Still can't pull them out of the NAS and read them by putting them
in a Windows PC.
Alternatively I could set up a Windows box as a server with lots of drives running as individual drives that could be pulled out and read on another Windows box. To me there would be no benefit in using a Linux box over
using a NAS.
Do I seem to be following OK?
I am getting itchy fingers and thinking about another visit to Bargain Hardware so I need to get myself under control!
I'm not sure this is JBOD as they are entirely independent of each other
On 07/07/2023 in message <xn0o42lmh25zx53004@news.individual.net> Jeff
Gaines wrote:
I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
that means the system is on one of the drives if that goes all is
lost, can't so far find any confirmation.
RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB
is fine for me. does that sound like a good plan?
Can I come back on this as it there have been some interesting comments
and I'd like to try and be clear.
I think I understand the statement "RAID is not a substitute for a back
up" - don't use RAID on a PC instead of making proper back ups. Is that correct?
Assuming it is then having original data plus a back up on the computer
then 2 x back ups to 2 separate NAS's plus 1 x backup to an external
USB connected (spinning) drive seems to overcome that objection.
When it comes to the NAS I can:
Run it as 4 x separate drives (I'm not sure this is JBOD as they are
entirely independent of each other) when I will lose data if the system
drive fails. This means the drives will be formatted to whatever the NAS
uses so I can't pull them out of the NAS and read them by putting them
in a Windows PC.
Run some version of RAID. RAID 6 having the advantage that it can
tolerate 2 x drives failing. Does that depend on which 2 drives or is it
any two drives? Still can't pull them out of the NAS and read them by
putting them in a Windows PC.
Alternatively I could set up a Windows box as a server with lots of
drives running as individual drives that could be pulled out and read on another Windows box. To me there would be no benefit in using a Linux
box over using a NAS.
Do I seem to be following OK?
I am getting itchy fingers and thinking about another visit to Bargain Hardware so I need to get myself under control!
Do I seem to be following OK?
I am getting itchy fingers and thinking about another visit to Bargain >>Hardware so I need to get myself under control!
I am wondering exactly what you want to achieve. Unless performance or
real-
time resilience are particularly important, you have data on your working >machine, and (presumably) a properly versioned backup on your NAS. (You >don't
mention a cloud or offsite copy - three copies is the gold standard.)
What's
missing?
In your case, it sounds like you've built your own NAS so you've had a
freer and wider choice of what hardware you're using plus you're
probably usign Freenas/Truenas/Nas4Free/OMV etc.
So its no surprise your
home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
you're using SSDs rather than HDDs though? :-)
So its no surprise your
home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
you're using SSDs rather than HDDs though? :-)
Surprisingly no, it's using four 14TB WD Reds (shucked out of WD My Book externals) and the bit density is high enough to keep the RAM cache well
fed and it'll happily push data around at 10gigabit/sec around for as
long as I've tried, three TB in one go I think.
I have not yet found out how quickly it resilvers one of those 14TB drives.... the previous iteration was 8x4TB RAIDZ2 (RAID6ish) and would
take 14ish hours to replace one disk.
Cheers - Jaimie
So its no surprise your
home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
you're using SSDs rather than HDDs though? :-)
Surprisingly no, it's using four 14TB WD Reds (shucked out of WD My Book
externals) and the bit density is high enough to keep the RAM cache well
fed and it'll happily push data around at 10gigabit/sec around for as
long as I've tried, three TB in one go I think.
I have not yet found out how quickly it resilvers one of those 14TB
drives.... the previous iteration was 8x4TB RAIDZ2 (RAID6ish) and would
take 14ish hours to replace one disk.
Cheers - Jaimie
Thats surprising.... quick calculation:
Sata3 is able to do 600 MB/s which is about 4.8 Gbit/sec
So 4 drives and assume 1/4 is lost to parity (hardware RAID or software
ZFS!)
so 3 x 4.8 Gbit/sec is 14.4 Gbit/sec
The hardest part is actually believing a spinning disc of rust can
actually sustain 600 MB/sec...
Look up Bathtub curve!
Some peopel routinely replace all the NAS drives onces they get to a
certain age or certain number of running hours.....
S.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 415 |
Nodes: | 16 (2 / 14) |
Uptime: | 34:55:26 |
Calls: | 8,720 |
Calls today: | 3 |
Files: | 13,276 |
Messages: | 5,956,103 |