• RAID Arrays

    From Jeff Gaines@21:1/5 to All on Fri Jul 7 22:47:52 2023
    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    --
    Jeff Gaines Dorset UK
    There are 10 types of people in the world, those who do binary and those
    who don't.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jaimie Vandenbergh@21:1/5 to All on Sat Jul 8 01:12:00 2023
    On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    Not necessarily the system, just that the four drives are all being used
    as one concatenated big one. If quarter of your big drive dies, you do
    indeed lose everything.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    Yep.

    If there is going to be data on here that is *only* on here, remember
    that RAID is not anything remotely like backup, so you should plan to
    have another 4tb somewhere (or larger) that you can run backups of this
    array off to, to prevent issues with accidental deletion or the chassis
    dying or whatever.

    You'll need another 4TB anyway to copy everything off and back on again.

    Cheers - Jaimie
    --
    Hell hath no fury like someone who is enraged that
    someone else is getting away with something they're
    scared to try. - lilairen, LJ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to u8baov$1mvcr$1@dont-email.me on Sat Jul 8 10:09:18 2023
    On 08/07/2023 in message <u8baov$1mvcr$1@dont-email.me> SH wrote:

    Echo the point made that RAID is NOT a backup. I once had a RAID
    controller die on me and there were 4 drives attached to it. Lost ALL the >data on those drives even through all of teh drives themseves were not >faulty.

    Many thanks Jaimie & SH, food for thought as always :-)

    I went ahead with RAID 6 earlier today and now the data is kept as follows
    (or will be when it's all in place):

    Original : Z620 NVMe
    Backup 1: Z620 SSD
    Backup 2: QNAP TS451 (RAID 6) also stream media from this
    Backup 3: QNAP TS431 (RAID 6)
    Backup4: USB connected 4 TB HD (laptop size) - my "grab it and run" drive.

    No external backup but I am wary of the "Cloud" since Dropbox was hacked,
    I probably ought to try and do something about this.

    I have to admit using exFAT on the Z620, it's refreshing that it no longer whines about permissions, I will re-think that once all is working though!

    Thanks again for input.

    --
    Jeff Gaines Dorset UK
    I can please only one person per day. Today is not your day.
    Tomorrow, isn't looking good either.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Jaimie Vandenbergh on Sat Jul 8 10:37:33 2023
    On 08/07/2023 02:12, Jaimie Vandenbergh wrote:
    On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    Not necessarily the system, just that the four drives are all being used
    as one concatenated big one. If quarter of your big drive dies, you do
    indeed lose everything.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    Yep.

    If there is going to be data on here that is *only* on here, remember
    that RAID is not anything remotely like backup, so you should plan to
    have another 4tb somewhere (or larger) that you can run backups of this
    array off to, to prevent issues with accidental deletion or the chassis
    dying or whatever.

    You'll need another 4TB anyway to copy everything off and back on again.

    Cheers - Jaimie


    and also consider the CPU load caused by the NAS having to compute the
    two sets of parity data for RAID 6.

    RAID 5 involves the computing of one set of parity data but this will
    only accommodate 1 drive failure instead of 2

    NAS CPU's tend to be on the low power side and also not close to the
    specs of todays desktops

    You might want to consider a RAID 10 array which does not involve parity calculations on the CPU, its essentially a mirrored AND a Striped array.
    So those 4 x 2TB drives would then become a single 4TB array.

    Echo the point made that RAID is NOT a backup. I once had a RAID
    controller die on me and there were 4 drives attached to it. Lost ALL
    the data on those drives even through all of teh drives themseves were
    not faulty.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Jeff Gaines on Sat Jul 8 11:46:31 2023
    On 08/07/2023 11:09, Jeff Gaines wrote:
    On 08/07/2023 in message <u8baov$1mvcr$1@dont-email.me> SH wrote:

    Echo the point made that RAID is NOT a backup. I once had a RAID
    controller die on me and there were 4 drives attached to it. Lost ALL
    the data on those drives even through all of teh drives themseves were
    not faulty.

    Many thanks Jaimie & SH, food for thought as always :-)

    I went ahead with RAID 6 earlier today and now the data is kept as
    follows (or will be when it's all in place):

    Original : Z620 NVMe
    Backup 1: Z620 SSD
    Backup 2: QNAP TS451 (RAID 6) also stream media from this
    Backup 3: QNAP TS431 (RAID 6)
    Backup4: USB connected 4 TB HD (laptop size) - my "grab it and run" drive.

    No external backup but I am wary of the "Cloud" since Dropbox was
    hacked, I probably ought to try and do something about this.

    I have to admit using exFAT on the Z620, it's refreshing that it no
    longer whines about permissions, I will re-think that once all is
    working though!

    Thanks again for input.



    one further point...

    Are the original and the 4 backups geolocated in the same location?

    What happens if there is a gas blast, flooding, house fire etc.... all
    your original and 4 backups are GONE!

    Have some of them OFF-SITE and connected via the internet so you can do
    backups online to save you tripping between the two locations

    Either do this to the cloud, MS offer an encrypted folder if your data
    is sensitive and will require a password to access, and if you forget
    the password, MS cannot recover the data.

    There is also googledrive, backblaze etc.

    you are right to be worried/concerned about hacking, but if you can set
    up a strong password, two factor authentication with either a smart
    phone or a yubikey etc and ENCRYPT while the data is in flight AND is
    ENCRYPTED at REST at teh remote location you should be pretty OK.

    Or set up wireguard at both ends with static IPs and then you should be
    able to rsync/timemachine/robocopy between the two locations. Teh
    latter will require a co-operative friend/family member and fast
    broadband both ways, preferably fibre to the home.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jaimie Vandenbergh@21:1/5 to All on Sat Jul 8 23:52:25 2023
    On 8 Jul 2023 at 11:09:18 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:

    No external backup but I am wary of the "Cloud" since Dropbox was hacked,
    I probably ought to try and do something about this.

    There's no *the* cloud, just other people's computers. Dropbox fucked
    up.

    Backblaze encrypts at your end and only passes encrypted backups to
    their datacentre. They're more tuned to local storage than remote
    though; their cheap product is "all you can eat *on one personal
    computer*" which doesn't include NASes :) For that you need to check out volume-based pricing.

    Cheers - Jaimie
    --
    Is everyone acting like a solipsist in here, or is it just me?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to u8beq8$1mvcr$2@dont-email.me on Sun Jul 9 07:35:26 2023
    On 08/07/2023 in message <u8beq8$1mvcr$2@dont-email.me> SH wrote:

    one further point...

    Are the original and the 4 backups geolocated in the same location?

    What happens if there is a gas blast, flooding, house fire etc.... all
    your original and 4 backups are GONE!

    Have some of them OFF-SITE and connected via the internet so you can do >backups online to save you tripping between the two locations

    They all sit together on the same desk!

    As I said that is a vulnerability that I need to think about.

    --
    Jeff Gaines Dorset UK
    I've been through the desert on a horse with no name.
    It was a right bugger to get him back when he ran off.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to Vandenbergh on Sun Jul 9 07:39:01 2023
    On 09/07/2023 in message <kgub5pFe6orU1@mid.individual.net> Jaimie
    Vandenbergh wrote:

    On 8 Jul 2023 at 11:09:18 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:

    No external backup but I am wary of the "Cloud" since Dropbox was hacked,
    I probably ought to try and do something about this.

    There's no the cloud, just other people's computers. Dropbox fucked
    up.

    Yes but the Dropbox screw up shows the vulnerability of the "Cloud"
    generally.


    Backblaze encrypts at your end and only passes encrypted backups to
    their datacentre. They're more tuned to local storage than remote
    though; their cheap product is "all you can eat *on one personal
    computer*" which doesn't include NASes :) For that you need to check out >volume-based pricing.

    $7 a month and their speed test shows it would take 79.54 days to upload
    3.5TB.

    I have a Giganet fibre manhole right outside my house, been there about 3 weeks, don't know when it will go live. Could offer better upload speeds.

    --
    Jeff Gaines Dorset UK
    Remember, the Flat Earth Society has members all around the globe.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Jeff Gaines on Sun Jul 9 09:53:04 2023
    On 09/07/2023 08:39, Jeff Gaines wrote:
    On 09/07/2023 in message <kgub5pFe6orU1@mid.individual.net> Jaimie Vandenbergh wrote:

    On 8 Jul 2023 at 11:09:18 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:

    No external backup but I am wary of the "Cloud" since Dropbox was
    hacked,
    I probably ought to try and do something about this.

    There's no the cloud, just other people's computers. Dropbox fucked
    up.

    Yes but the Dropbox screw up shows the vulnerability of the "Cloud" generally.


    Backblaze encrypts at your end and only passes encrypted backups to
    their datacentre. They're more tuned to local storage than remote
    though; their cheap product is "all you can eat *on one personal
    computer*" which doesn't include NASes :) For that you need to check out
    volume-based pricing.

    $7 a month and their speed test shows it would take 79.54 days to upload 3.5TB.

    I have a Giganet fibre manhole right outside my house, been there about
    3 weeks, don't know when it will go live. Could offer better upload speeds.



    I used to be with Virginmedia..... I had 300 Mb/s download and just 30
    Mb/s upload.

    Mow switched to Vodafone FTTH, now get 500 Mb/s BOTH ways and I can
    upgrade to 900 Mb/s BOTH ways if I wish.

    Its worth checking what the cloud bandwidth is like as Microsfot
    Onedrive does not use the full FTTH fibre bandwidth :-( so when opening
    Camera roll, it takes ages to update between the cloud copy and the
    local copy

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to u8dshh$23c99$1@dont-email.me on Sun Jul 9 09:58:09 2023
    On 09/07/2023 in message <u8dshh$23c99$1@dont-email.me> SH wrote:

    I used to be with Virginmedia..... I had 300 Mb/s download and just 30
    Mb/s upload.

    Mow switched to Vodafone FTTH, now get 500 Mb/s BOTH ways and I can
    upgrade to 900 Mb/s BOTH ways if I wish.

    Its worth checking what the cloud bandwidth is like as Microsfot Onedrive >does not use the full FTTH fibre bandwidth :-( so when opening Camera
    roll, it takes ages to update between the cloud copy and the local copy

    That's interesting, thank you :-)

    I know what to ask Giganet when they start selling!

    --
    Jeff Gaines Dorset UK
    Those are my principles – and if you don’t like them, well, I have
    others.
    (Groucho Marx)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to All on Sun Jul 9 10:53:00 2023
    On 09/07/2023 in message <kgvgqhFjpejU2@mid.individual.net> Andy Burns
    wrote:

    Jeff Gaines wrote:

    Its worth checking what the cloud bandwidth is like as Microsfot Onedrive >>does not use the full FTTH fibre bandwidth

    When copying one humungous file, or thousands of tiddlers?

    I didn't, SH did, but it's a good point! I have FS2002 installed which has 5,013 scenery files < 4 KB.

    --
    Jeff Gaines Dorset UK
    There are 10 types of people in the world, those who do binary and those
    who don't.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Philip Herlihy@21:1/5 to All on Sun Jul 9 11:56:55 2023
    In article <xn0o42lmh25zx53004@news.individual.net>, Jeff Gaines wrote...

    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    It seems to me the RAID is often adopted for reasons that don't quite fit the situation. People (reasonably) want to "protect" their data, but the various flavours of RAID are really to enhance performance and/or resilience. Resilience is about keeping a system going (or restoring it) with minimal downtime. For many of us here, a bit of downtime isn't as big a deal as it would be if a business had to suspend transaction processing, for example. And as has been pointed out, RAID has its own risks, and costs. Mirroring (if monitored) can give you early warning of disk failure, but a decent SMART monitor can give you much earlier warning that this might happen. And a backup strategy matched sensibly to the significant risks (including, say, fire, but perhaps not comet impact)is worth more, if losing data altogether (rather than time) is your main concern.

    --

    Phil, London

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andy Burns@21:1/5 to Jeff Gaines on Sun Jul 9 11:35:00 2023
    Jeff Gaines wrote:

    Its worth checking what the cloud bandwidth is like as Microsfot
    Onedrive does not use the full FTTH fibre bandwidth

    When copying one humungous file, or thousands of tiddlers?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to All on Sun Jul 9 13:46:36 2023
    On 09/07/2023 in message <MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
    wrote:

    In article <xn0o42lmh25zx53004@news.individual.net>, Jeff Gaines wrote...

    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost, >>can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is >>fine for me. does that sound like a good plan?

    It seems to me the RAID is often adopted for reasons that don't quite fit
    the
    situation. People (reasonably) want to "protect" their data, but the
    various
    flavours of RAID are really to enhance performance and/or resilience. >Resilience is about keeping a system going (or restoring it) with minimal >downtime. For many of us here, a bit of downtime isn't as big a deal as it >would be if a business had to suspend transaction processing, for example.
    And
    as has been pointed out, RAID has its own risks, and costs. Mirroring (if >monitored) can give you early warning of disk failure, but a decent SMART >monitor can give you much earlier warning that this might happen. And a >backup
    strategy matched sensibly to the significant risks (including, say, fire,
    but
    perhaps not comet impact)is worth more, if losing data altogether (rather >than
    time) is your main concern.

    I tend to agree. Many moons ago I had a small PC that looked like a NAS in
    that it had a row of drive carriers on the front. I used it as a home
    server but it had power supply problems and so wasn't reliable, can't
    remember the make. I just used NTFS on the drives which were standalone.

    I use RAID6 on the NAS because it means two drives can fail before I lose
    data, I appreciate RAID isn't a back up but my 2 x NAS are!

    --
    Jeff Gaines Dorset UK
    By the time you can make ends meet they move the ends

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Abandoned_Trolley@21:1/5 to All on Sun Jul 9 16:44:31 2023
    On 09/07/2023 09:53, SH wrote:
    Microsfot Onedrive does not use the full FTTH fibre bandwidth 🙁 so when opening Camera roll, it takes ages to update between the cloud copy and
    the local copy



    can you please explain what you mean by "Microsfot Onedrive does not use
    the full FTTH fibre bandwidth"

    and tell me how MS can tell what the delivery medium is at the end of
    the line ?


    --
    random signature text inserted here

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Jeff Gaines on Sun Jul 9 17:59:19 2023
    On 09/07/2023 14:46, Jeff Gaines wrote:
    On 09/07/2023 in message <MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
    wrote:

    In article <xn0o42lmh25zx53004@news.individual.net>, Jeff Gaines wrote... >>>
    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    It seems to me the RAID is often adopted for reasons that don't quite
    fit the
    situation.  People (reasonably) want to "protect" their data, but the
    various
    flavours of RAID are really to enhance performance and/or resilience.
    Resilience is about keeping a system going (or restoring it) with minimal
    downtime.  For many of us here, a bit of downtime isn't as big a deal
    as it
    would be if a business had to suspend transaction processing, for
    example.  And
    as has been pointed out, RAID has its own risks, and costs.  Mirroring
    (if
    monitored) can give you early warning of disk failure, but a decent SMART
    monitor can give you much earlier warning that this might happen.  And
    a backup
    strategy matched sensibly to the significant risks (including, say,
    fire, but
    perhaps not comet impact)is worth more, if losing data altogether
    (rather than
    time) is your main concern.

    I tend to agree. Many moons ago I had a small PC that looked like a NAS
    in that it had a row of drive carriers on the front. I used it as a home server but it had power supply problems and so wasn't reliable, can't remember the make. I just used NTFS on the drives which were standalone.

    I use RAID6 on the NAS because it means two drives can fail before I
    lose data, I appreciate RAID isn't a back up but my 2 x NAS are!


    For 4 drives, both RAID 6 and RAID 10 means you have half the total
    drive capacity. but there is no parity calculation overheard with 10
    like there is with 6.

    So for a 4 drive set up, you'd actually gain going to RAID 10 rather
    than RAID 6 as then the CPU inyour NAS is then freed up to do other stuff.

    (For RAID 6, you lose two hard drives, for RAID 10, you lose half the
    drives. so 4 drives is the mnimum no of drives and the breakeven point
    of the two.

    Obviousl if yoo have more than 6 drives, then RAID 10 is rather wasteful
    of drive space than RAID 6 is.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to All on Sun Jul 9 17:55:55 2023
    On 09/07/2023 16:44, Abandoned_Trolley wrote:
    On 09/07/2023 09:53, SH wrote:
    Microsfot Onedrive does not use the full FTTH fibre bandwidth 🙁 so
    when opening Camera roll, it takes ages to update between the cloud
    copy and the local copy



    can you please explain what you mean by "Microsfot Onedrive does not use
    the full FTTH fibre bandwidth"

    and tell me how MS can tell what the delivery medium is at the end of
    the line ?




    When syncing a 1TB drive to Onedrive, the upload speed is LOWER than my
    500 Mb/s fibre service

    Ditto when Syncing onedrive's contents to a brand new SSD, the download
    speed is lower than my 500 Mb's service.

    Testing mt fibre with both Speedtest and fast.com shows that my fibre
    really can do 500 Mb/s BOTH ways.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jaimie Vandenbergh@21:1/5 to i.love@spam.com on Sun Jul 9 20:02:22 2023
    On 9 Jul 2023 at 17:59:19 BST, "SH" <i.love@spam.com> wrote:

    On 09/07/2023 14:46, Jeff Gaines wrote:
    On 09/07/2023 in message
    <MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
    wrote:

    I tend to agree. Many moons ago I had a small PC that looked like a NAS
    in that it had a row of drive carriers on the front. I used it as a home
    server but it had power supply problems and so wasn't reliable, can't
    remember the make. I just used NTFS on the drives which were standalone.

    I use RAID6 on the NAS because it means two drives can fail before I
    lose data, I appreciate RAID isn't a back up but my 2 x NAS are!


    For 4 drives, both RAID 6 and RAID 10 means you have half the total
    drive capacity. but there is no parity calculation overheard with 10
    like there is with 6.

    While true, parity is basically negligible unless you're running on a 20
    year old CPU or minimal (half gig) RAM in the NAS. Even if it is, it
    really only affects rebuild speed after a drive fail.

    A RAID10 can only safely have one drive die; a second failed drive might
    be the mirror of the first, which would kill the array - you'd need do
    to an offline restore from backup after replacing the deads. Very very
    slow, no availability.

    I wouldn't generally bother with RAID10 unless performance was more
    important than redundancy and resilience and uptime - which it often is
    for commercial use. For home though... well, since my own NAS with four
    drives in RAID5 (or the ZFS equivalent, RAIDZ1) can fill a 10gigE
    network pipe, turns out you can get plenty enough performance from
    RAID5. It's a 2014 Dell server chassis (R420) that you can buy for about
    £300 plus drives.

    Cheers - Jaimie
    --
    The Daily Mail should be forced to print the words
    'The Paper That Supported Hitler' on its masthead,
    just so that there is something that's true on the
    front page every day. -- Mark Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Abandoned_Trolley@21:1/5 to All on Mon Jul 10 10:14:04 2023
    When syncing a 1TB drive to Onedrive, the upload speed is LOWER than my
    500 Mb/s fibre service

    Ditto when Syncing onedrive's contents to a brand new SSD, the download
    speed is lower than my 500 Mb's service.

    Testing mt fibre with both Speedtest and fast.com shows that my fibre
    really can do 500 Mb/s BOTH ways.


    Which probably means that one of the links on the route has either
    reached its capacity limit, or is being deliberately throttled - and 500
    Mb/S is quite a lot for a single user

    I dont see any conclusive proof that its MS doing the throttling



    --
    random signature text inserted here

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Gordon@21:1/5 to Jaimie Vandenbergh on Mon Jul 10 08:19:51 2023
    On 2023-07-08, Jaimie Vandenbergh <jaimie@usually.sessile.org> wrote:
    On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    Not necessarily the system, just that the four drives are all being used
    as one concatenated big one. If quarter of your big drive dies, you do
    indeed lose everything.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    Yep.

    If there is going to be data on here that is *only* on here, remember
    that RAID is not anything remotely like backup, so you should plan to
    have another 4tb somewhere (or larger) that you can run backups of this
    array off to, to prevent issues with accidental deletion or the chassis
    dying or whatever.

    You'll need another 4TB anyway to copy everything off and back on again.

    Cheers - Jaimie

    There is also the point to remember that is one HD fails the chances are
    high that another will fail sooner rather than later. Rebuilding the array places a stress on the remaining disks.

    As you need a backup of the data at any rate the RAID number will only come into play if down time is of importance, cost or time.

    As pointed out RAID is not a backup.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Gordon on Mon Jul 10 09:25:28 2023
    On 10/07/2023 09:19, Gordon wrote:
    On 2023-07-08, Jaimie Vandenbergh <jaimie@usually.sessile.org> wrote:
    On 7 Jul 2023 at 23:47:52 BST, ""Jeff Gaines"" <jgnewsid@outlook.com>
    wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    Not necessarily the system, just that the four drives are all being used
    as one concatenated big one. If quarter of your big drive dies, you do
    indeed lose everything.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?

    Yep.

    If there is going to be data on here that is *only* on here, remember
    that RAID is not anything remotely like backup, so you should plan to
    have another 4tb somewhere (or larger) that you can run backups of this
    array off to, to prevent issues with accidental deletion or the chassis
    dying or whatever.

    You'll need another 4TB anyway to copy everything off and back on again.

    Cheers - Jaimie

    There is also the point to remember that is one HD fails the chances are
    high that another will fail sooner rather than later. Rebuilding the array places a stress on the remaining disks.

    As you need a backup of the data at any rate the RAID number will only come into play if down time is of importance, cost or time.

    As pointed out RAID is not a backup.


    yes, totally agreed with!

    particularly if all the HDDs in use are all the same date of
    manufacture, same make, same model and have the same number of hours
    running time.

    Look up Bathtub curve!

    Some peopel routinely replace all the NAS drives onces they get to a
    certain age or certain number of running hours.....

    S.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Jaimie Vandenbergh on Mon Jul 10 09:23:10 2023
    On 09/07/2023 21:02, Jaimie Vandenbergh wrote:
    On 9 Jul 2023 at 17:59:19 BST, "SH" <i.love@spam.com> wrote:

    On 09/07/2023 14:46, Jeff Gaines wrote:
    On 09/07/2023 in message
    <MPG.3f1357a6a63bb1b5989a70@news.eternal-september.org> Philip Herlihy
    wrote:

    I tend to agree. Many moons ago I had a small PC that looked like a NAS
    in that it had a row of drive carriers on the front. I used it as a home >>> server but it had power supply problems and so wasn't reliable, can't
    remember the make. I just used NTFS on the drives which were standalone. >>>
    I use RAID6 on the NAS because it means two drives can fail before I
    lose data, I appreciate RAID isn't a back up but my 2 x NAS are!


    For 4 drives, both RAID 6 and RAID 10 means you have half the total
    drive capacity. but there is no parity calculation overheard with 10
    like there is with 6.

    While true, parity is basically negligible unless you're running on a 20
    year old CPU or minimal (half gig) RAM in the NAS. Even if it is, it
    really only affects rebuild speed after a drive fail.

    A RAID10 can only safely have one drive die; a second failed drive might
    be the mirror of the first, which would kill the array - you'd need do
    to an offline restore from backup after replacing the deads. Very very
    slow, no availability.

    I wouldn't generally bother with RAID10 unless performance was more
    important than redundancy and resilience and uptime - which it often is
    for commercial use. For home though... well, since my own NAS with four drives in RAID5 (or the ZFS equivalent, RAIDZ1) can fill a 10gigE
    network pipe, turns out you can get plenty enough performance from
    RAID5. It's a 2014 Dell server chassis (R420) that you can buy for about £300 plus drives.

    Cheers - Jaimie


    I was thinking more of consumer grade NAS units like Synology, Buffalo, AsusStor etc as they are well known for using low power CPUs,
    particularly celeron or Atom processors and skimp on the memory too.

    If they are haaving to calculate 2 lots of P & Q parity data on the fly,
    that is going to impact on the data transfer rate to & from the RAID
    array and also on other services like Plex, transcoding etc

    In your case, it sounds like you've built your own NAS so you've had a
    freer and wider choice of what hardware you're using plus you're
    probably usign Freenas/Truenas/Nas4Free/OMV etc. So its no surprise your
    home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
    you're using SSDs rather than HDDs though? :-)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to Gaines on Mon Jul 10 10:12:33 2023
    On 07/07/2023 in message <xn0o42lmh25zx53004@news.individual.net> Jeff
    Gaines wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost,
    can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is
    fine for me. does that sound like a good plan?


    Can I come back on this as it there have been some interesting comments
    and I'd like to try and be clear.

    I think I understand the statement "RAID is not a substitute for a back
    up" - don't use RAID on a PC instead of making proper back ups. Is that correct?

    Assuming it is then having original data plus a back up on the computer
    then 2 x back ups to 2 separate NAS's plus 1 x backup to an external USB connected (spinning) drive seems to overcome that objection.

    When it comes to the NAS I can:

    Run it as 4 x separate drives (I'm not sure this is JBOD as they are
    entirely independent of each other) when I will lose data if the system
    drive fails. This means the drives will be formatted to whatever the NAS
    uses so I can't pull them out of the NAS and read them by putting them in
    a Windows PC.

    Run some version of RAID. RAID 6 having the advantage that it can tolerate
    2 x drives failing. Does that depend on which 2 drives or is it any two
    drives? Still can't pull them out of the NAS and read them by putting them
    in a Windows PC.

    Alternatively I could set up a Windows box as a server with lots of drives running as individual drives that could be pulled out and read on another Windows box. To me there would be no benefit in using a Linux box over
    using a NAS.

    Do I seem to be following OK?

    I am getting itchy fingers and thinking about another visit to Bargain
    Hardware so I need to get myself under control!

    --
    Jeff Gaines Dorset UK
    I take full responsibility for what happened - that is why the person that
    was responsible went immediately.
    (Gordon Brown, April 2009)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Philip Herlihy@21:1/5 to All on Mon Jul 10 11:42:09 2023
    In article <xn0o469hs5pc58100c@news.individual.net>, Jeff Gaines wrote...

    On 07/07/2023 in message <xn0o42lmh25zx53004@news.individual.net> Jeff
    Gaines wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is lost, >can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB is >fine for me. does that sound like a good plan?


    Can I come back on this as it there have been some interesting comments
    and I'd like to try and be clear.

    I think I understand the statement "RAID is not a substitute for a back
    up" - don't use RAID on a PC instead of making proper back ups. Is that correct?

    Assuming it is then having original data plus a back up on the computer
    then 2 x back ups to 2 separate NAS's plus 1 x backup to an external USB connected (spinning) drive seems to overcome that objection.

    When it comes to the NAS I can:

    Run it as 4 x separate drives (I'm not sure this is JBOD as they are
    entirely independent of each other) when I will lose data if the system
    drive fails. This means the drives will be formatted to whatever the NAS
    uses so I can't pull them out of the NAS and read them by putting them in
    a Windows PC.

    Run some version of RAID. RAID 6 having the advantage that it can tolerate
    2 x drives failing. Does that depend on which 2 drives or is it any two drives? Still can't pull them out of the NAS and read them by putting them
    in a Windows PC.

    Alternatively I could set up a Windows box as a server with lots of drives running as individual drives that could be pulled out and read on another Windows box. To me there would be no benefit in using a Linux box over
    using a NAS.

    Do I seem to be following OK?

    I am getting itchy fingers and thinking about another visit to Bargain Hardware so I need to get myself under control!

    I am wondering exactly what you want to achieve. Unless performance or real- time resilience are particularly important, you have data on your working machine, and (presumably) a properly versioned backup on your NAS. (You don't mention a cloud or offsite copy - three copies is the gold standard.) What's missing?

    --

    Phil, London

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andy Burns@21:1/5 to Jeff Gaines on Mon Jul 10 11:36:43 2023
    Jeff Gaines wrote:

    I'm not sure this is JBOD as they are entirely independent of each other

    That *is* JBOD ... just a bunch of disks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to Jeff Gaines on Mon Jul 10 12:20:30 2023
    On 10/07/2023 11:12, Jeff Gaines wrote:
    On 07/07/2023 in message <xn0o42lmh25zx53004@news.individual.net> Jeff
    Gaines wrote:


    I have a QNAP TS431 running 4 x 2TB as JBOD. Read somewhere that since
    that means the system is on one of the drives if that goes all is
    lost, can't so far find any confirmation.

    RAID 6 would give me 4 TB usable and 2 drive failure tolerance. 4 TB
    is fine for me. does that sound like a good plan?


    Can I come back on this as it there have been some interesting comments
    and I'd like to try and be clear.

    I think I understand the statement "RAID is not a substitute for a back
    up" - don't use RAID on a PC instead of making proper back ups. Is that correct?

    Assuming it is then having original data plus a back up on the computer
    then 2  x back ups to 2 separate NAS's plus 1 x backup to an external
    USB connected (spinning) drive seems to overcome that objection.

    When it comes to the NAS I can:

    Run it as 4 x separate drives (I'm not sure this is JBOD as they are
    entirely independent of each other) when I will lose data if the system
    drive fails. This means the drives will be formatted to whatever the NAS
    uses so I can't pull them out of the NAS and read them by putting them
    in a Windows PC.

    Run some version of RAID. RAID 6 having the advantage that it can
    tolerate 2 x drives failing. Does that depend on which 2 drives or is it
    any two drives? Still can't pull them out of the NAS and read them by
    putting them in a Windows PC.

    Alternatively I could set up a Windows box as a server with lots of
    drives running as individual drives that could be pulled out and read on another Windows box. To me there would be no benefit in using a Linux
    box over using a NAS.

    Do I seem to be following OK?

    I am getting itchy fingers and thinking about another visit to Bargain Hardware so I need to get myself under control!



    What I have found with many consumer NASes is that they seem to be based
    on Linux and use EXT 4 or EXT 5 or BTRFS as a disc structure.

    DIY NAS software like Freenas, Nas4Free, Truenas and OMV seem to use EXT
    4 or ZFS (and possibly Reiser) and use either Linux of FreeBSD.

    so if you stick to EXT4 and use just single disks, in the event of any
    calamity or hardware failure, just pull the disk and stick it in a PC
    either via SATA or SATA to USB adapter running a Linux Live CD like
    Knoppix or Ubuntu and getting at your data is trivial.

    With Live linux CD's you seem to be the ROOT user so accesing data owned
    by someone else on the HDD/SSD does not seem to be an issue or you could
    use teh CHOWN command in SUDO mode.

    So that means avoiding hardware RAID (but software RAID might be
    different as Linux as a MDM raid driver?) and sticking to a disk
    partiton that is readable by other OS'es

    I think Ubuntu Live can read NTFS partitions, but You may run into
    issues around file ownership and ACLs? (I'm not well versed in this so
    others may want to comment)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeff Gaines@21:1/5 to All on Mon Jul 10 14:19:56 2023
    On 10/07/2023 in message <MPG.3f15f0d3fb1782ae989a71@news.eternal-september.org> Philip Herlihy
    wrote:

    Do I seem to be following OK?

    I am getting itchy fingers and thinking about another visit to Bargain >>Hardware so I need to get myself under control!

    I am wondering exactly what you want to achieve. Unless performance or
    real-
    time resilience are particularly important, you have data on your working >machine, and (presumably) a properly versioned backup on your NAS. (You >don't
    mention a cloud or offsite copy - three copies is the gold standard.)
    What's
    missing?

    I think I am happy with what I have just getting twitchy about comments relating to type of RAID and whether I'm doing the right thing.

    Also haven't built a new PC for a while so have itchy fingers!

    --
    Jeff Gaines Dorset UK
    The world will not be destroyed by those who do evil but by those who
    watch them without doing anything. (Albert Einstein)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jaimie Vandenbergh@21:1/5 to i.love@spam.com on Mon Jul 10 17:07:28 2023
    On 10 Jul 2023 at 09:23:10 BST, "SH" <i.love@spam.com> wrote:

    In your case, it sounds like you've built your own NAS so you've had a
    freer and wider choice of what hardware you're using plus you're
    probably usign Freenas/Truenas/Nas4Free/OMV etc.

    Yup - the aforementioned Dell PowerEdge R420, with a low power Xeon E5
    of some sort and 128gig RAM. Which costs peanuts, it's crazy the specs
    you get for old server kit because no-one wants to buy rackmount stuff.
    It's not even noisy (after, admittedly, a *lot* of experimentation) -
    the hdds are louder than the rest of the machine.

    So its no surprise your
    home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
    you're using SSDs rather than HDDs though? :-)

    Surprisingly no, it's using four 14TB WD Reds (shucked out of WD My Book externals) and the bit density is high enough to keep the RAM cache well
    fed and it'll happily push data around at 10gigabit/sec around for as
    long as I've tried, three TB in one go I think.

    I have not yet found out how quickly it resilvers one of those 14TB
    drives.... the previous iteration was 8x4TB RAIDZ2 (RAID6ish) and would
    take 14ish hours to replace one disk.

    Cheers - Jaimie

    --
    Sent from my Atari 400

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From SH@21:1/5 to All on Mon Jul 10 19:11:07 2023
    So its no surprise your
    home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
    you're using SSDs rather than HDDs though? :-)

    Surprisingly no, it's using four 14TB WD Reds (shucked out of WD My Book externals) and the bit density is high enough to keep the RAM cache well
    fed and it'll happily push data around at 10gigabit/sec around for as
    long as I've tried, three TB in one go I think.

    I have not yet found out how quickly it resilvers one of those 14TB drives.... the previous iteration was 8x4TB RAIDZ2 (RAID6ish) and would
    take 14ish hours to replace one disk.

    Cheers - Jaimie



    Thats surprising.... quick calculation:

    Sata3 is able to do 600 MB/s which is about 4.8 Gbit/sec

    So 4 drives and assume 1/4 is lost to parity (hardware RAID or software
    ZFS!)

    so 3 x 4.8 Gbit/sec is 14.4 Gbit/sec

    The hardest part is actually believing a spinning disc of rust can
    actually sustain 600 MB/sec...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jaimie Vandenbergh@21:1/5 to i.love@spam.com on Mon Jul 10 22:03:00 2023
    On 10 Jul 2023 at 19:11:07 BST, "SH" <i.love@spam.com> wrote:

    So its no surprise your
    home brewed NAS can saturate a 10 GbE pipe, it does rather sound like
    you're using SSDs rather than HDDs though? :-)

    Surprisingly no, it's using four 14TB WD Reds (shucked out of WD My Book
    externals) and the bit density is high enough to keep the RAM cache well
    fed and it'll happily push data around at 10gigabit/sec around for as
    long as I've tried, three TB in one go I think.

    I have not yet found out how quickly it resilvers one of those 14TB
    drives.... the previous iteration was 8x4TB RAIDZ2 (RAID6ish) and would
    take 14ish hours to replace one disk.

    Cheers - Jaimie



    Thats surprising.... quick calculation:

    Sata3 is able to do 600 MB/s which is about 4.8 Gbit/sec

    So 4 drives and assume 1/4 is lost to parity (hardware RAID or software
    ZFS!)

    It's capacity that's lost, not speed.

    so 3 x 4.8 Gbit/sec is 14.4 Gbit/sec

    The hardest part is actually believing a spinning disc of rust can
    actually sustain 600 MB/sec...

    I get over 900megabytes/sec sustained over the 10gigE pipe. Sometimes
    well over, which I presume to be sampling issues :)

    225megabytes/sec each from four drives, buffered in the previously
    mentioned 128gig of RAM cache though... perfectly reasonable.

    Cheers - Jaimie

    --
    "Some people think that noise abatement should be a higher
    priority for ATC. I say safety is noise abatement. You have no
    idea how much noise it makes to have a 737 fall out of the sky
    after an accident."
    -- anonymous air traffic controller

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adrian Caspersz@21:1/5 to All on Tue Jul 11 08:47:02 2023
    On 10/07/2023 09:25, SH wrote:

    Look up Bathtub curve!

    Some peopel routinely replace all the NAS drives onces they get to a
    certain age or certain number of running hours.....

    S.

    My NAS drives seemingly fail about 2 minutes after their warranty runs
    out, and they get replaced sequentially with bigger drives (3TD > 4TB)
    that don't reveal their combined true capacity until I reformat the NAS
    - a job that I had been putting off for years.

    Last week looking at my shelf of bare drives, and some re-certified replacements I had previously received (these of the larger size), a
    final drive swap with one of these would allow online RAID expansion and
    give me almost a whopping 2TB more.

    So I changed over the last small 'un, rather than waiting for its
    perishment - it had been running 5 years, so it had good innings.

    I'll be shopping for a new NAS sometime, think it might be time to plan
    the replacement and find somewhere cooler to locate it. Think I might be cooking these drives.

    --
    Adrian C

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)