• Too many eggs in 1 HD basket?

    From Mike Halmarack@21:1/5 to All on Fri Apr 8 10:52:17 2022
    Until recently I was using a network of one Surface pro 2 and one Dell
    laptop. This along with 3 external USB drives for storage.

    It was Ok using the external drives as mapped network drives with
    fixed drive letters, until I temporarily split up the system to go
    roaming, then changed the storage structures to suit the temporary
    situation. This made too much housekeeping work on return.

    I built a PC and am now intending to use the external USBs for less
    demanding purposes and adding some larger storage HDs to the PC.

    So, I'm considering a 3.5" Western Digital Black Sata drive.
    Amazon has 6, 8, and 10TB drives on offer.
    I'm tempted to go for the 8 or 10TB drive but see that this could risk
    big data loss if things go wrong at some stage.
    Any advice on this please?

    Also, I could vastly reduce my need for storage by tidying up my
    archives and dumping outdated and obsolete files, but this would
    probably take me long time. so increasing storage capacity might be
    the best option.

    --

    Mike

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Philip Herlihy@21:1/5 to All on Fri Apr 8 13:09:43 2022
    In article <n0005htjnedjfqtpqt2doupc0kkh6sa2pl@4ax.com>, Mike Halmarack wrote...

    Until recently I was using a network of one Surface pro 2 and one Dell laptop. This along with 3 external USB drives for storage.

    It was Ok using the external drives as mapped network drives with
    fixed drive letters, until I temporarily split up the system to go
    roaming, then changed the storage structures to suit the temporary situation. This made too much housekeeping work on return.

    I built a PC and am now intending to use the external USBs for less
    demanding purposes and adding some larger storage HDs to the PC.

    So, I'm considering a 3.5" Western Digital Black Sata drive.
    Amazon has 6, 8, and 10TB drives on offer.
    I'm tempted to go for the 8 or 10TB drive but see that this could risk
    big data loss if things go wrong at some stage.
    Any advice on this please?

    Also, I could vastly reduce my need for storage by tidying up my
    archives and dumping outdated and obsolete files, but this would
    probably take me long time. so increasing storage capacity might be
    the best option.

    First step is to tot up how much unique data you have and anticipate: if you have 8TB of files you can't presently dump, then the 6TB disk is a non-starter!

    I can only tell you what I do. My main PC is getting quite elderly now, though it's still pleasingly sprightly and would outlast Windows 10's end-of-support. I have an SSD for the system disk, and another for data - about 100GB of the stuff. Some of that changes daily; some not even annually. Some is personally vital.

    You have to assume that any disk store will eventually fail, so anything of any value has to be verified to be in more than one place at any time. I have a 4TB spinning disk within my case, and I have Windows 10's File History configured to write to it. If I smudge a file, I can simply right-click it and pick from the list of previous versions offered. (Don't fall into the trap of a "backup" system that would copy a damaged file over your only earlier version.) File History - which by default makes copies of anything in a "Library" (named collection of locations, including standard ones like
    "Photos" and "Music") works beautifully - set and forget.

    But of course a missile strike (we are in 2022, after all) would take out the lot, so from the bunker I'd be able to access my most valued stuff via OneDrive (other cloud storage options are available). OneDrive also has a "version history", so you can retrieve damaged files.

    For the physical disks, I rely on HD Sentinel to monitor, constantly, the SMART metrics within every disk to assess the health of the disk. That way you'll see (almost always; barring sudden calamitous failure which is rare) the beginning of the slow degradation which gives up to a year's warning that it's going to need to be replaced. (That degradation actually starts on day one, but manufacturers hide it until it becomes concerning, so a disk will read "perfect" for years and then appear to decline over months.) HD Sentinel isn't free, but it isn't at all expensive. Once you've realised that it's saved you from losing data through just not knowing your disk was on the way out, you'll realise what good value it is. There are some free ones about, but most are run on demand (which you forget to do). There was a free real-time monitor in Acronis Drive Monitor, but it hasn't been update for years and plays nicely with fewer and fewer disks.

    Other folk here use NAS devices ("Network Attached Storage"). I have a Microserver but seldom use it. Some use commercial backup/storage services - I've heard good things about Crashplan.

    HTH.

    --

    Phil, London

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mike Halmarack@21:1/5 to PhillipHerlihy@SlashDevNull.invalid on Fri Apr 8 14:39:48 2022
    On Fri, 8 Apr 2022 13:09:43 +0100, Philip Herlihy <PhillipHerlihy@SlashDevNull.invalid> wrote:

    In article <n0005htjnedjfqtpqt2doupc0kkh6sa2pl@4ax.com>, Mike Halmarack >wrote...

    Until recently I was using a network of one Surface pro 2 and one Dell
    laptop. This along with 3 external USB drives for storage.

    It was Ok using the external drives as mapped network drives with
    fixed drive letters, until I temporarily split up the system to go
    roaming, then changed the storage structures to suit the temporary
    situation. This made too much housekeeping work on return.

    I built a PC and am now intending to use the external USBs for less
    demanding purposes and adding some larger storage HDs to the PC.

    So, I'm considering a 3.5" Western Digital Black Sata drive.
    Amazon has 6, 8, and 10TB drives on offer.
    I'm tempted to go for the 8 or 10TB drive but see that this could risk
    big data loss if things go wrong at some stage.
    Any advice on this please?

    Also, I could vastly reduce my need for storage by tidying up my
    archives and dumping outdated and obsolete files, but this would
    probably take me long time. so increasing storage capacity might be
    the best option.

    First step is to tot up how much unique data you have and anticipate: if you >have 8TB of files you can't presently dump, then the 6TB disk is a non-starter!

    I can only tell you what I do. My main PC is getting quite elderly now, though
    it's still pleasingly sprightly and would outlast Windows 10's end-of-support. >I have an SSD for the system disk, and another for data - about 100GB of the >stuff. Some of that changes daily; some not even annually. Some is personally >vital.

    You have to assume that any disk store will eventually fail, so anything of any
    value has to be verified to be in more than one place at any time. I have a >4TB spinning disk within my case, and I have Windows 10's File History >configured to write to it. If I smudge a file, I can simply right-click it and
    pick from the list of previous versions offered. (Don't fall into the trap of >a "backup" system that would copy a damaged file over your only earlier >version.) File History - which by default makes copies of anything in a >"Library" (named collection of locations, including standard ones like >"Photos" and "Music") works beautifully - set and forget.

    But of course a missile strike (we are in 2022, after all) would take out the >lot, so from the bunker I'd be able to access my most valued stuff via OneDrive
    (other cloud storage options are available). OneDrive also has a "version >history", so you can retrieve damaged files.

    For the physical disks, I rely on HD Sentinel to monitor, constantly, the SMART
    metrics within every disk to assess the health of the disk. That way you'll >see (almost always; barring sudden calamitous failure which is rare) the >beginning of the slow degradation which gives up to a year's warning that it's >going to need to be replaced. (That degradation actually starts on day one, >but manufacturers hide it until it becomes concerning, so a disk will read >"perfect" for years and then appear to decline over months.) HD Sentinel isn't
    free, but it isn't at all expensive. Once you've realised that it's saved you >from losing data through just not knowing your disk was on the way out, you'll >realise what good value it is. There are some free ones about, but most are >run on demand (which you forget to do). There was a free real-time monitor in >Acronis Drive Monitor, but it hasn't been update for years and plays nicely >with fewer and fewer disks.

    Other folk here use NAS devices ("Network Attached Storage"). I have a >Microserver but seldom use it. Some use commercial backup/storage services - >I've heard good things about Crashplan.

    HTH.

    It most certainly does help. Thanks very much. I'm going to go through
    your suggestions and employ all of them I'm able to.
    That was a great explanation.
    --

    Mike

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Raj Kundra@21:1/5 to Mike Halmarack on Sat Apr 9 14:14:45 2022
    On 08/04/2022 10:52, Mike Halmarack wrote:
    Until recently I was using a network of one Surface pro 2 and one Dell laptop. This along with 3 external USB drives for storage.

    It was Ok using the external drives as mapped network drives with
    fixed drive letters, until I temporarily split up the system to go
    roaming, then changed the storage structures to suit the temporary situation. This made too much housekeeping work on return.

    I built a PC and am now intending to use the external USBs for less
    demanding purposes and adding some larger storage HDs to the PC.

    So, I'm considering a 3.5" Western Digital Black Sata drive.
    Amazon has 6, 8, and 10TB drives on offer.
    I'm tempted to go for the 8 or 10TB drive but see that this could risk
    big data loss if things go wrong at some stage.
    Any advice on this please?

    Also, I could vastly reduce my need for storage by tidying up my
    archives and dumping outdated and obsolete files, but this would
    probably take me long time. so increasing storage capacity might be
    the best option.

    Buy a Motherboard which supports RAID and install 2 identical Hard
    drives and set them as Mirror RAID.

    Or use software like Acronis to sync on fly on 2 HDD in a PC or 2 Hard
    drives in 2PC's.

    That is first safety line. Then you can back up using Acronis to some
    other external HDD, device or PC.

    I got 3 PC use Acronis 2021 from Amazon for £55, no subscription, the
    stand alone version and it does all above nicely.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mike Halmarack@21:1/5 to raj@kundracomputers.co.uk on Sat Apr 9 14:46:12 2022
    On Sat, 9 Apr 2022 14:14:45 +0100, Raj Kundra
    <raj@kundracomputers.co.uk> wrote:

    On 08/04/2022 10:52, Mike Halmarack wrote:
    Until recently I was using a network of one Surface pro 2 and one Dell
    laptop. This along with 3 external USB drives for storage.

    It was Ok using the external drives as mapped network drives with
    fixed drive letters, until I temporarily split up the system to go
    roaming, then changed the storage structures to suit the temporary
    situation. This made too much housekeeping work on return.

    I built a PC and am now intending to use the external USBs for less
    demanding purposes and adding some larger storage HDs to the PC.

    So, I'm considering a 3.5" Western Digital Black Sata drive.
    Amazon has 6, 8, and 10TB drives on offer.
    I'm tempted to go for the 8 or 10TB drive but see that this could risk
    big data loss if things go wrong at some stage.
    Any advice on this please?

    Also, I could vastly reduce my need for storage by tidying up my
    archives and dumping outdated and obsolete files, but this would
    probably take me long time. so increasing storage capacity might be
    the best option.

    Buy a Motherboard which supports RAID and install 2 identical Hard
    drives and set them as Mirror RAID.

    Or use software like Acronis to sync on fly on 2 HDD in a PC or 2 Hard
    drives in 2PC's.

    That is first safety line. Then you can back up using Acronis to some
    other external HDD, device or PC.

    I got 3 PC use Acronis 2021 from Amazon for 55, no subscription, the
    stand alone version and it does all above nicely.

    Mirror RAID looks like the sort of belt and braces option I like when
    combined with Philips suggestions. Thanks
    --

    Mike

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Raj Kundra@21:1/5 to Mike Halmarack on Sat Apr 9 15:00:36 2022
    On 09/04/2022 14:46, Mike Halmarack wrote:
    On Sat, 9 Apr 2022 14:14:45 +0100, Raj Kundra
    <raj@kundracomputers.co.uk> wrote:

    On 08/04/2022 10:52, Mike Halmarack wrote:
    Until recently I was using a network of one Surface pro 2 and one Dell
    laptop. This along with 3 external USB drives for storage.

    It was Ok using the external drives as mapped network drives with
    fixed drive letters, until I temporarily split up the system to go
    roaming, then changed the storage structures to suit the temporary
    situation. This made too much housekeeping work on return.

    I built a PC and am now intending to use the external USBs for less
    demanding purposes and adding some larger storage HDs to the PC.

    So, I'm considering a 3.5" Western Digital Black Sata drive.
    Amazon has 6, 8, and 10TB drives on offer.
    I'm tempted to go for the 8 or 10TB drive but see that this could risk
    big data loss if things go wrong at some stage.
    Any advice on this please?

    Also, I could vastly reduce my need for storage by tidying up my
    archives and dumping outdated and obsolete files, but this would
    probably take me long time. so increasing storage capacity might be
    the best option.

    Buy a Motherboard which supports RAID and install 2 identical Hard
    drives and set them as Mirror RAID.

    Or use software like Acronis to sync on fly on 2 HDD in a PC or 2 Hard
    drives in 2PC's.

    That is first safety line. Then you can back up using Acronis to some
    other external HDD, device or PC.

    I got 3 PC use Acronis 2021 from Amazon for £55, no subscription, the
    stand alone version and it does all above nicely.

    Mirror RAID looks like the sort of belt and braces option I like when combined with Philips suggestions. Thanks

    I found best RAID is the one allows you to farmat it NTFS under Windows.

    If one Hard drive or PC fails, you just plug the Hard drive in another
    Windows PC or external USB caddy and you can see everything on Windows
    PC or on Apple Mac to recover.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jaimie Vandenbergh@21:1/5 to mikehalmarack@gmail.com on Sat Apr 9 21:07:14 2022
    On 9 Apr 2022 at 14:46:12 BST, "Mike Halmarack"
    <mikehalmarack@gmail.com> wrote:


    Mirror RAID looks like the sort of belt and braces option I like when combined with Philips suggestions. Thanks

    Raid is belt *or* braces, it gives you continuity when one HDD dies. It
    does not protect you from the filesystem going bad, deleting stuff, or
    your OS trashing them.

    You get belt *and* braces by implementing an actual backup. Local
    backups are a pain and don't tend to get done, unless you have an always
    on NAS and a scheduled backup enabled (all OSes come with those builtin
    now). I commend at least getting an Internet-based backup: Backblaze has
    a very cheap "all you can eat for one machine" service which I've been
    using for several years now.

    Any data you actually want to keep? You need it to be in at least three
    places, one of them offsite.

    Cheers - Jaimie
    --
    Homographic homophonic autantonyms: "They're words
    that do their job in the most sullen, passive-aggressive
    way possible, and they totally get away with it!"
    -- http://www.qwantz.com/index.php?comic=1104

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Philip Herlihy@21:1/5 to All on Sun Apr 10 14:38:06 2022
    In article <jbeas2Faka1U1@mid.individual.net>, Jaimie Vandenbergh wrote...

    On 9 Apr 2022 at 14:46:12 BST, "Mike Halmarack"
    <mikehalmarack@gmail.com> wrote:


    Mirror RAID looks like the sort of belt and braces option I like when combined with Philips suggestions. Thanks

    Raid is belt *or* braces, it gives you continuity when one HDD dies. It
    does not protect you from the filesystem going bad, deleting stuff, or
    your OS trashing them.

    You get belt *and* braces by implementing an actual backup. Local
    backups are a pain and don't tend to get done, unless you have an always
    on NAS and a scheduled backup enabled (all OSes come with those builtin
    now). I commend at least getting an Internet-based backup: Backblaze has
    a very cheap "all you can eat for one machine" service which I've been
    using for several years now.

    Any data you actually want to keep? You need it to be in at least three places, one of them offsite.

    Cheers - Jaimie

    Jaimie - as ever - is right on the nail here. RAID (strictly, Raid 1) isn't a backup, it's a copy (of everything). If, say, you open an important spreadsheet, accidentally delete a vital tab, and then save it, RAID-1 will promptly and gleefully mirror that over to the other disk. If you don't have historic versions, you don't have a backup, only a copy of the very latest version. In my view RAID is most useful for "Enterprise" configurations where time-to-restoration is critical. They will still have backups as well, for the above reasons.
    https://en.wikipedia.org/wiki/Standard_RAID_levels

    Looked at one way, RAID-1 halves the useful capacity of your disk set! Incidentally Windows has something like RAID available as "Storage Spaces" implemented in software. I found the reference (in one configuration) to "...at least seven drives to protect you from two drive failures..." a useful clue! https://support.microsoft.com/en-gb/windows/storage-spaces-in-windows-b6c8b540- b8d8-fb8a-e7ab-4a75ba11f9f2#WindowsVersion=Windows_10

    To set up RAID-1, you'll need to slot in another disk, ideally an identical one (same size, same speed, same model if possible) and navigate the mysteries of the BIOS/Setup on your particular machine. Have a look at that and you'll quickly see why most people don't bother (though the untypically smart folk in this group would take it in their stride). Then you still have to set up some sort of backup - but if a disk did suddenly need to be replaced, it's quick and easy. If that's a mirrored system disk, then no need for the system images (e.g. Acronis True Image, Macrium, etc) that some of us will have maintained in case of disk failure. These days, with the option to Reset an installation, and easier clean reinstallation with many settings preserved via a Microsoft Account, I don't bother with imaging any more, accepting that there are some third-party applications I'd have to reinstall. But with proper monitoring in place (e.g. HD Sentinel) you would usually get plenty of warning to replace a disk, though a "lights out" can still happen, particularly with SSDs, though there are suggestions they are overall more durable.

    My setup meets Jaimie's test of three locations. Files are stored on the PC's dedicated Data disk (could just as well be on the system disk) and are backed up without me ever thinking about it using File History to the big spinner in the same case. Two copies so far. The important folders are also configured in OneDrive, so that stuff's all in the cloud as well. The OneDrive folders are configured _not_ to use "Files-on-demand" (see link) so those folders remain fully populated (or I'd be down to two copies): https://support.microsoft.com/en-us/office/save-disk-space-with-onedrive-files- on-demand-for-windows-10-0e6860d3-d9f3-4971-b321-7092438fb38e

    Bottom line: There are lots of options for managing your data. If you stick to Jaimie's "two copies + Cloud" principle in any way convenient for you then you shouldn't go far wrong.

    Postscript: File History also works well with removeable drives (so that you can keep your backups somewhere else). If the drive isn't detected when a file changes, it caches the data until such time as the drive is detected. If you run Disk Cleanup on a disk you can see this "User File History" as an option to be purged (assuming FH is configured). (Don't!)

    --

    Phil, London

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mike Halmarack@21:1/5 to jaimie@usually.sessile.org on Mon Apr 11 08:12:06 2022
    On 9 Apr 2022 21:07:14 GMT, Jaimie Vandenbergh
    <jaimie@usually.sessile.org> wrote:

    On 9 Apr 2022 at 14:46:12 BST, "Mike Halmarack"
    <mikehalmarack@gmail.com> wrote:


    Mirror RAID looks like the sort of belt and braces option I like when
    combined with Philips suggestions. Thanks

    Raid is belt *or* braces, it gives you continuity when one HDD dies. It
    does not protect you from the filesystem going bad, deleting stuff, or
    your OS trashing them.

    You get belt *and* braces by implementing an actual backup. Local
    backups are a pain and don't tend to get done, unless you have an always
    on NAS and a scheduled backup enabled (all OSes come with those builtin
    now). I commend at least getting an Internet-based backup: Backblaze has
    a very cheap "all you can eat for one machine" service which I've been
    using for several years now.

    Any data you actually want to keep? You need it to be in at least three >places, one of them offsite.

    Cheers - Jaimie

    Thanks for clearing that up so succinctly. A big help.
    --

    Mike

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mike Halmarack@21:1/5 to PhillipHerlihy@SlashDevNull.invalid on Mon Apr 11 08:14:14 2022
    On Sun, 10 Apr 2022 14:38:06 +0100, Philip Herlihy <PhillipHerlihy@SlashDevNull.invalid> wrote:

    In article <jbeas2Faka1U1@mid.individual.net>, Jaimie Vandenbergh wrote...

    On 9 Apr 2022 at 14:46:12 BST, "Mike Halmarack"
    <mikehalmarack@gmail.com> wrote:


    Mirror RAID looks like the sort of belt and braces option I like when
    combined with Philips suggestions. Thanks

    Raid is belt *or* braces, it gives you continuity when one HDD dies. It
    does not protect you from the filesystem going bad, deleting stuff, or
    your OS trashing them.

    You get belt *and* braces by implementing an actual backup. Local
    backups are a pain and don't tend to get done, unless you have an always
    on NAS and a scheduled backup enabled (all OSes come with those builtin
    now). I commend at least getting an Internet-based backup: Backblaze has
    a very cheap "all you can eat for one machine" service which I've been
    using for several years now.

    Any data you actually want to keep? You need it to be in at least three
    places, one of them offsite.

    Cheers - Jaimie

    Jaimie - as ever - is right on the nail here. RAID (strictly, Raid 1) isn't a >backup, it's a copy (of everything). If, say, you open an important >spreadsheet, accidentally delete a vital tab, and then save it, RAID-1 will >promptly and gleefully mirror that over to the other disk. If you don't have >historic versions, you don't have a backup, only a copy of the very latest >version. In my view RAID is most useful for "Enterprise" configurations where >time-to-restoration is critical. They will still have backups as well, for the
    above reasons.
    https://en.wikipedia.org/wiki/Standard_RAID_levels

    Looked at one way, RAID-1 halves the useful capacity of your disk set! >Incidentally Windows has something like RAID available as "Storage Spaces" >implemented in software. I found the reference (in one configuration) to >"...at least seven drives to protect you from two drive failures..." a useful >clue! >https://support.microsoft.com/en-gb/windows/storage-spaces-in-windows-b6c8b540-
    b8d8-fb8a-e7ab-4a75ba11f9f2#WindowsVersion=Windows_10

    To set up RAID-1, you'll need to slot in another disk, ideally an identical one
    (same size, same speed, same model if possible) and navigate the mysteries of >the BIOS/Setup on your particular machine. Have a look at that and you'll >quickly see why most people don't bother (though the untypically smart folk in >this group would take it in their stride). Then you still have to set up some >sort of backup - but if a disk did suddenly need to be replaced, it's quick and
    easy. If that's a mirrored system disk, then no need for the system images >(e.g. Acronis True Image, Macrium, etc) that some of us will have maintained in
    case of disk failure. These days, with the option to Reset an installation, >and easier clean reinstallation with many settings preserved via a Microsoft >Account, I don't bother with imaging any more, accepting that there are some >third-party applications I'd have to reinstall. But with proper monitoring in >place (e.g. HD Sentinel) you would usually get plenty of warning to replace a >disk, though a "lights out" can still happen, particularly with SSDs, though >there are suggestions they are overall more durable.

    My setup meets Jaimie's test of three locations. Files are stored on the PC's >dedicated Data disk (could just as well be on the system disk) and are backed >up without me ever thinking about it using File History to the big spinner in >the same case. Two copies so far. The important folders are also configured >in OneDrive, so that stuff's all in the cloud as well. The OneDrive folders >are configured _not_ to use "Files-on-demand" (see link) so those folders >remain fully populated (or I'd be down to two copies): >https://support.microsoft.com/en-us/office/save-disk-space-with-onedrive-files-
    on-demand-for-windows-10-0e6860d3-d9f3-4971-b321-7092438fb38e

    Bottom line: There are lots of options for managing your data. If you stick to
    Jaimie's "two copies + Cloud" principle in any way convenient for you then you >shouldn't go far wrong.

    Postscript: File History also works well with removeable drives (so that you >can keep your backups somewhere else). If the drive isn't detected when a file
    changes, it caches the data until such time as the drive is detected. If you >run Disk Cleanup on a disk you can see this "User File History" as an option to
    be purged (assuming FH is configured). (Don't!)

    Thanks a million Philip. Definitely waving, not drowning :-)
    --

    Mike

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)