I run a laptop, using Ubuntu, with a portable HDD permanently connected
to a USB port, for backups as required. Usually, when the laptop
hibernates, or whatever it's called, a couple of 'Enter' taps brings it
back to life, I enter my password, and away we go. Yesterday, it would
not wake up, just showing me a screen with the sign-on page, and some chevrons moving up from the bottom of the screen. Nothing I could do
would get it out of this, so eventually I had to do a hard shut-down and reboot. When it was back in action, the main back-up folder on the
portable HDD was missing. Lots of other stuff was there, but not the
major folder with all the up-to-date files.
I can recreate it from other sources, so it's not the end of the world,
but is there a way I could try to find the missing folder on the HDD? It
is not in the Rubbish bin, and seems to have disappeared without trace.
Has it gone to the great bit-bucket in the sky?
Questions:Not that I know of, but I can't say 100% that were were none. Surely
- were there any power surges or dropouts recently?
- how old is the laptop and what sort of condition is it *and its
battery* in?
- How old is that backup drive? You'd probably need to need to have
smartd installed to know that.
- do you run fsck after a backup? If not, then run it now.
What does it say about the health of that disk?
- what does fsck say about the health of the laptop's internal drive?
- what do fsck and smartd say about the age and health of the disk you
were backing up?
FWIW I do something similar except that:
- I have a WD Essentials USB drive permanently attached to my house
server. A cron job runs an overnight backup to it every night using
rsync and followed by a fsck check of the backup disk. This backup
is merely protection against finger trouble because mains spikes,
house fires, etc could destroy it.
- my main backups are run weekly to a cycle of two WD Essentials USB
drives which are kept offline in a fire safe when not being used to
take a backup. Each backup run uses rsync to make the backup and
then immediately runs fask to check that the backup is valid.
On Sat, 23 Apr 2022 08:59:31 -0000 (UTC)
Martin Gregorie <martin@mydomain.invalid> wrote:
Questions:
- were there any power surges or dropouts recently?Not that I know of, but I can't say 100% that were were none. Surely
the laptop PSU would ride that out, though?
- how old is the laptop and what sort of condition is it *and its
battery* in?
The laptop is about a year old, and as far as I know, the battery is
in good condition. I'll run without the mains connected for a while to confirm.
- How old is that backup drive? You'd probably need to need to have
smartd installed to know that.
That is now several years, maybe three or four. I have never used
smartd, so I need to research that to use it.
- do you run fsck after a backup? If not, then run it now.
What does it say about the health of that disk?
No, I don't.
- what does fsck say about the health of the laptop's internal$ sudo fsck /media/Expansion_Drive
drive?
fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
/dev/sdb2 contains a filesystem with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
/lost+found not found. Create<y>? yes
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb2: ***** FILESYSTEM WAS MODIFIED *****
/dev/sdb2: 66294/14958592 files (1.9% non-contiguous),
27144482/59813632 block
- what do fsck and smartd say about the age and health of the disk
you were backing up?
Don't know yet, need to unmount HDD to run fsck.
FWIW I do something similar except that:That is similar to my regime. I run an rsync job each evening on
- I have a WD Essentials USB drive permanently attached to my house
server. A cron job runs an overnight backup to it every night
using rsync and followed by a fsck check of the backup disk. This
backup is merely protection against finger trouble because mains
spikes, house fires, etc could destroy it.
fsck
shutdown, to both my Zoneminder PC and the portable HDD. Never done
fsck, though.
- my main backups are run weekly to a cycle of two WD Essentials USB
drives which are kept offline in a fire safe when not being used
to take a backup. Each backup run uses rsync to make the backup and
then immediately runs fask to check that the backup is valid.
snip.
On Sat, 23 Apr 2022 10:47:00 +0100
Davey <davey@example.invalid> wrote:
On Sat, 23 Apr 2022 08:59:31 -0000 (UTC)
Martin Gregorie <martin@mydomain.invalid> wrote:
Questions:
- were there any power surges or dropouts recently?Not that I know of, but I can't say 100% that were were none. Surely
the laptop PSU would ride that out, though?
- how old is the laptop and what sort of condition is it *and its battery* in?
The laptop is about a year old, and as far as I know, the battery is
in good condition. I'll run without the mains connected for a while
to confirm.
- How old is that backup drive? You'd probably need to need to
have smartd installed to know that.
That is now several years, maybe three or four. I have never used
smartd, so I need to research that to use it.
- do you run fsck after a backup? If not, then run it now.
What does it say about the health of that disk?
No, I don't.
- what does fsck say about the health of the laptop's internal$ sudo fsck /media/Expansion_Drive
drive?
fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
/dev/sdb2 contains a filesystem with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
/lost+found not found. Create<y>? yes
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb2: ***** FILESYSTEM WAS MODIFIED *****
/dev/sdb2: 66294/14958592 files (1.9% non-contiguous),
27144482/59813632 block
- what do fsck and smartd say about the age and health of the disk
you were backing up?
Don't know yet, need to unmount HDD to run fsck.
FWIW I do something similar except that:That is similar to my regime. I run an rsync job each evening on
- I have a WD Essentials USB drive permanently attached to my
house server. A cron job runs an overnight backup to it every
night using rsync and followed by a fsck check of the backup
disk. This backup is merely protection against finger trouble
because mains spikes, house fires, etc could destroy it.
fsck
shutdown, to both my Zoneminder PC and the portable HDD. Never done
fsck, though.
- my main backups are run weekly to a cycle of two WD Essentialssnip.
USB drives which are kept offline in a fire safe when not being
used to take a backup. Each backup run uses rsync to make the
backup and then immediately runs fask to check that the backup is
valid.
Update:
smartd (short) test on the laptop drive passes. Only odd thing is
there are several Unknown Attributes, as in:
160 Unknown_Attribute 0x0032 100 100 050 Old_age
Always - 0 161 Unknown_Attribute 0x0033 100
100 050 Pre-fail Always - 100 163
Unknown_Attribute 0x0032 100 100 050 Old_age
Always - 17 164 Unknown_Attribute 0x0032 100
100 050 Old_age Always - 470 etc etc....
Trying to configure smartd to recognise the portable HDD, it is a
Passport, on /media/Expansion_Drive. I cannot get smartd to recognise
it, though. See above for fsck report.
To my admittedly very unprofessional eye, it looks good
On Sat, 23 Apr 2022 11:19:23 +0100 Davey <davey@example.invalid> wrote:
To my admittedly very unprofessional eye, it looks good. I can only
assume that the failed Hibernation interrupted the normal data exchange process, and f****d up the file system. I'll run a long test tonight.
On Sat, 23 Apr 2022 11:34:08 +0100, Davey wrote:
On Sat, 23 Apr 2022 11:19:23 +0100 Davey <davey@example.invalid>
wrote:
To my admittedly very unprofessional eye, it looks good. I can onlyAgreed, though the reported age of 47K hours is possibly a warning
assume that the failed Hibernation interrupted the normal data
exchange process, and f****d up the file system. I'll run a long
test tonight.
sign: I've run a few disks into the ground and all have started to
report errors at around the 50,000 hours mark.
In both cases I bought new disks at when the errors started to
appear, got hold of a CloneZilla CD image, burnt it, and used it to
duplicate the failing disk.
I've not had any backup disks fail (yet) but they're easier to
replace: simply format them to suit your OS, and bin the old disk as
soon as you've made a backup on the new one.
Apart from that all I'd suggest is:
- installing smartd on all your machines, configured to report disk
status on a weekly basis.
I run logwatch on all my systems, configured to send their daily
reports to the computer I usually read mail on, so this means I'll
see trouble reports within 24 hours of their happening
- extend your backup scripts to run fsck after a backup is made.
- do you run fsck after a backup? If not, then run it now.
What does it say about the health of that disk?
- what does fsck say about the health of the laptop's internal drive?
- what do fsck and smartd say about the age and health of the disk you
were backing up?
Davey wrote:
To my admittedly very unprofessional eye, it looks good
Did anything end-up in the lost+found directory?
On 23/04/2022 09:59, Martin Gregorie wrote:
- do you run fsck after a backup? If not, then run it now.
What does it say about the health of that disk?
- what does fsck say about the health of the laptop's internal
drive?
- what do fsck and smartd say about the age and health of the disk
you were backing up?
Probably a bit late now but for future consideration if I had a
suspect drive then running fsck against the drive wouldn't be a first
move as the drive could degrade further and it only permits single
recovery options.
My first action would be use dd or similar to write the data off to a
raw image file and then run fsck/other disk recovery utils against
copies of that copy.
This enables the original disk to be powered down and set aside to
avoid further damage and also permits multiple recovery options
(against the image files).
On Sat, 23 Apr 2022 13:12:51 +0100
Mike Civil <mike@duncodin.org> wrote:
Probably a bit late now but for future consideration if I had a
suspect drive then running fsck against the drive wouldn't be a first
move as the drive could degrade further and it only permits single
recovery options.
My first action would be use dd or similar to write the data off to a
raw image file and then run fsck/other disk recovery utils against
copies of that copy.
This enables the original disk to be powered down and set aside to
avoid further damage and also permits multiple recovery options
(against the image files).
It's probably not worth my making a copy, as the main folder that was
its purpose is the one that's missing!
Since I see no way of finding it, I will recreate it from the other
backups, and also I will unmount the drive unless I am using it.
I don't even know if this drive is suspect, it passes the tests. I
still reckon it was a corruption of communication when the laptop didn't hibernate properly.
But I appreciate the thoughts.
It's now nearly three hours of on/off action, and the battery is shown- how old is the laptop and what sort of condition is it *and its
battery* in?
The laptop is about a year old, and as far as I know, the battery is
in good condition. I'll run without the mains connected for a while to confirm.
It's probably not worth my making a copy, as the main folder that was
its purpose is the one that's missing!
Since I see no way of finding it, I will recreate it from the other
backups, and also I will unmount the drive unless I am using it.
I don't even know if this drive is suspect, it passes the tests. I
still reckon it was a corruption of communication when the laptop
didn't hibernate properly.
But I appreciate the thoughts.
While a plug-in backup disk can be useful, for example when you're
doing significant work away from home or your normal workplace, they
are rather vulnerable to this sort of failure. Rather than buy
another one, if that's where you're heading, I'd invest in a NAS.
They're more expensive up front, but IME more reliable long-term.
Most modern ones will run smartd, and RAID, so you should get some
warning if things are going pear-shaped, and be able to swap out a
failing disk, though in fact my version of RAID is actually to use
two identical NASs! I hate to tempt fate, but in this way I haven't
lost significant amounts of data for a long time.
You can back up via your network just changed data using rsync either manually or on an overnight schedule, or an entire operating system periodically. I keep both Linux and Windows PCs backed up this way.
The Linux OS partitions are backed up using Clonezilla, the Windows
OS partitions using Ghost (others favour other imaging software which
is just as good), and for both data is backed up daily using rsync,
which is basically what the DeltaCopy programme for Windows is, it's
a GUI rsync launcher.
Davey wrote:
To my admittedly very unprofessional eye, it looks good
Did anything end-up in the lost+found directory?
On Sat, 23 Apr 2022 11:41:00 +0100 Andy Burns <usenet@andyburns.uk>
wrote:
Davey wrote:
To my admittedly very unprofessional eye, it looks good
Did anything end-up in the lost+found directory?
Update, as in "WTF?"
All worked well during the day, with several rsync runs, all working properly.
Then at evening shutdown, it reported that the backup folder again did
not exist! And it was correct.
What I think I will do is to create the backup folder under a different
name, change the name in the rsync script to match, and re-re-create the backup folder, and see if that makes any difference.
Other things to try might be to re-format the drive.
Again, fsck reports a happy healthy drive.
A busy morning looms for other reasons, so this will be delayed.
On Mon, 25 Apr 2022 08:23:32 +0100, Davey wrote:
On Sat, 23 Apr 2022 11:41:00 +0100 Andy Burns <usenet@andyburns.uk>
wrote:
Davey wrote:
To my admittedly very unprofessional eye, it looks good
Did anything end-up in the lost+found directory?
Update, as in "WTF?"
All worked well during the day, with several rsync runs, all working properly.
Then at evening shutdown, it reported that the backup folder again
did not exist! And it was correct.
What I think I will do is to create the backup folder under a
different name, change the name in the rsync script to match, and re-re-create the backup folder, and see if that makes any
difference. Other things to try might be to re-format the drive.
Again, fsck reports a happy healthy drive.
A busy morning looms for other reasons, so this will be delayed.
Might be best to buy one or two USB hard drives (or USB memory
sticks) of the same or bigger capacity than the drive you're backing
and use them rather than relying on a backup folder on the same
ageing disk.
IMO 2 backup drives, used in rotation, are better then one because
this means that you'll always have a backup for your most recent
backup and, since the backup drive(s) are mostly offline, they will
age a lot slower than your usual main drive.
Last, but not least, doing overnight backups is a good idea: that way
you're never hanging about waiting for a backup to finish. Next
morning you check logwatch to make sure the backup went OK and swap
backup drives ready for next night's backup.
--
Martin | martin at
Gregorie | gregorie dot org
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 296 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 85:26:18 |
| Calls: | 6,658 |
| Files: | 12,203 |
| Messages: | 5,333,708 |