Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write operation, lots of sectors may be internally rejiggered to provide
uniform wear (so things that are never rewritten from the application
point of view are actually moved around and rewritten). This activity is invisible from normal application's point of view, but takes some time.
If the device is powered off during such activity, data corruption...
So, its necessary to provide the device some time after the last write
before it is powered off. Plenty of embedded stuff I've seen too often corrupts memory at power off, and of course the device manufacturers
blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write operation,
lots of sectors may be internally rejiggered to provide uniform wear (so things
that are never rewritten from the application point of view are actually moved
around and rewritten). This activity is invisible from normal application's point of view, but takes some time. If the device is powered off during such activity, data corruption... So, its necessary to provide the device some time
after the last write before it is powered off. Plenty of embedded stuff I've seen too often corrupts memory at power off, and of course the device manufacturers blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
On 7/24/2021 8:19 AM, Dave Nadler wrote:
Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write
operation, lots of sectors may be internally rejiggered to provide
uniform wear (so things that are never rewritten from the application
point of view are actually moved around and rewritten). This activity
is invisible from normal application's point of view, but takes some
time. If the device is powered off during such activity, data
corruption... So, its necessary to provide the device some time after
the last write before it is powered off. Plenty of embedded stuff I've
seen too often corrupts memory at power off, and of course the device
manufacturers blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
Two (transient) copies of each "chunk" of data. Update the second
copy (on or before impending power fail). When complete, toggle
a pointer to reference it as the most recent (assuming you don't
have metadata that automatically provides that sort of information).
The "first" copy can now be removed and its space reallocated to
some other "second copy"
Then, move on to the next "chunk". Design your algorithms so that
you can tolerate some chunks being "stale" (not updated in time)
wrt the others.
So, how do you avoid this kind of corruption in your designs?
Two (transient) copies of each "chunk" of data. Update the second
copy (on or before impending power fail). When complete, toggle
a pointer to reference it as the most recent (assuming you don't
have metadata that automatically provides that sort of information).
The "first" copy can now be removed and its space reallocated to
some other "second copy"
Then, move on to the next "chunk". Design your algorithms so that
you can tolerate some chunks being "stale" (not updated in time)
wrt the others.
On 7/24/2021 9:10 PM, Don Y wrote:
So, how do you avoid this kind of corruption in your designs?
Two (transient) copies of each "chunk" of data. Update the second
copy (on or before impending power fail). When complete, toggle
a pointer to reference it as the most recent (assuming you don't
have metadata that automatically provides that sort of information).
The "first" copy can now be removed and its space reallocated to
some other "second copy"
Then, move on to the next "chunk". Design your algorithms so that
you can tolerate some chunks being "stale" (not updated in time)
wrt the others.
Don, I don't think that works. The entire contents of the volume can
be corrupted if power is removed during wear-leveling operations that are invisible to the host OS. And I guess you are assuming no file system is in use
(because file system internals would still get corrupted)?
On 7/25/2021 4:10, Don Y wrote:
On 7/24/2021 8:19 AM, Dave Nadler wrote:
Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write
operation, lots of sectors may be internally rejiggered to provide uniform >>> wear (so things that are never rewritten from the application point of view >>> are actually moved around and rewritten). This activity is invisible from >>> normal application's point of view, but takes some time. If the device is >>> powered off during such activity, data corruption... So, its necessary to >>> provide the device some time after the last write before it is powered off. >>> Plenty of embedded stuff I've seen too often corrupts memory at power off, >>> and of course the device manufacturers blame the memory manufacturers... >>>
So, how do you avoid this kind of corruption in your designs?
Two (transient) copies of each "chunk" of data. Update the second
copy (on or before impending power fail). When complete, toggle
a pointer to reference it as the most recent (assuming you don't
have metadata that automatically provides that sort of information).
The "first" copy can now be removed and its space reallocated to
some other "second copy"
Then, move on to the next "chunk". Design your algorithms so that
you can tolerate some chunks being "stale" (not updated in time)
wrt the others.
Don, this strategy is obviously good (standard?) for say disk writes
etc., but I don't know if it will be OK for flash as it involves erase
then write, AFAIK "erase" is the most killing part. Perhaps they do
a lot of "refresh", i.e. read while they can then rewrite the same
locations? I don't really know, just wondering how they do it.
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write >operation, lots of sectors may be internally rejiggered to provide
uniform wear (so things that are never rewritten from the application
point of view are actually moved around and rewritten). This activity is >invisible from normal application's point of view, but takes some time.
If the device is powered off during such activity, data corruption...
So, its necessary to provide the device some time after the last write
before it is powered off. Plenty of embedded stuff I've seen too often >corrupts memory at power off, and of course the device manufacturers
blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
Thanks,
Best Regards, Dave
On 7/24/2021 9:10 PM, Don Y wrote:
So, how do you avoid this kind of corruption in your designs?
Two (transient) copies of each "chunk" of data. Update the second
copy (on or before impending power fail). When complete, toggle
a pointer to reference it as the most recent (assuming you don't
have metadata that automatically provides that sort of information).
The "first" copy can now be removed and its space reallocated to
some other "second copy"
Then, move on to the next "chunk". Design your algorithms so that
you can tolerate some chunks being "stale" (not updated in time)
wrt the others.
Don, I don't think that works. The entire contents of the volume can
be corrupted if power is removed during wear-leveling operations that
are invisible to the host OS. And I guess you are assuming no file
system is in use (because file system internals would still get corrupted)?
On 7/25/2021 12:51 PM, Dimiter_Popoff wrote:
......
Of course again
you will be prone to data loss on unexpected power loss but updating
the write cache will be done at (much) longer intervals and in
relatively brief bursts, thus the probability to be hit gets lower.
The usual trick is to just have a mirror of the FLASH contents
and flush that to the media when you're pretty sure the ship
is sinking.
But, if this is *large*, then you risk only partial writes before
power drops to an unusable/unreliable level.
The rest can only be got rid of by having large enough caps on
the power supply and an early warning that power has been lost
(usually at the input of the stepdowns and you are running on fumes.
I'm not sure that will work with consumer devices. You have no way of knowing how long it will take to reliably flush ALL of the data
out to the physical store.
If you have no knowledge of how the flash medium is written - which
would normally be the case with USB sticks or SD cards etc. - the most efficient way to mitigate data loss probability is to do as much
write cacheing as you can afford at system level.
Of course again
you will be prone to data loss on unexpected power loss but updating
the write cache will be done at (much) longer intervals and in
relatively brief bursts, thus the probability to be hit gets lower.
The rest can only be got rid of by having large enough caps on
the power supply and an early warning that power has been lost
(usually at the input of the stepdowns and you are running on fumes.
Of course you can employ various strategies to mimick flash drive
behaviour at a higher level, Don suggested one, Kent also did,
but these mean having different drivers for different media and
all the consequences of that.
I prefer to talk to standard SCSI
or ATA behaving things and leave it to them to handle their
unique specifics. [I remember how I *had to* introduce write
cacheing at system level, in the mid 90-s I started to do backups
under DPS to magneto-optical devices; early days DPS would
simply update its CAT (Cluster Allocation Table) every time
it got modifier and these sectors got corrupted before the backup
would finish.... :-) ].
In article <sdhb29$ibh$1@gioia.aioe.org>, Dave Nadler <drn@nadler.com> wrote:
Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write
operation, lots of sectors may be internally rejiggered to provide
uniform wear (so things that are never rewritten from the application
point of view are actually moved around and rewritten). This activity is
invisible from normal application's point of view, but takes some time.
If the device is powered off during such activity, data corruption...
So, its necessary to provide the device some time after the last write
before it is powered off. Plenty of embedded stuff I've seen too often
corrupts memory at power off, and of course the device manufacturers
blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
Thanks,
Best Regards, Dave
It is not necessary to do this. Here's a very simple way to recover with
no data loss ever. It is simply "log-only" or "journal-only" storage.
The device keeps 10% capacity reserved (pre-erased) (or less, this is a performance number). The driver has enough RAM to track where every logical block is (this isn't much RAM). And then writing is always done as a logfile--write whatever the user data is starting at block 0, and move up. After writing user data, write a log entry (this can be just right after the user data). Writes never go to the block the user indicates, they always go only to the log pointer. Reading looks up where that block's latest copy is, and reads that.
When you're at less than 10% space left (near the end of the device for first-pass writes), you simply read from +10% from the current log write position, and write it to the log (compacting it to avoid re-writing any blocks which are now obsolete) until you free up space. Since many blocks are
rewritten, when we compact them we free space. Once copied, then erase the blocks which were compacted. Then just keep growing the "log" through the newly erased blocks.
Upon power up, scan entire device for the log info, and rebuild the RAM index.
Power can be removed at any time, and fully recovered, although obviously the most recent data may be lost, but we can roll back to the last consistent state just like journaled file systems (the log entries need a checksum so
we can validate them). Wear leveling is achieved by design.
There are lots of small ways to improve the above, so do that.
I have no idea what actual devices do, I suspect it's something much more complex and less robust.
On 7/25/2021 23:27, Don Y wrote:
On 7/25/2021 12:51 PM, Dimiter_Popoff wrote:
......
Of course again
you will be prone to data loss on unexpected power loss but updating
the write cache will be done at (much) longer intervals and in
relatively brief bursts, thus the probability to be hit gets lower.
The usual trick is to just have a mirror of the FLASH contents
and flush that to the media when you're pretty sure the ship
is sinking.
But, if this is *large*, then you risk only partial writes before
power drops to an unusable/unreliable level.
Well I think tens of gigabytes for small drives nowadays, what can you
do. Cacheing it all just is not feasible.
What DPS does is cache directories and CATs on a 4k piece basis
(regardless of the disk block size, 512 2048, 4k) and maintains a
bitmap for its dirty regions - so when it updates to the medium
it writes at least 4k and no more than twice 4095 excessive bytes
or something like that.
The rest can only be got rid of by having large enough caps on
the power supply and an early warning that power has been lost
(usually at the input of the stepdowns and you are running on fumes.
I'm not sure that will work with consumer devices. You have no way of
knowing how long it will take to reliably flush ALL of the data
out to the physical store.
Oh with consumer devices you have not built yourself it will probably
never work, of course. They are unlikely to give you an early warning
in the first place. If you want to be safer you just use good rotating disks.... I think I read somewhere they do their soft shutdown using the energy in the rotor (and do that since times immemorial), but I have
never verified this.
Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write operation, lots of sectors may be internally rejiggered to provide
uniform wear (so things that are never rewritten from the application
point of view are actually moved around and rewritten). This activity is invisible from normal application's point of view, but takes some time.
If the device is powered off during such activity, data corruption...
So, its necessary to provide the device some time after the last write
before it is powered off. Plenty of embedded stuff I've seen too often corrupts memory at power off, and of course the device manufacturers
blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
Thanks,
Best Regards, Dave
On 24-07-2021 17:19, Dave Nadler wrote:
Hi All - I'm wondering what other folks do about this issue...
Consumer flash storage devices (USB memory stick, SD card, etc)
have a nice internal wear-leveling controller. When one does a write
operation, lots of sectors may be internally rejiggered to provide uniform >> wear (so things that are never rewritten from the application point of view >> are actually moved around and rewritten). This activity is invisible from
normal application's point of view, but takes some time. If the device is
powered off during such activity, data corruption... So, its necessary to
provide the device some time after the last write before it is powered off. >> Plenty of embedded stuff I've seen too often corrupts memory at power off, >> and of course the device manufacturers blame the memory manufacturers...
So, how do you avoid this kind of corruption in your designs?
Thanks,
Best Regards, Dave
A number of upper tier memory device manufacturers sell flash drives, USB sticks, SS Sata drives and M2 drives with "UPS" capabilities. When they sense a
loss of power they will have just enough onboard juice to power down in a controller manner.
I have used drives from InnoDisk ( https://www.innodisk.com/en/products/flash-storage/ssd ) on Linux boxes inside
machines with hard on/off switches. With regular drives we'd see boot problems
after 20 to 30 power cycles, with the "UPS" protected ones we could do hundreds
of hard cycles without ill effect.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 293 |
Nodes: | 16 (2 / 14) |
Uptime: | 211:42:06 |
Calls: | 6,619 |
Calls today: | 1 |
Files: | 12,168 |
Messages: | 5,317,311 |