Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well.
Howdy,
I've pretty much reached a limit on my backups. I'm up to a 16TB hard
drive for one and even that won't last long. Larger drives are much
more costly. A must have NAS is quickly approaching. I've been
searching around and find some things confusing. I'm hoping someone can clear up that confusion. I'm also debating what path to travel down.
I'd also like to keep costs down as well. That said, I don't mind
paying a little more for one that would offer a much better option.
Path one, buy a NAS, possibly used, that has no drives. If possible, I
may even replace the OS that comes on it or upgrade if I can. I'm not looking for fancy, or even RAID. Just looking for a two bay NAS that
will work. First, what is a DAS? Is that totally different than a
NAS? From what I've found, a DAS is not what I'm looking for since I
want a ethernet connection and the ability to control things over the network. It seems DAS lacks that feature but not real sure. I'm not
sure I can upgrade the software/OS on a DAS either.
Next thing. Let's say a NAS comes with two 4TB drives for a total of
8TB of capacity from the factory, using LVM or similar software I
assume. Is that limited to that capacity or can I for example replace
one or both drives with for example 14TB drives for a total of 28TBs of capacity? If one does that, let's say it uses LVM, can I somehow move
data as well or is that beyond the abilities of a NAS? Could it be done inside my computer for example? Does this vary by brand or even model?
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well. The
old slower models with small amounts of ram don't fair as well. While I
want a descent speed, I'm not looking for or expecting it to be
blazingly fast. I just wonder, if from a upgrade and expansion point of view, if building a NAS would be better. I've also noticed, it seems
all Raspberry things come with a display port. That means I could hook
up a monitor and mouse/keyboard when needed. That could be a bonus.
Heck, I may can even put some sort of Gentoo on that thing. :-D
One reason I'm wanting to go this route, I'm trying to keep it small and
able to fit inside my fire safe. I plan to buy a media type safe that
is larger but right now, it needs to fit inside my current safe. Most
of the 2 bay NAS or a Raspberry Pi based NAS are fairly small. They not
much bigger than the three external hard drives and a couple bare drives
that currently occupy my safe.
One thing I'd like to have no matter what path I go down, the ability to encrypt the data. My current backup drives are encrypted and I'd like
to keep it that way. If that is possible to do. I suspect the
Raspberry option would since I'd control the OS/software placed on it.
I could be wrong tho.
One last thing. Are there any NAS type boxes that I should absolutely
avoid if I go that route? Maybe it is a model that has serious
limitations or has other problems. I think the DAS thing may be one for
me to avoid but I'm not for sure what limits it has. Google didn't help
a lot. It also could be as simple as, avoid any model that says this in
the description or uses some type of software that is bad or limits
options.
Thoughts? Info to share? Ideas on a best path forward? Buy already
built or build?
Thanks.
Dale
:-) :-)
Howdy,
I've pretty much reached a limit on my backups. I'm up to a 16TB hard
drive for one and even that won't last long. Larger drives are much
more costly. A must have NAS is quickly approaching.
Path one, buy a NAS, possibly used, that has no drives. If possible, I
may even replace the OS that comes on it or upgrade if I can.
I'm not looking for fancy, or even RAID. Just looking for a two bay NAS that will work.
First, what is a DAS? Is that totally different than a NAS? From what I've found, a DAS is not what I'm looking for since I want a ethernet connection and the ability to control things over the network.
It seems DAS lacks that feature but not real sure. I'm not sure I can upgrade the software/OS on a DAS either.
Next thing. Let's say a NAS comes with two 4TB drives for a total of
8TB of capacity from the factory, using LVM or similar software I
assume.
Is that limited to that capacity or can I for example replace one or both drives with for example 14TB drives for a total of 28TBs of capacity?
If one does that, let's say it uses LVM, can I somehow move data as well
or is that beyond the abilities of a NAS?
Could it be done inside my computer for example?
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well.
The old slower models with small amounts of ram don't fair as well. While
I want a descent speed, I'm not looking for or expecting it to be
blazingly fast.
I just wonder, if from a upgrade and expansion point of view, if building
a NAS would be better.
I've also noticed, it seems all Raspberry things come with a display port.
That means I could hook up a monitor and mouse/keyboard when needed. That could be a bonus. Heck, I may can even put some sort of Gentoo on that thing. :-D
One reason I'm wanting to go this route, I'm trying to keep it small and
able to fit inside my fire safe.
I plan to buy a media type safe that is larger but right now, it needs to
fit inside my current safe.
One thing I'd like to have no matter what path I go down, the ability to encrypt the data. My current backup drives are encrypted and I'd like
to keep it that way. If that is possible to do.
I suspect the Raspberry option would since I'd control the OS/software
placed on it. I could be wrong tho.
One last thing. Are there any NAS type boxes that I should absolutely
avoid if I go that route?
I think the DAS thing may be one for me to avoid but I'm not for sure what limits it has.
Thoughts? Info to share? Ideas on a best path forward? Buy already built or build?
On Thu, Dec 8, 2022 at 5:38 AM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
Howdy,
I've pretty much reached a limit on my backups. I'm up to a 16TB hard drive for one and even that won't last long. Larger drives are much
more costly. A must have NAS is quickly approaching. I've been searching around and find some things confusing. I'm hoping someone can clear up that confusion. I'm also debating what path to travel down.
I'd also like to keep costs down as well. That said, I don't mind
paying a little more for one that would offer a much better option.
Path one, buy a NAS, possibly used, that has no drives. If possible, I may even replace the OS that comes on it or upgrade if I can. I'm not looking for fancy, or even RAID. Just looking for a two bay NAS that
will work. First, what is a DAS? Is that totally different than a NAS? From what I've found, a DAS is not what I'm looking for since I
want a ethernet connection and the ability to control things over the network. It seems DAS lacks that feature but not real sure. I'm not sure I can upgrade the software/OS on a DAS either.
Next thing. Let's say a NAS comes with two 4TB drives for a total of
8TB of capacity from the factory, using LVM or similar software I
assume. Is that limited to that capacity or can I for example replace
one or both drives with for example 14TB drives for a total of 28TBs of capacity? If one does that, let's say it uses LVM, can I somehow move data as well or is that beyond the abilities of a NAS? Could it be done inside my computer for example? Does this vary by brand or even model?
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well. The
old slower models with small amounts of ram don't fair as well. While I want a descent speed, I'm not looking for or expecting it to be
blazingly fast. I just wonder, if from a upgrade and expansion point of view, if building a NAS would be better. I've also noticed, it seems
all Raspberry things come with a display port. That means I could hook
up a monitor and mouse/keyboard when needed. That could be a bonus.
Heck, I may can even put some sort of Gentoo on that thing. :-D
One reason I'm wanting to go this route, I'm trying to keep it small and able to fit inside my fire safe. I plan to buy a media type safe that
is larger but right now, it needs to fit inside my current safe. Most
of the 2 bay NAS or a Raspberry Pi based NAS are fairly small. They not much bigger than the three external hard drives and a couple bare drives that currently occupy my safe.
One thing I'd like to have no matter what path I go down, the ability to encrypt the data. My current backup drives are encrypted and I'd like
to keep it that way. If that is possible to do. I suspect the
Raspberry option would since I'd control the OS/software placed on it.
I could be wrong tho.
One last thing. Are there any NAS type boxes that I should absolutely avoid if I go that route? Maybe it is a model that has serious limitations or has other problems. I think the DAS thing may be one for me to avoid but I'm not for sure what limits it has. Google didn't help
a lot. It also could be as simple as, avoid any model that says this in
the description or uses some type of software that is bad or limits options.
Thoughts? Info to share? Ideas on a best path forward? Buy already built or build?
Thanks.
Dale
:-) :-)
DAS is direct-attached-storage. I don't think you want that.
Synology (sp?) is sort of a big name in home & small office NAS boxes.
You can buy the boxes with or without drives. I suspect you won't like
the prices.
I wonder if you might consider what data on your backups needs to be immediately available and which doesn't. Possibly buy an 8TB USB
drive, take a bunch of the lower priority data off of your current
backup thus system freeing space and move on from there?
I built my NAS devices using old computers ala Wol's suggestion to me
maybe a year ago. They work for me but don't have the fastest network interfaces.
Raspberry Pi 4 B's are hard to get and expensive right now. Still,
they are nice little devices but you would probably be limited to USB
hard drive storage.
You could, but this is either a sink-hole for time, or you need to get up to speed with cross-compiling and binhosts. I went with the standard Debian and evaluate Arch from time to time. But I do run Gentoo on my DIY NAS with an i3-2000. Gentoo has ZFS in portage without overlays, which–for me–is one of
its biggest appeals.
Mark Knecht wrote:<SNIP>
DAS is direct-attached-storage. I don't think you want that.
I was thinking DAS was not a good option. It seems like a featureremoved and cheaper version of NAS.
<SNIP><br>> I was thinking DAS was not a good option. It seems like a feature removed and cheaper version of NAS. <br> <div><br></div><div>I've never touched a DAS box but my limited understanding is that it's an</div><div>externalbox that interfaces to your computer. In your case that might</div><div>attached to your backup machine, or is Rich or someone can point</div><div>you toward an appropriate RP4 or some other single board computer</div><div>it could attach to that. The
On Thu, Dec 8, 2022 at 5:38 AM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
Howdy,
I've pretty much reached a limit on my backups. I'm up to a 16TB hard drive for one and even that won't last long. Larger drives are much
more costly. A must have NAS is quickly approaching. I've been searching around and find some things confusing. I'm hoping someone can clear up that confusion. I'm also debating what path to travel down.
I'd also like to keep costs down as well. That said, I don't mind
paying a little more for one that would offer a much better option.
Path one, buy a NAS, possibly used, that has no drives. If possible, I may even replace the OS that comes on it or upgrade if I can. I'm not looking for fancy, or even RAID. Just looking for a two bay NAS that will work. First, what is a DAS? Is that totally different than a NAS? From what I've found, a DAS is not what I'm looking for since I want a ethernet connection and the ability to control things over the network. It seems DAS lacks that feature but not real sure. I'm not sure I can upgrade the software/OS on a DAS either.
Next thing. Let's say a NAS comes with two 4TB drives for a total of
8TB of capacity from the factory, using LVM or similar software I assume. Is that limited to that capacity or can I for example replace one or both drives with for example 14TB drives for a total of 28TBs of capacity? If one does that, let's say it uses LVM, can I somehow move data as well or is that beyond the abilities of a NAS? Could it be done inside my computer for example? Does this vary by brand or even model?
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well. The
old slower models with small amounts of ram don't fair as well. While I want a descent speed, I'm not looking for or expecting it to be
blazingly fast. I just wonder, if from a upgrade and expansion point of view, if building a NAS would be better. I've also noticed, it seems
all Raspberry things come with a display port. That means I could hook up a monitor and mouse/keyboard when needed. That could be a bonus. Heck, I may can even put some sort of Gentoo on that thing. :-D
One reason I'm wanting to go this route, I'm trying to keep it small and able to fit inside my fire safe. I plan to buy a media type safe that
is larger but right now, it needs to fit inside my current safe. Most
of the 2 bay NAS or a Raspberry Pi based NAS are fairly small. They not much bigger than the three external hard drives and a couple bare drives that currently occupy my safe.
One thing I'd like to have no matter what path I go down, the ability to encrypt the data. My current backup drives are encrypted and I'd like
to keep it that way. If that is possible to do. I suspect the Raspberry option would since I'd control the OS/software placed on it.
I could be wrong tho.
One last thing. Are there any NAS type boxes that I should absolutely avoid if I go that route? Maybe it is a model that has serious limitations or has other problems. I think the DAS thing may be one for me to avoid but I'm not for sure what limits it has. Google didn't help a lot. It also could be as simple as, avoid any model that says this in the description or uses some type of software that is bad or limits options.
Thoughts? Info to share? Ideas on a best path forward? Buy already built or build?
Thanks.
Dale
:-) :-)
DAS is direct-attached-storage. I don't think you want that.
Synology (sp?) is sort of a big name in home & small office NAS boxes.
You can buy the boxes with or without drives. I suspect you won't like
the prices.
I wonder if you might consider what data on your backups needs to be immediately available and which doesn't. Possibly buy an 8TB USB drive,
take a bunch of the lower priority data off of your current backup thus system freeing space and move on from there?
I built my NAS devices using old computers ala Wol's suggestion to me
maybe a year ago. They work for me but don't have the fastest network interfaces.
<SNIP>
DAS is direct-attached-storage. I don't think you want that.
Depends. If it fits in the safe, and can be connected using one of these eSATA thingy connectors, it might be a very good choice.
Synology (sp?) is sort of a big name in home & small office NAS boxes.
You can buy the boxes with or without drives. I suspect you won't like
the prices.
I've been looking :-) I think the empty box costs more than the drives
you're going to put in it ...
I wonder if you might consider what data on your backups needs to be immediately available and which doesn't. Possibly buy an 8TB USB drive, take a bunch of the lower priority data off of your current backup thus system freeing space and move on from there?
I built my NAS devices using old computers ala Wol's suggestion to me
maybe a year ago. They work for me but don't have the fastest network interfaces.
I get the impression Dale isn't actually PLANNING his disk storage. It's
just a case of "help I'm downloading all this stuff where do I put it!!!"
How much storage do you have in your actual computer? How much space do
you need IN ONE PARTITION? Can you get an external disk caddy that you
just slot bare drives in?
> just a case of "help I'm downloading all this stuff where do I put it!!!"<br>><br>> How much storage do you have in your actual computer? How much space do<br>> you need IN ONE PARTITION? Can you get an external disk caddythat you<br>> just slot bare drives in?<br>></div><div><br></div><div>This is a good point. With planning he could have multiple 4TB type</div><div>drives with 2 or 3 slots and you plug in the drive you want to use</div><div>that day. It would take
I've pretty much reached a limit on my backups. I'm up to a 16TB hard drive for one and even that won't last long. Larger drives are much more costly. A must have NAS is quickly approaching. I've been searching around and find some things confusing. I'm hoping someone can
clear up that confusion. I'm also debating what path to travel down. I'd also like to keep costs down as well. That said, I don't mind paying a little more for one that would offer a much better option.
Path one, buy a NAS, possibly used, that has no drives. If possible, I may even replace the OS that comes on it or upgrade if I can. I'm not looking for fancy, or even RAID. Just looking for a two bay NAS that will work. First, what is a DAS? Is that totally different than a NAS? From what I've found, a DAS is not what I'm looking for since I want a ethernet connection and the ability to control things over the network. It seems DAS lacks that feature but not real sure. I'm not sure I can upgrade the software/OS on a DAS either.
[…]
DAS is direct-attached-storage. I don't think you want that.
Depends. If it fits in the safe, and can be connected using one of these eSATA thingy connectors, it might be a very good choice.
[…]
I get the impression Dale isn't actually PLANNING his disk storage. It's
just a case of "help I'm downloading all this stuff where do I put it!!!"
Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all
this stuff, and plug in and remove drives as required. You don't need all these huge drives if you think about what you're going to do with it all.
(And while it takes time and hammers the system, I regularly record off the TV getting a 2GB .ts file, convert it to mp4 - same resolution - and reduce the size by an order of magnitude - maybe more.
Pine64 has an interesting array of SBCs which are both cheaper and (some are) possibly better suited to becoming a NAS than a Pi. One of them even has a PCIe socket I think.
On 08/12/2022 13:31, Mark Knecht wrote:
On Thu, Dec 8, 2022 at 5:38 AM Dale <rdalek1967@gmail.com
<mailto:rdalek1967@gmail.com>> wrote:
;hard
Howdy,
;
I've pretty much reached a limit on my backups. I'm up to a 16TB
drive for one and even that won't last long. Larger drives are muchsomeone can
more costly. A must have NAS is quickly approaching. I've been
searching around and find some things confusing. I'm hoping
clear up that confusion. I'm also debating what path to travel down. >> > I'd also like to keep costs down as well. That said, I don't mindpossible, I
paying a little more for one that would offer a much better option.
;
Path one, buy a NAS, possibly used, that has no drives. If
may even replace the OS that comes on it or upgrade if I can. I'mnot
looking for fancy, or even RAID. Just looking for a two bay NAS that >> > will work. First, what is a DAS? Is that totally different than areplace
NAS? From what I've found, a DAS is not what I'm looking for since I >> > want a ethernet connection and the ability to control things over the
network. It seems DAS lacks that feature but not real sure. I'm not >> > sure I can upgrade the software/OS on a DAS either.
;
Next thing. Let's say a NAS comes with two 4TB drives for a total of >> > 8TB of capacity from the factory, using LVM or similar software I
assume. Is that limited to that capacity or can I for example
one or both drives with for example 14TB drives for a total of28TBs of
capacity? If one does that, let's say it uses LVM, can I somehowmove
data as well or is that beyond the abilities of a NAS? Could itbe done
inside my computer for example? Does this vary by brand or evenmodel?
;8GB as
Path two, I've researched building a NAS using a Raspberry Pi 4
another option. They come as parts, cases too, but the newer andfaster
models of Raspberry Pi 4 with more ram seem to work pretty well. The >> > old slower models with small amounts of ram don't fair as well.While I
want a descent speed, I'm not looking for or expecting it to bepoint of
blazingly fast. I just wonder, if from a upgrade and expansion
view, if building a NAS would be better. I've also noticed, it seems >> > all Raspberry things come with a display port. That means I couldhook
up a monitor and mouse/keyboard when needed. That could be a bonus.small and
Heck, I may can even put some sort of Gentoo on that thing. :-D
;
One reason I'm wanting to go this route, I'm trying to keep it
able to fit inside my fire safe. I plan to buy a media type safethat
is larger but right now, it needs to fit inside my current safe.Most
of the 2 bay NAS or a Raspberry Pi based NAS are fairly small.They not
much bigger than the three external hard drives and a couple baredrives
that currently occupy my safe.ability to
;
One thing I'd like to have no matter what path I go down, the
encrypt the data. My current backup drives are encrypted and I'dlike
to keep it that way. If that is possible to do. I suspect theit.
Raspberry option would since I'd control the OS/software placed on
I could be wrong tho.absolutely
;
One last thing. Are there any NAS type boxes that I should
avoid if I go that route? Maybe it is a model that has seriousone for
limitations or has other problems. I think the DAS thing may be
me to avoid but I'm not for sure what limits it has. Googledidn't help
a lot. It also could be as simple as, avoid any model that saysthis in
the description or uses some type of software that is bad or limits
options.
;
Thoughts? Info to share? Ideas on a best path forward? Buy already >> > built or build?
;
Thanks.
;
Dale
;
:-) :-)
DAS is direct-attached-storage. I don't think you want that.
Depends. If it fits in the safe, and can be connected using one of
these eSATA thingy connectors, it might be a very good choice.
Synology (sp?) is sort of a big name in home & small office NAS
boxes. You can buy the boxes with or without drives. I suspect you
won't like the prices.
I've been looking :-) I think the empty box costs more than the drives
you're going to put in it ...
I get the impression Dale isn't actually PLANNING his disk storage.
I wonder if you might consider what data on your backups needs to be
immediately available and which doesn't. Possibly buy an 8TB USB
drive, take a bunch of the lower priority data off of your current
backup thus system freeing space and move on from there?
I built my NAS devices using old computers ala Wol's suggestion to me
maybe a year ago. They work for me but don't have the fastest network
interfaces.
It's just a case of "help I'm downloading all this stuff where do I
put it!!!"
How much storage do you have in your actual computer? How much space
do you need IN ONE PARTITION? Can you get an external disk caddy that
you just slot bare drives in?
I've no doubt you have good reason for wanting all this storage. I
just fail to see why you need huge drives for it if most of the time
you're not doing anything with it.
Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting
all this stuff, and plug in and remove drives as required. You don't
need all these huge drives if you think about what you're going to do
with it all. (And while it takes time and hammers the system, I
regularly record off the TV getting a 2GB .ts file, convert it to mp4
- same resolution - and reduce the size by an order of magnitude -
maybe more.
If you've got two hot-swap JBOD enclosures, that's brilliant. You can
stream from your media centre to a drive, swap it out, and use a
second system to then organise your collection.
Oh - and if you are worried about disks going walkabout, just LUKS the
whole disk, and without the key nobody can read it ... build your
partitions or whatever over it.
Cheers,
Wol
On Thu, Dec 8, 2022 at 7:37 AM Dale <rdalek1967@gmail.com> wrote:
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB asFor this sort of application the key improvement of the Pi4 over its predecessors is IO. The Pi4 has USB3 and gigabit ethernet, and they
another option. They come as parts, cases too, but the newer and faster
models of Raspberry Pi 4 with more ram seem to work pretty well.
are independent, so you get the full bandwidth of both (in theory).
That is a massive step up over USB2 and 100Mbps ethernet that consumes
the USB2 bandwidth.
I can't really speak to the commercial solutions as I haven't used
them. Main concern there is just the limited capacity, lack of expandability, and so on. Some are no doubt better than others in
those regards.
As far as DIY goes, you can definitely do all of that with a Pi4.
Don't expect it to perform as well as sticking it on a decent amd64 motherboard, but for backup and saturating the throughput of 1 hard
drive at a time it can probably mostly make do. Encryption can be accomplished either with cryptsetup or a filesystem that has native encryption like ZFS. I've done both on Pi4s for storage. I will warn
you that zfs encryption is not hardware-optimized on ARM, so that will
not perform very well - it will be completely functional, but you will
get CPU-bound. Linux-native encryption (ie cryptsetup/LUKS) will use hardware capabilities on the Pi4, assuming you're using something it
supports (I think I'm using AES which performs adequately).
For the Pi4 you would need to use USB storage, but for hard drives IMO
this is perfectly acceptable, especially on a Pi. The gigabit
ethernet and internal IO of the Pi is only going to max out one hard
drive no matter how you connect it, so the USB3 interface will not be
a bottleneck. On ARM SBCs that have PCIe you don't really get any
better performance with an HBA and SATA/SCSI simply because the board
IO is already pretty limited. USB3 is actually pretty fast for
spinning disks, but depending on the number of hosts/etc it could
become a bottleneck on a decent motherboard with a large number of
drives. If you're talking about an amd64 with a 10GbE NIC and a
decent HBA with sufficient PCIe lanes for both then obviously that is
going to saturate more spinning disks. For NVMe you absolutely need
to go that route (probably need to consider server-class hardware
too).
I use USB3 hard drives on Pis for my bulk storage because I care about capacity far more than performance, and with a distributed filesystem
the performance is still good enough for what I'm doing. If I needed
block storage for containers/VMs/whatever then use a different
solution, but that gets expensive fast.
Oh, one other thing. One of your issues is that you're using a backup solution that just dumps everything into a single file/directory and
requires all the backup storage to be mounted at the same time in a
single filesystem. There are solutions that do not have this
requirement - particularly ones that are adaptable to tape.
Unfortunately the best FOSS option I've found for this on linux is
bacula and that is a serious PITA to use. If anybody has a better one
I'm all ears (the requirement is to be able to store a backup across
multiple hard drives, and this can't involve first storing it all in
one place and then splitting it up later, or having more than one
storage drive attached at the same time - basically I want to treat
hard drives like tapes).
If you're storing a LOT of backups then LTO is another option. Every
time I do the math on that option it never makes sense unless you're
backing up a LOT of data. If you got to a point where your backups
consumed 10+ max-capacity hard drives it might start to make sense.
Those USB3 hard drives on sale for $15/TB though are just really hard
to beat when the tapes aren't all that much cheaper and the drives
cost $1k.
I was thinking DAS was not a good option. It seems like a feature
removed and cheaper version of NAS.
I think I've seen a couple Synology NAS boxes but I think even used they
were a bit pricey. Still, used could make that a option. Maybe. It
could fall into the category of pay a little more for a much better
option, even if it is used.
I've considered using older systems I have for NAS but they are large.
Way to large. It would require a lot of effort to shrink them down if
it is even possible. A NAS is smaller and designed for what I need as well. This is what I found that goes with the Raspberry Pi.
https://shop.allnetchina.cn/collections/sata-hat/products/dual-sata-hat-open-frame-for-raspberry-pi-4
There is a two bay and a four bay version. I think the case is the same
for both so I may go with four for future expansion. Price isn't bad
for that part but as you say, Raspberry Pi board is a bit pricey. Thing
is, given the amount of control I'd have over it, it could be a better
option long term. I might add, I think this board is somewhat new. I meant to include a link to it but forget. It could be that you are not aware of that, or many other people either. Also, I'd like to buy it
from a more local vendor. I've bought things from China through Ebay
but it has a guarantee and refund option that is fairly good. It's a
option I've had to exercise a time or two.
Part of me wants to buy a used but well featured NAS box. Part of me
thinks a Raspberry would be better and have upgrade options in the
future as well. I'm pretty sure I could have encryption on a Raspberry
NAS as well. I'm not sure if a prebuilt NAS box has encryption or not.
Hope for some good ideas tho. I'd like to avoid buying something that
won't come close to serving even current needs or just plain doesn't work.
Dale
:-) :-)
One thing I like about the Raspberry option, I can upgrade it later. I
can simply take out the old, put in new, upgrade done. If I buy a
prebuilt NAS, they pretty much are what they are if upgrading isn't a
option. Some of the more expensive ones may be upgradable, maybe.
I just wonder, could I use that board and just hook it to my USB port
and a external power supply and skip the Raspberry Pi part? I'd bet not
tho. ;-)
I use USB3 hard drives on Pis for my bulk storage because I care about capacity far more than performance, and with a distributed filesystem
the performance is still good enough for what I'm doing. If I needed
block storage for containers/VMs/whatever then use a different
solution, but that gets expensive fast.
[…]
From my understanding, you are right about USB3 and GB ethernet being
the big change. They also have more memory and faster CPUs but if you bottleneck the data with slow USB and ethernet with the old ones, who
needs a fast CPU? I think they realized that the USB and ethernet had
to improve. It got better from there.
https://shop.allnetchina.cn/collections/sata-hat/products/dual-sata-hat-open-frame-for-raspberry-pi-4
I found the above. From my understanding, it allows a SATA drive to
connect to either 2 or 4 bays.
One thing I like about the Raspberry option, I can upgrade it later. I
can simply take out the old, put in new, upgrade done. If I buy a
prebuilt NAS, they pretty much are what they are if upgrading isn't a
option.
I just wonder, could I use that board and just hook it to my USB port
and a external power supply and skip the Raspberry Pi part? I'd bet not tho. ;-)
On Thu, Dec 8, 2022 at 6:30 PM Dale <rdalek1967@gmail.com> wrote:
One thing I like about the Raspberry option, I can upgrade it later. IThe NAS gets you a nice box. The nice box means fixed capacity.
can simply take out the old, put in new, upgrade done. If I buy a
prebuilt NAS, they pretty much are what they are if upgrading isn't a
option. Some of the more expensive ones may be upgradable, maybe.
I just use USB3 external hard drives. They're cheaper and easy to
interface. USB3 also has been less likely to give me ATA interface
errors compared to SATA.
I just wonder, could I use that board and just hook it to my USB portNot that one, but USB3-SATA interfaces exist and aren't that
and a external power supply and skip the Raspberry Pi part? I'd bet not
tho. ;-)
expensive. You can also get nice little enclosures. You can have as
many hard drives as you want on a PC that way, or whatever the USB3
limit is.
I think back sometimes, I started out with a 30GB hard drive waaaay back
in 2003. I thought I had problems then.
On Friday, 9 December 2022 00:03:29 GMT Dale wrote:
I think back sometimes, I started out with a 30GB hard drive waaaay backThen you won't want to know that I paid extra in 1990 for an 85MB drive in my first PC. No, not GB: MB.
in 2003. I thought I had problems then.
On 12/8/22 05:58, Dale wrote:
I was thinking DAS was not a good option. It seems like a feature
removed and cheaper version of NAS.
I think I've seen a couple Synology NAS boxes but I think even used
they were a bit pricey. Still, used could make that a option.
Maybe. It could fall into the category of pay a little more for a
much better option, even if it is used.
I've considered using older systems I have for NAS but they are
large. Way to large. It would require a lot of effort to shrink
them down if it is even possible. A NAS is smaller and designed for
what I need as well. This is what I found that goes with the
Raspberry Pi.
https://shop.allnetchina.cn/collections/sata-hat/products/dual-sata-hat-open-frame-for-raspberry-pi-4
There is a two bay and a four bay version. I think the case is the
same for both so I may go with four for future expansion. Price
isn't bad for that part but as you say, Raspberry Pi board is a bit
pricey. Thing is, given the amount of control I'd have over it, it
could be a better option long term. I might add, I think this board
is somewhat new. I meant to include a link to it but forget. It
could be that you are not aware of that, or many other people
either. Also, I'd like to buy it from a more local vendor. I've
bought things from China through Ebay but it has a guarantee and
refund option that is fairly good. It's a option I've had to
exercise a time or two.
Part of me wants to buy a used but well featured NAS box. Part of me
thinks a Raspberry would be better and have upgrade options in the
future as well. I'm pretty sure I could have encryption on a
Raspberry NAS as well. I'm not sure if a prebuilt NAS box has
encryption or not.
Hope for some good ideas tho. I'd like to avoid buying something
that won't come close to serving even current needs or just plain
doesn't work.
Dale
:-) :-)
Dale,
DAS is direct attached storage. If your intention is to share the data
with multiple devices for backup you will need to keep in mind that
you will need a PC to share the data the DAS device is storing. In
general, most DAS require some sort of HBA (some of these HBAs can be hundreds to thousands of dollars.) I've seen some eSata ones but they
usually don't have stellar reviews. It's also getting harder to find
eSata devices.
In contrast NAS devices are designed to plug in to the network and be
shared with multiple devices on the LAN right from the get-go.
You are probably interested in a NAS, not a DAS.
I have an aging ix4-300d NAS. The display has started crashing now but
the device is still rock solid. However I'm in the same boat and have
been researching options - I think for my case I will get a small cube
case and mini-ITX board and roll my own Gentoo install. The cost may
even be slightly cheaper as 4 bay NAS here are quite expensive where I
am (with no drives installed) and building my own will be a bit
cheaper and I can choose what drives to run in it. Vulnerabilities on
devices like QNAP and Synology are very real and at least if you can
roll your own you can keep that to a minimum (like an example not
running a web browser to configure things.)
Dan
Am Thu, Dec 08, 2022 at 05:30:18PM -0600 schrieb Dale:
Looking at the pics, it looks all very wibbly-wobbly. You will either have the parts lying around open on a desk or you need to find a case for allI use USB3 hard drives on Pis for my bulk storage because I care aboutFrom my understanding, you are right about USB3 and GB ethernet being
capacity far more than performance, and with a distributed filesystem
the performance is still good enough for what I'm doing. If I needed
block storage for containers/VMs/whatever then use a different
solution, but that gets expensive fast.
[…]
the big change. They also have more memory and faster CPUs but if you
bottleneck the data with slow USB and ethernet with the old ones, who
needs a fast CPU? I think they realized that the USB and ethernet had
to improve. It got better from there.
https://shop.allnetchina.cn/collections/sata-hat/products/dual-sata-hat-open-frame-for-raspberry-pi-4
I found the above. From my understanding, it allows a SATA drive to
connect to either 2 or 4 bays.
that stuff which adheres to no industry standard form factor. Pi accessories are quite hard to come by, since they’re often sold out.
One thing I like about the Raspberry option, I can upgrade it later. IIf you just do storage, what do you need upgrades for, anyway? All it needs to do is receive your data and write it to disk. And then return it later when asked for. I don’t remember you mentioning running VMs or some such. Any current commercial NAS has enough oomph for that, unless it’s a very cheap ARM-based one. (Only the ecryption part remains to be solved with a ready-made NAS.)
can simply take out the old, put in new, upgrade done. If I buy a
prebuilt NAS, they pretty much are what they are if upgrading isn't a
option.
I just wonder, could I use that board and just hook it to my USB portFrom a practical standpoint, what is the difference then to an HDD dock or a simple USB-SATA-Adapter? Except that a dock is a “proper”, clean solution with a nice case, a secure stand on your desk and no finnicky open SATA cables that could cause disconnects during operation if you touch them the wrong way.
and a external power supply and skip the Raspberry Pi part? I'd bet not
tho. ;-)
I know what it’s like to ponder all kinds of options, and it’s fun. But it
seems to me, you’re looking for a solution for a problem you’re still looking for.
I just wonder, could I use that board and just hook it to my USB port
and a external power supply and skip the Raspberry Pi part? I'd bet not tho. ;-)
Dale
:-) :-)
Given the size of one of the directories I have, it takes two drives, or
soon will, and the use of LVM or something similar. I can't do that as
it is now. I've even wondered if I hooked two eSATA drives up and gave
both plenty of time to spin up if LVM would see them both and me be able
to use two drives as one that way. Thing is, I don't know how LVM
reacts if the two drives become available at separate times, maybe even
many seconds or a minute or so apart.
My problem is a growing directory. I admit, It's not increasing as fast
as it was. When I was on DSL, it limited my speed a lot. With this new fiber internet, I can download huge amounts of data in a really short
period of time. I can download it faster than I can verify it. I'm
still checking things I downloaded over a month ago. I'm having fun
doing it tho. 😉
I thought about breaking up that huge directory. Split it into two
parts, the 'a' through 'l' and 'm' through 'z' thing. Then use two
drives to back it up.
Thing is, the external drive enclosures that I
really like and trust, I can't buy anymore. They are Rosewill eSATA
drive enclosures. It has a fan to keep things cool and a display on the front. They are really nice and rock solid. All the USB type drive enclosures I've tried caused all sorts of problems. I bricked a couple
hard drives and eventually, the enclosures wouldn't work at all. The Rosewill enclosures are the most stable things I've ever seen. I wish I could find a few more of them, as spares if nothing else.
I think back sometimes, I started out with a 30GB hard drive waaaay back
in 2003. I thought I had problems then. O_O
On 09/12/2022 01:15, Dale wrote:
Given the size of one of the directories I have, it takes two drives, or
soon will, and the use of LVM or something similar. I can't do that as
it is now. I've even wondered if I hooked two eSATA drives up and gave
both plenty of time to spin up if LVM would see them both and me be able
to use two drives as one that way. Thing is, I don't know how LVM
reacts if the two drives become available at separate times, maybe even
many seconds or a minute or so apart.
If you're using LVM to link them together, it will wait until they
become available. Okay, not quite the same, but I run raid over
dm-integrity, and it always unnerves me when systemd fires up this job
and it says "waiting for lvm/home". But the system just sits there
while dm-integrity checks its drives, makes them available, raid spots
and loads them, and then the raid is there, lvm spots it, makes
lvm/home available, and the system is up and running ...
Cheers,
Wol
On 09/12/2022 00:45, Dale wrote:
I even remember when 512KBs of ram was a big deal. I also remember
having expansion cards that would add a few MBs of ram. Jeez I'm
getting old. o_O We talk about TBs like they are nothing. My first
puter was a old Vic-20. 4Kbs of ram it had. I played music on that
thing and freaked my Dad out. ROFL
I remember those things. About 16 MB per platter. I remember my work
buying a 300MB drive (19 platters in a disk pack, the size of a washing machine) for our multi-user mini that served the entire company with
256KB of ram ...
I get the impression Dale isn't actually PLANNING his disk storage. It's just a case of "help I'm downloading all this stuff where do I put it!!!"
Haha, thanks for the laugh.
Actually this had me thinking what is the need to back up the ... Internet? […]
I appreciate some of these video files may be rare finds, or there may be a risk some of these may be taken off the interwebs sooner or later. This should leave a rather small subset of all downloads, which may merit a local backup, just in case. I'd thought the availability of higher fiber download speeds negates the need for local backups, of readily downloadable media.
Well, ts uses mpeg2 encoding, just like old video DVDs, which is very inefficient when compared with modern h264/h265. Modern digital TV broadcast
uses h264 by now.
Depending on the PVR make/model I've seen 1080p resolution recordings with .m2ts and .ts file extensions, while the codecs inside them are the same.
¹ I do have several external USB disks, plus the big NAS. All of which don’t
run very often. And I don’t want to turn them on just to look for a certain
file. That’s why I have another little script. ;-) It uses the `tree` command to save the complete content listing of a directory into a text file and names the file automatically by the name of the directory it crawls. So if I want to find a file, I just need to grep through my text files.
Backup scripts utilising rsync, tar, etc. can output a log file which contains
(some) details of all the backed up files. Nothing as sophisticated as Frank's script, but it allows for a quick search against the name of the file or directory, before extraction.
Am Thu, Dec 08, 2022 at 06:36:14PM +0000 schrieb Wols Lists:
I've pretty much reached a limit on my backups. I'm up to a 16TB
hard
drive for one and even that won't last long. Larger drives are much more costly. A must have NAS is quickly approaching. I've been searching around and find some things confusing. I'm hoping someone can
clear up that confusion. I'm also debating what path to travel down. I'd also like to keep costs down as well. That said, I don't mind paying a little more for one that would offer a much better option.
Path one, buy a NAS, possibly used, that has no drives. If possible, I
may even replace the OS that comes on it or upgrade if I can. I'm
not
looking for fancy, or even RAID. Just looking for a two bay NAS that will work. First, what is a DAS? Is that totally different than a NAS? From what I've found, a DAS is not what I'm looking for since I want a ethernet connection and the ability to control things over the network. It seems DAS lacks that feature but not real sure. I'm not sure I can upgrade the software/OS on a DAS either.
[…]
DAS is direct-attached-storage. I don't think you want that.
Depends. If it fits in the safe, and can be connected using one of these eSATA thingy connectors, it might be a very good choice.
[…]
I get the impression Dale isn't actually PLANNING his disk storage. It's just a case of "help I'm downloading all this stuff where do I put it!!!"
Haha, thanks for the laugh.
Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all this stuff, and plug in and remove drives as required. You don't need all these huge drives if you think about what you're going to do with it all.
That’s actually a good idea. Either use a hot swap frame for an internal 5¼″
PC bay, a desktop dock for bare drives or a multi-bay enclosure. The market is big, you have lots of choices. USB (with or without integrated hub), eSATA, one or two bays, etc: https://skinflint.co.uk/?cat=hddocks
Advantages:
- no separate system to maintain just for storage: save $$$, time and power
- very flexible: no chassis limitation on number of disks
- no bulky external enclosures, each using a different power brick and cable - minimum volume to put into a safe (just get or make a bulk storage case)
Disadvantages:
- not as “fancy” as a NAS
- possibly not all disks can be used at the same time
- physical handling of naked disks takes more care
- LVM is not practical, so use each disk separately
- you gotta remember which files are where¹
- SATA connectors aren’t made for very many insertion cycles (I think the
spec says 50?), which doesn’t mean they endure much more, but still …
(And while it takes time and hammers the system, I regularly record off
the
TV getting a 2GB .ts file, convert it to mp4 - same resolution - and
reduce
the size by an order of magnitude - maybe more.
Well, ts uses mpeg2 encoding, just like old video DVDs, which is very inefficient when compared with modern h264/h265. Modern digital TV broadcast uses h264 by now.
Incidentally, I got myself a new HDD today: an external 2.5″ WD Passport Ultra 5 TB with USB-C 3.0. Just because I like portable storage and also because I need temporary space if I want to convert my NAS RAID-Z2 to Z1.
¹ I do have several external USB disks, plus the big NAS. All of which don’t
run very often. And I don’t want to turn them on just to look for a certain file. That’s why I have another little script. ;-) It uses the `tree` command to save the complete content listing of a directory into a text
file and names the file automatically by the name of the directory it
crawls. So if I want to find a file, I just need to grep through my text files.
Howdy,
<<<SNIP>>>
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well. The
old slower models with small amounts of ram don't fair as well. While I want a descent speed, I'm not looking for or expecting it to be
blazingly fast. I just wonder, if from a upgrade and expansion point of view, if building a NAS would be better. I've also noticed, it seems
all Raspberry things come with a display port. That means I could hook
up a monitor and mouse/keyboard when needed. That could be a bonus.
Heck, I may can even put some sort of Gentoo on that thing. :-D
<<<SNIP>>>
Thoughts? Info to share? Ideas on a best path forward? Buy already built or build?
Thanks.
Dale
:-) :-)
Actually this had me thinking what is the need to back up the ... Internet?
Am Fri, Dec 09, 2022 at 01:13:50PM +0000 schrieb Michael:
Good points. I am a big fan of having stuff locally as well, because I dont want to be dependent on a companys servers and a working Internet connection.
But this mostly applies to my mobile device, because I dont have a data plan for mobile Internet.
I was thinking DAS was not a good option. It seems like a feature
removed and cheaper version of NAS.
Depending on the PVR make/model I've seen 1080p resolution recordings with >> .m2ts and .ts file extensions, while the codecs inside them are the same.
I wasn’t aware that ts could contain h264. But then again—I never really bothered with live TV recordings in recent years. These days, if I find something interesting, I download the show form the TV channel’s website (called Mediathek in Germany, a word play on Bibliothek, meaning library). Interestingly though, the picture quality is noticably worse than what I receive via DVB-T.
-------------------------------------^^^^^^^^^^^^^^^^^^^^^^^Depending on the PVR make/model I've seen 1080p resolution recordings with
.m2ts and .ts file extensions, while the codecs inside them are the same.
---------------------------------^^^^^^^I wasn’t aware that ts could contain h264. But then again—I never really
bothered with live TV recordings in recent years.
I think this is confusing CONTAINER and CODEC.
.ts is a container format, h264 is a codec. I don't understand it myself, either but think of ts as your directory structure and h264 as your file structure.
Incidentally, sticking this stuff in a .tar is probably okay - that's just another container, but sticking it in a .tar.gz is not, the gz is your codec and will make the file BIGGER in all probability.
Where do we confuse those two? We specifically talked of codecs and “contain”.
On 10/12/2022 16:19, Frank Steinmetzger wrote:
Where do we confuse those two? We specifically talked of codecs and “contain”.
"I didn't know .ts could contain h264".
If .ts is the container, then surely the assumption is it can contain
any codec? If not, why not?
On Saturday, 10 December 2022 16:30:03 GMT Wols Lists wrote:
On 10/12/2022 16:19, Frank Steinmetzger wrote:
Where do we confuse those two? We specifically talked of codecs and “contain”.
"I didn't know .ts could contain h264".
If .ts is the container, then surely the assumption is it can contain
any codec? If not, why not?
Not any codec. Some container formats are only compatible with certain codecs, or rather the other way around. Have a look here:
https://en.wikipedia.org/wiki/Comparison_of_video_container_formats
I suppose the answer to 'why not' boils down to the whatever structure and data the container format is designed to be compatible with, but I don't
know
more than this.
https://en.wikipedia.org/wiki/File:Informatik-Containerformate-Beispiele.svg
Howdy,
I've pretty much reached a limit on my backups. I'm up to a 16TB hard
drive for one and even that won't last long. Larger drives are much
more costly. A must have NAS is quickly approaching. I've been
searching around and find some things confusing. I'm hoping someone can clear up that confusion. I'm also debating what path to travel down.
I'd also like to keep costs down as well. That said, I don't mind
paying a little more for one that would offer a much better option.
Path one, buy a NAS, possibly used, that has no drives. If possible, I
may even replace the OS that comes on it or upgrade if I can. I'm not looking for fancy, or even RAID. Just looking for a two bay NAS that
will work. First, what is a DAS? Is that totally different than a
NAS? From what I've found, a DAS is not what I'm looking for since I
want a ethernet connection and the ability to control things over the network. It seems DAS lacks that feature but not real sure. I'm not
sure I can upgrade the software/OS on a DAS either.
Next thing. Let's say a NAS comes with two 4TB drives for a total of
8TB of capacity from the factory, using LVM or similar software I
assume. Is that limited to that capacity or can I for example replace
one or both drives with for example 14TB drives for a total of 28TBs of capacity? If one does that, let's say it uses LVM, can I somehow move
data as well or is that beyond the abilities of a NAS? Could it be done inside my computer for example? Does this vary by brand or even model?
Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as another option. They come as parts, cases too, but the newer and faster models of Raspberry Pi 4 with more ram seem to work pretty well. The
old slower models with small amounts of ram don't fair as well. While I want a descent speed, I'm not looking for or expecting it to be
blazingly fast. I just wonder, if from a upgrade and expansion point of view, if building a NAS would be better. I've also noticed, it seems
all Raspberry things come with a display port. That means I could hook
up a monitor and mouse/keyboard when needed. That could be a bonus.
Heck, I may can even put some sort of Gentoo on that thing. :-D
One reason I'm wanting to go this route, I'm trying to keep it small and
able to fit inside my fire safe. I plan to buy a media type safe that
is larger but right now, it needs to fit inside my current safe. Most
of the 2 bay NAS or a Raspberry Pi based NAS are fairly small. They not much bigger than the three external hard drives and a couple bare drives
that currently occupy my safe.
One thing I'd like to have no matter what path I go down, the ability to encrypt the data. My current backup drives are encrypted and I'd like
to keep it that way. If that is possible to do. I suspect the
Raspberry option would since I'd control the OS/software placed on it.
I could be wrong tho.
One last thing. Are there any NAS type boxes that I should absolutely
avoid if I go that route? Maybe it is a model that has serious
limitations or has other problems. I think the DAS thing may be one for
me to avoid but I'm not for sure what limits it has. Google didn't help
a lot. It also could be as simple as, avoid any model that says this in
the description or uses some type of software that is bad or limits options.
Thoughts? Info to share? Ideas on a best path forward? Buy already built or build?
Thanks.
Dale
:-) :-)
It does appear that several people are making it so NAS boxes can be
easily built by us nerdy types and not be huge. Things are a bit
interesting right now for hardware and such. Some parts are expensive,
hard to find or just plain unavailable. Still, I suspect that this will
get better later on. The good thing, people are working on this. There
is interest in having a option.
<div>so there was a little learning to do but it wasn't bad.</div><div><br></div><div>Mine has a small SSD as the boot drive and then RAID1 pairs</div><div>running OpenZFS for storage. It's inexpensive if you have an </div><div>old computerto build on.</div><div><br></div><div>Good luck,</div><div>Mark</div></div>
On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
<SNIP>
It does appear that several people are making it so NAS boxes can be
easily built by us nerdy types and not be huge. Things are a bit interesting right now for hardware and such. Some parts are expensive, hard to find or just plain unavailable. Still, I suspect that this will get better later on. The good thing, people are working on this. There
is interest in having a option.
I know I've said this before but yes, people are working on it and it
is buildable by people like us.
https://www.truenas.com/truenas-core/
Any old x64 PC and a few disks will get you up and running. It's
open source as much as BSD is open source. It's not Linux
so there was a little learning to do but it wasn't bad.
Mine has a small SSD as the boot drive and then RAID1 pairs
running OpenZFS for storage. It's inexpensive if you have an
old computer to build on.
Good luck,
Mark
Dave
On Sat, Dec 10, 2022, 4:28 PM Mark Knecht <markknecht@gmail.com> wrote:
On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
It does appear that several people are making it so NAS boxes can be
easily built by us nerdy types and not be huge. Things are a bit
interesting right now for hardware and such. Some parts are expensive,
hard to find or just plain unavailable. Still, I suspect that this will >> > get better later on. The good thing, people are working on this. There
is interest in having a option.
I know I've said this before but yes, people are working on it and it
is buildable by people like us.
https://www.truenas.com/truenas-core/
Any old x64 PC and a few disks will get you up and running. It's
open source as much as BSD is open source. It's not Linux
so there was a little learning to do but it wasn't bad.
Mine has a small SSD as the boot drive and then RAID1 pairs
running OpenZFS for storage. It's inexpensive if you have an
old computer to build on.
Good luck,
Mark
On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
It does appear that several people are making it so NAS boxes can be
easily built by us nerdy types and not be huge. Things are a bit interesting right now for hardware and such. Some parts are expensive, hard to find or just plain unavailable. Still, I suspect that this will get better later on. The good thing, people are working on this. There
is interest in having a option.
I know I've said this before but yes, people are working on it and it
is buildable by people like us.
https://www.truenas.com/truenas-core/
Any old x64 PC and a few disks will get you up and running. It's
open source as much as BSD is open source. It's not Linux
so there was a little learning to do but it wasn't bad.
Mine has a small SSD as the boot drive and then RAID1 pairs
running OpenZFS for storage. It's inexpensive if you have an
old computer to build on.
Good luck,
Mark
I know I've said this before but yes, people are working on it and it</div><div>is buildable by people like us.</div><div><br></div><div><a href="https://www.truenas.com/truenas-core/" target="_blank" rel="noreferrer">https://www.truenas.com/truenas-core/</a><br></div><div><br></div><div>Any old x64 PC and a few disks will get you up and running. It's </div><div>open source as much as BSD is open source. It's not Linux</div><div>so there was a little learning to do but it wasn't
</div></blockquote></div>
I got a old rig I can use. I actually burned OpenNAS, TrueNAS or FreeNASon a USB stick. I can't recall which one I put on it tho. I downloaded
I'll have to work with what I got for now but I really like the Raspberryoption for its size and good options to upgrade later. I'll just make do
Dale
:-) :-)
Tanks
Dave
On Sat, Dec 10, 2022, 9:46 PM David Rosenbaum <rosenbaumd181@gmail.com> wrote:
Dave
On Sat, Dec 10, 2022, 4:28 PM Mark Knecht <markknecht@gmail.com> wrote:
On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
It does appear that several people are making it so NAS boxes can bewill
easily built by us nerdy types and not be huge. Things are a bit
interesting right now for hardware and such. Some parts are expensive, >>> > hard to find or just plain unavailable. Still, I suspect that this
get better later on. The good thing, people are working on this. There >>> > is interest in having a option.
I know I've said this before but yes, people are working on it and it
is buildable by people like us.
https://www.truenas.com/truenas-core/
Any old x64 PC and a few disks will get you up and running. It's
open source as much as BSD is open source. It's not Linux
so there was a little learning to do but it wasn't bad.
Mine has a small SSD as the boot drive and then RAID1 pairs
running OpenZFS for storage. It's inexpensive if you have an
old computer to build on.
Good luck,
Mark
</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Dec 10, 2022, 9:46 PM David Rosenbaum <<a href="mailto:rosenbaumd181@gmail.com" target="_blank" rel="noreferrer">rosenbaumd181@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><br><br><div data-smartmail="gmail_signature">Dave</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Dec
Am Sat, Dec 10, 2022 at 09:20:17AM +0000 schrieb Wols Lists:
recordings withDepending on the PVR make/model I've seen 1080p resolution
same..m2ts and .ts file extensions, while the codecs inside them are the
-------------------------------------^^^^^^^^^^^^^^^^^^^^^^^
reallyI wasn’t aware that ts could contain h264. But then again—I never
---------------------------------^^^^^^^
bothered with live TV recordings in recent years.
I think this is confusing CONTAINER and CODEC.
Where do we confuse those two? We specifically talked of codecs and “contain”.
.ts is a container format, h264 is a codec. I don't understand it myself, either but think of ts as your directory structure and h264 as your file structure.
Now you are confusing me. You say you don’t understand it, but then explain it. TS is like AVI and MKV: a file structure for the payload data. And payload data can be all kinds of stuff, from ASS plaintext subtitles, over opus audio to mpeg2 or h264 video.
Incidentally, sticking this stuff in a .tar is probably okay - that'sjust
another container, but sticking it in a .tar.gz is not, the gz is yourcodec
and will make the file BIGGER in all probability.
Tar does not compress, it simply puts all inputs in a 1:1 stream. It does
add some metadata (filename and so on). Packers reduce data volume by increasing information-per-byte. So if the total information stays the same (for lossless coding), the number of bytes decreases. Encoded video data ideally has even entropy. It is indistinguishable from random noise. That’s why compressing it again does not yield anything, or even adds some volume again.
--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.
“Mankind must put an end to war or war will put an end to mankind.”
– John F. Kennedy
<SNIP>
I got a old rig I can use. I actually burned OpenNAS, TrueNAS orFreeNAS on a USB stick. I can't recall which one I put on it tho. I downloaded all three. lol If you know that one is better than the
others, feel free to share. Also, I'd like to keep using LVM if I
can. If nothing else, I already got the data on the drives and won't
have to reformat and copy again. It took almost 100 hours to copy to
the new 16TB drive. Using LVM would make that easier, and faster.
I'll have to work with what I got for now but I really like theRaspberry option for its size and good options to upgrade later. I'll
just make do with something else until that option is doable. Maybe
it won't be to long.
Dale
:-) :-)
TrueNAS Core. It's the free one. Works great. Very stable, but it is
BSD, not Linux so you'll be frustrated sometimes. None the less it
works very well.
Mark Knecht wrote:
<SNIP>
I got a old rig I can use. I actually burned OpenNAS, TrueNAS orFreeNAS on a USB stick. I can't recall which one I put on it tho. I downloaded all three. lol If you know that one is better than the
others, feel free to share. Also, I'd like to keep using LVM if I
can. If nothing else, I already got the data on the drives and won't
have to reformat and copy again. It took almost 100 hours to copy to
the new 16TB drive. Using LVM would make that easier, and faster.
I'll have to work with what I got for now but I really like theRaspberry option for its size and good options to upgrade later. I'll
just make do with something else until that option is doable. Maybe
it won't be to long.
Dale
:-) :-)
TrueNAS Core. It's the free one. Works great. Very stable, but it is
BSD, not Linux so you'll be frustrated sometimes. None the less it
works very well.
Well, I booted it and it is FreeNAS. I got it on a USB stick tho.
Well, I put the installer on one stick and then installed on a second
stick. Kinda odd but I get it. I also noticed it is BSD based. I
played with BSD once before. One thing I can say, it's secure. Big time.
I see it uses ZFS or something. No mention of LVM. I figured that. Oh well.
If I can't hammer FreeNAS into shape, I'll try TrueNAS next. If it
works, that's fine too. ;-)
<br>> I see it uses ZFS or something. No mention of LVM. I figured that. Oh<br>> well.</div><div><br></div><div>I see LVM as something that belongs on your machine, not your NAS</div><div>device. Your LVM volumes will just be directorieson the NAS. You will</div><div>make your pools as large as you can afford and the NAS will just store</div><div>your data. You don't really need to worry about that much. My NAS</div><div>stores backups from 3 different machine, but all the backup
I think I'm going to switch. I need to start over anyway. I set up auser account and a large pool but while I can mount it, I can't put
By the way, when I got it installed, it did update to a newer version. Ididn't look to see if it was dated in any way but updates seem to be
Thanks for the info.
Dale
:-) :-)
Also, I think there are ways for you to build complex pools like a RAID0
from your 6TB and 8TB drives, and then a RAID1 using the RAID0 and your
14TB drive but I've never tried it because mine don't have enough drive
slots for that.
Also, turn on compression. It saves me between 15-20% so 14TB becomes 16TB storage. YMMV. Video files don't compress, at least not much. Data files generally do.
Hope this helps. I think you'll find TrueNAS fun actually but there is a learning curve. I've used it for about a year and barely scratched the surface.
Dale wrote:
Interesting info. Since this is a duplicate copy already, I'm not to worried about RAID stuff. I'd rather have two separate backups myself. Store them in different places for even more safety. Still, one of
these days. ;-)
I'm still getting this error.
root@fireball ~ # mkdir /mnt/Backup/Videos
mkdir: cannot create directory ‘/mnt/Backup/Videos’: Permission denied root@fireball ~ #
I've tried every permission in every place I can find. I have a user
set up, set permissions under storage and pools, all the little
directory thingys under it too. It mounts so I'd think I got everything set up correctly in the Share section. I actually followed a guide and
I don't think I missed anything. Still, I can't write anything to the thing. It mounts fine, even shows it is mounted rw. I can't find
anything wrong on the puter or NAS ends. This reminds me of the last
time I played with BSD. It works so different, it just throws a person upside down and shakes them.
I did switch to TrueNAS tho. Thing is, I got the same result from both, exact same error. Can't be a bug. It has to be me. No idea what I
missed but I bet I did. lol Just gonna keep banging on it. :-D I do like the GUI part. That's pretty neat.
Dale
:-) :-)
Update. I thought about just using Dolphin to play with the directory
that's actually on the NAS. I could create folders and files just fine
with Dolphin. Well, isn't that interesting. Finally I found a comment
with google that gave me a clue. I needed to change the options I use
with rsync plus it appears on my computer, I need to su to dale, same
user as on NAS. Now it works. I can rsync my files over.
Then I noticed something else. The network card in the NAS box, it's a
old 100MB card. Has anyone ever poured cold molasses before? Does
waiting on leap year sound familiar? ROFL Just saying.
Hey, at least I got the silly thing to work, slowly, but works. :/
Dale
:-) :-)
<div>I could disconnect and leave it running in the background</div><div>without having to watch cold molasses flow. I'm looking</div><div>to upgrade everything to 2.5Gb/S or higher one of these </div><div>days but once the initial backup isdone regular backups</div><div>are reasonably fast.</div><div><br></div><div>I was disappointed in the info Frank found about not being</div><div>able to bond Pools, but it still makes me wonder if someone</div><div>could create a 14TB RAID0 outside of
Howdy,
<<< SNIP >>>
Thoughts? Info to share? Ideas on a best path forward? Buy already built or build?
Thanks.
Dale
:-) :-)
I have a couple questions. I currently have the NAS thingy on a older
Dell machine. It has a 4 core CPU and 8GBs of ram so it is acceptable,
for the time being at least. Bad thing is, only two drive bays. :/ I have a few questions that I can't quite find answers to with google.
1: I have the OS on a USB stick. From what I've read, they do fail due
to wear at some point.
If I reinstall TrueNAS on a new USB stick, will it automatically see the previous pools and such or do I have to set everything up again fresh?
In other words, will I lose data?
This also includes if it is encrypted.
2: Hardware change. The Dell comes with a 100MB network card. I
ordered a 1GB card. I plan to put it in when it gets here. Will it see the new card and work automatically or will it take some work to get the network going?
and recompile. I'm not sure about BSD tho. Since it is sort of a
binary thing, does TrueNAS handle hardware changes such as a network
card well?
I also found out something power wise. The Dell when booted and sitting idle consumes about 120 watts monitor and all.
My main rig consumes just under 200 watts. Not to bad
but a Raspberry Pi would likely consume 15, 20 watts max according to what I've read.
Given the number of hard drives, it could pull 25 or 30 watts max but doubtful it would get that high. I'm looking at 4 bays but also found a 6 bay. I think 6 is overkill tho.
2: Hardware change. The Dell comes with a 100MB network card. I<SNIP>
ordered a 1GB card. I plan to put it in when it gets here. Will it see
the new card and work automatically or will it take some work to get the network going? On my Gentoo rig, I have to enable drivers in the kernel
and recompile. I'm not sure about BSD tho. Since it is sort of a
binary thing, does TrueNAS handle hardware changes such as a network
card well?
<div>interface which in my case is called 're0'. On the right you might, hopefully</div><div>possibly see a big blue button called "ADD". Consider giving that button </div><div>a push after you've installed your new card.</div><
<br></div><div> I believe you will get this figured out very soon. Continue exploring</div><div>the NGL world. </div><div><br></div><div>Mark</div></div>
Am Thu, Dec 15, 2022 at 10:08:02PM -0600 schrieb Dale:
I have a couple questions. I currently have the NAS thingy on a olderOTOH, TrueNAS is designed to run from it, so I would assume it handles its root drive with care. Perhaps you can disable verbose logging and such.
Dell machine. It has a 4 core CPU and 8GBs of ram so it is acceptable,
for the time being at least. Bad thing is, only two drive bays. :/ I >> have a few questions that I can't quite find answers to with google.
1: I have the OS on a USB stick. From what I've read, they do fail due >> to wear at some point.
If I reinstall TrueNAS on a new USB stick, will it automatically see thePools and their metadata are stored inside the pools. In Linux, you don’t even need to set up fstab. The pool stores its mount point internally. So
previous pools and such or do I have to set everything up again fresh?
you just start the zfs daemon and it does everything magically.
In other words, will I lose data?You won’t lose data, of course. But I think you meant settings(?). Probably about users, shares and such. Perhaps it has an export feature which can be run periodically.
This also includes if it is encrypted.Encryption is a built-in ZFS feature. So yes, it will remember that. Not
sure about the decryption process (keyfile).
2: Hardware change. The Dell comes with a 100MB network card. II assume the kernel is built like many general-purpose-distros: with everything in it you may need for the purpose. But since it is BSD, it may have driver issues (availability and stability for certain cards).
ordered a 1GB card. I plan to put it in when it gets here. Will it see >> the new card and work automatically or will it take some work to get the
network going?
Sometimes, when I read news about a new product, people complain that the
NIC is not Intel and will thus cause problems with BSD, especially with
niche stuff like the Killer-brand ethernet cards.
and recompile. I'm not sure about BSD tho. Since it is sort of aI don’t see a connection between being a “binary thing” and hardware change.
binary thing, does TrueNAS handle hardware changes such as a network
card well?
Your gentoo is also a binary thing once it is compiled. ;-)
I also found out something power wise. The Dell when booted and sittingI figured as much when you mentioned its 100 Mbps card. It must be old then, and back then, idle power was a non-issue.
idle consumes about 120 watts monitor and all.
My main rig consumes just under 200 watts. Not to badThat’s a very lot for my taste. With a lower mid-range GPU (110 W Radeon R7 370) and one spinning rust, my 8-year-old PC used to idle at 50 W. Without the HDD and with Intel graphics it is now at 27 W. Still not a good number when compared with today’s hardware.
but a Raspberry Pi would likely consume 15, 20 watts max according to what >> I've read.My 3B idles at 5 W tops, I think. It cannot be much more under load since it comes without a built-in heat spreader.
Given the number of hard drives, it could pull 25 or 30 watts max butMy four-bay NAS has four 6 TB drives and it draws around 50 W at idle. But that’s because it is a server board, incuding IPMI chip (and—interestingly—
doubtful it would get that high. I'm looking at 4 bays but also found a 6 >> bay. I think 6 is overkill tho.
an internal USB-A for an OS stick). And it’s Haswell generation, so almost a
decade old design. For this reason I switch it on only every few weeks or even months and only keep it running for a short time.
On Thu, Dec 15, 2022 at 9:08 PM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
<SNIP>
2: Hardware change. The Dell comes with a 100MB network card. I ordered a 1GB card. I plan to put it in when it gets here. Will it see the new card and work automatically or will it take some work to get the network going? On my Gentoo rig, I have to enable drivers in the kernel and recompile. I'm not sure about BSD tho. Since it is sort of a<SNIP>
binary thing, does TrueNAS handle hardware changes such as a network
card well?
At the risk of repeating a bit of what Frank said I'll put in my 2
cents as a
TrueNAS user. No intention to be snarky on my side, just pointing
a few things out.
1) Welcome to the world of "not Gentoo". What I think you are describing
as a 'binary thing' is, for the most part, the way we work out here.
2) I disagree with your description of how "it's done" in Gentoo Land. You made a choice to put your network driver in the kernel. You could have
built it as a module and loaded that module. Both would have worked.
3) Out here in Not Gentoo Land (NGL) they supply you with 100's of
modules and generally installers figure out which ones to load. My main
NGL machine has literally over 100 modules loaded. I don't know what
they do (for the most part) and mostly I don't care.
4) While I cannot tell you if the card you ordered has a Linux or FreeBSD driver, I can point out that on the left hand side of your TrueNAS dashboard, accessible in your browser hopefully, there is a pulldown
called 'Network'. It should hopefully show you the current network
interface which in my case is called 're0'. On the right you might,
hopefully
possibly see a big blue button called "ADD". Consider giving that button
a push after you've installed your new card.
I believe you will get this figured out very soon. Continue exploring the NGL world.
Mark
4) While I cannot tell you if the card you ordered has a Linux or FreeBSD driver, I can point out that on the left hand side of your TrueNAS dashboard, accessible in your browser hopefully, there is a pulldown
called 'Network'. It should hopefully show you the current network interface which in my case is called 're0'. On the right you might, hopefully
possibly see a big blue button called "ADD". Consider giving that button a push after you've installed your new card.
4: I have a monitor hooked up still so I can do it the text way if needed. It mentions about setting up the network as one of the
options.
Since you mentioned it has a GUI option, I may just do that.
So long as it works.
Am Fri, Dec 16, 2022 at 04:43:25PM -0600 schrieb Dale:
That’s what I referred to earlier: you get a web interface built-in that takes care of all the chores. I always wanted to set up shares on my NAS so that guests could easily access it anonymously, but safely (meaning: no4) While I cannot tell you if the card you ordered has a Linux or FreeBSD >>> driver, I can point out that on the left hand side of your TrueNAS4: I have a monitor hooked up still so I can do it the text way if
dashboard, accessible in your browser hopefully, there is a pulldown
called 'Network'. It should hopefully show you the current network
interface which in my case is called 're0'. On the right you might,
hopefully
possibly see a big blue button called "ADD". Consider giving that button >>> a push after you've installed your new card.
needed. It mentions about setting up the network as one of the
options.
Since you mentioned it has a GUI option, I may just do that.
So long as it works.
write access). I never had the patience to go through the whole setup of ftp and/or samba with the proper users and directories. But a web UI could take care of all of that. Plus it’s shiny. :D
My point is: there is no need for a monitor. ;-)
My point is: there is no need for a monitor. ;-)
I noticing that now. Once the install is done and you have the IP
address, heck, you don't need a monitor for much of anything it
appears. I even found a console so one can type in things to do. I wonder, could one install the LVM stuff and use that? That would be interesting.
I wonder if there is a NAS software that uses LVM instead. Interesting thought. I just may go bug google on that one. o_O
Am Fri, Dec 16, 2022 at 09:50:01PM -0600 schrieb Dale:
Interestingly, after feeding LVM NAS to my search engine, one of the first results was a Qnap user forum thread which reads that Qnaps use LVM. ItMy point is: there is no need for a monitor. ;-)I noticing that now. Once the install is done and you have the IP
address, heck, you don't need a monitor for much of anything it
appears. I even found a console so one can type in things to do. I
wonder, could one install the LVM stuff and use that? That would be
interesting.
I wonder if there is a NAS software that uses LVM instead. Interesting
thought. I just may go bug google on that one. o_O
makes sense, since they most likely use a “normal” linux software RAID underneath, and LVM is then the best way to dynamically manage the space on top of that.
While ZFS has the same pooling feature as LVM, meaning you can bunch several disks together to create a JBOD, it has one big disadvantage over LVM: you can grow a pool, but not shrink it. Actually, while reading up on stuff for this thread, I learned that these days it is actually possible to remove a mirror vdev from a mirror-only pool (a mirror can technically also be a single device). But according to https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWhyNoVdevRemoval
it’s not perfect either.
While ZFS has the same pooling feature as LVM, meaning you can bunch several
disks together to create a JBOD, it has one big disadvantage over LVM: you can grow a pool, but not shrink it. Actually, while reading up on stuff for this thread, I learned that these days it is actually possible to remove a mirror vdev from a mirror-only pool (a mirror can technically also be a single device). But according to https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWhyNoVdevRemoval
it’s not perfect either.
One thing about LVM, I already know how to use it. I've got notes on
how to do things and it has worked. With ZFS I'd have to learn it all
over again plus it could confuse me with my using LVM on my main rig.
In a way, if I build a Raspberry NAS, I'd like to have Linux and LVM on
the thing. At least then I have experience moving data and such. Right now, I don't even have the basics of ZFS or BSD. I'm not saying TrueNAS
is bad or anything, just that I don't really care for the learning
curve. Then the confusion part on top of that.
I've read there are distros built for Raspberry thingys. I'd be shocked
if it isn't doable. I'd be shocked if someone hasn't already done it even.
On Fri, Dec 16, 2022 at 8:50 PM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
Ione. o_O
wonder, could one install the LVM stuff and use that? That would be interesting. I wonder if there is a NAS software that uses LVM
instead. Interesting thought. I just may go bug google on that
Maybe I'm missing the point but why would you want LVM on a
storage pool? If I'm doing backups I just want space. I let TrueNas
put it on disk and give it back if asked. Why put another layer
of indirection?
If you're intending to use it as simple NAS - i.e. - the only copy
of some data on your network - then possibly LVM might
be interesting, but then you need a second TrueNAS box
to back that up. NAS as a mountable data location is
different than NAS doing backups which is what I thought
this thread was about. What am I not remembering?
- Mark
I noticing that now. Once the install is done and you have the IP
address, heck, you don't need a monitor for much of anything it
appears. I even found a console so one can type in things to do.
I
wonder, could one install the LVM stuff and use that? That would be interesting. I wonder if there is a NAS software that uses LVM
instead. Interesting thought. I just may go bug google on that one. o_O
Mark Knecht wrote:
On Fri, Dec 16, 2022 at 8:50 PM Dale <rdalek1967@gmail.com> wrote:o_O
I
wonder, could one install the LVM stuff and use that? That would be interesting. I wonder if there is a NAS software that uses LVM
instead. Interesting thought. I just may go bug google on that one.
Maybe I'm missing the point but why would you want LVM on a
storage pool? If I'm doing backups I just want space. I let TrueNas
put it on disk and give it back if asked. Why put another layer
of indirection?
If you're intending to use it as simple NAS - i.e. - the only copy
of some data on your network - then possibly LVM might
be interesting, but then you need a second TrueNAS box
to back that up. NAS as a mountable data location is
different than NAS doing backups which is what I thought
this thread was about. What am I not remembering?
- Mark
It's more about me being more used to using LVM. Also, more used toLinux as well. BSD is not something I have much experience with and until recently, none with ZFS. Even the little experience I have with BSD was
By replacing ZFS with LVM, I'm working with something I'm familiar withand less likely to mess up things. Things get messed up enough without
Dale
:-) :-)
That is true. Thing is, I've learned how to manage LVM even withencrypted data. I've also learned how to expand storage without losing
It's not that I want to compile things, it's that I want to use thingsI'm already really familiar with. If I bought a Raspberry Pi and built a
My network card is out for delivery. A few days late but better thannever I guess. I'll see if the drivers needed for it are available as
><div><br></div><div>Maybe just live with TrueNAS for a month or two and get to know it on a friendly level first?<br><div><br></div><div>I think I'm still not understanding the need for LVM. Help me understand. I see ZFS on TrueNAS as acombination of mdadm and a file system. I'm not sure I even have partitions. My ZFS pool is just using the whole drive, but that was all I wanted. Are you planning on using your drive space for other things? (Plex server in a jail, or remote real-
Do you want the system layered, with each layer doing one job? Use dm-integrity to protect against corruption, raid to join the disks, lvm
to partition them, and ext to manage the directories and files.
I do the latter ...
</div><div>want to have flexibility in the future. Desktop computers or</div><div>system drives certainly. You didn't tell me what replaces</div><div>the compression aspect of the problem but I'm sure there's</div><div>something. It'sa great strategy if you have the expertise and</div><div>time to set it up and then manage it when a problem arises,</div><div>if it ever arises. </div><div><br></div><div>I'm just asking what's the purpose of doing LVM, or your</div><div>
<br></div></div>
Hummm...I don't know Dale, I don't know... ZFS is a file system.
LVM is an abstraction on top (or underneath?) of a file system.
My understanding of LVM is that it frees you from hard decisions
on partition sizes, not that it replaces ZFS or ext3/4/5.
On Sat, Dec 17, 2022 at 8:52 AM Dale <rdalek1967@gmail.com> wrote:
I
wonder, could one install the LVM stuff and use that? That would be interesting. I wonder if there is a NAS software that uses LVM
instead. Interesting thought. I just may go bug google on that one.
Maybe I'm missing the point but why would you want LVM on aIt's more about me being more used to using LVM. Also, more used to
storage pool? If I'm doing backups I just want space. I let TrueNas
put it on disk and give it back if asked. Why put another layer
of indirection?
[…]
Linux as well. BSD is not something I have much experience with and until recently, none with ZFS. Even the little experience I have with BSD was
well over a decade ago, maybe two decades ago. I barely remember it really.
Hummm...I don't know Dale, I don't know... ZFS is a file system.
LVM is an abstraction on top (or underneath?) of a file system.
My understanding of LVM is that it frees you from hard decisions
on partition sizes, not that it replaces ZFS or ext3/4/5.
WRT you I recommend that you try living in NGL for a while. Possibly
you are just a bit too indoctrinated in the religion of building packages 30-50 times a year believing (without hard data) that it provides value.
I'm just asking what's the purpose of doing LVM, or your
suggested layering, specifically on a storage pool for a
home user like Dale? That's the part I don't understand,
especially for a new NAS user like Dale?
Mark Knecht wrote:
On Sat, Dec 17, 2022 at 8:52 AM Dale <rdalek1967@gmail.com> wrote:
Mark Knecht wrote:
On Fri, Dec 16, 2022 at 8:50 PM Dale <rdalek1967@gmail.com> wrote:
o_OI
wonder, could one install the LVM stuff and use that? That would be interesting. I wonder if there is a NAS software that uses LVM
instead. Interesting thought. I just may go bug google on that one.
Maybe I'm missing the point but why would you want LVM on a
storage pool? If I'm doing backups I just want space. I let TrueNas
put it on disk and give it back if asked. Why put another layer
of indirection?
If you're intending to use it as simple NAS - i.e. - the only copy
of some data on your network - then possibly LVM might
be interesting, but then you need a second TrueNAS box
to back that up. NAS as a mountable data location is
different than NAS doing backups which is what I thought
this thread was about. What am I not remembering?
- Mark
It's more about me being more used to using LVM. Also, more used toLinux as well. BSD is not something I have much experience with and until recently, none with ZFS. Even the little experience I have with BSD was
well over a decade ago, maybe two decades ago. I barely remember it really.
By replacing ZFS with LVM, I'm working with something I'm familiar withand less likely to mess up things. Things get messed up enough without adding more confusion. ;-)
Dale
:-) :-)
Hummm...I don't know Dale, I don't know... ZFS is a file system.
LVM is an abstraction on top (or underneath?) of a file system.
My understanding of LVM is that it frees you from hard decisions
on partition sizes, not that it replaces ZFS or ext3/4/5.
That is true. Thing is, I've learned how to manage LVM even with
encrypted data. I've also learned how to expand storage without losing
data or getting confused about what I'm doing. To me, using LVM is pretty easy given the notes I have for the tasks I do most often.
You may or may not know this but TrueNAS is available as a
Linux version:
https://www.truenas.com/blog/first-release-of-truenas-on-linux/
I don't recommend it. It's new. Let someone else figure it out. However
it might be more to your liking, and because it's Linux you'd be more comfortable messing it up. ;0-
WRT you I recommend that you try living in NGL for a while. Possibly
you are just a bit too indoctrinated in the religion of building packages 30-50 times a year believing (without hard data) that it provides value. Instead you might just consider relaxing and letting the system
take care of itself. In the last year I've only updated my TrueNAS box
twice that I can remember.
On the other hand if system tweaking is what brings you joy then
Que Sera Sera .....
Good luck,
Mark
It's not that I want to compile things, it's that I want to use things I'm already really familiar with. If I bought a Raspberry Pi and built a NAS with it, I don't care if I compile the software on it or not as long as it has the software I need or I can install what I need. From what I've read, compiling on a Raspberry isn't much fun. It's very time consuming. Having
a OS, binary one at that, that is Linux based is a big plus. I can run
LVM, cryptsetup and such in likely every Linux distro out there and get the same result as on my Gentoo box. Switching to BSD, using ZFS, means I have to learn a whole new set of tools and methods. I had enough fun learning
LVM and I don't think LVM is going to die anytime soon. It should be
around for the foreseeable future. As it is, even now, I still don't get
how ZFS works. I just followed a guide to get it working, sort of. It's still not encrypted. Figuring that out is next. That should be fun.
My network card is out for delivery. A few days late but better than
never I guess. I'll see if the drivers needed for it are available as modules or not. I suspect they are tho based on info in this thread.
Dale
:-) :-)
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
My reasoning is simple, I'm already familiar with LVM and how to manageit.
<SNIP></div><div><br></div><div>Take the machine, wipe it and build a NAS from scratch with Gentoo. If all you want is an NFS mount that won't be difficult. Add an NFS server, export your mount and you're done, right? Managing it overthe long term will be far more work than TrueNAS but you will be comfortable with changing disks and adding network cards which is important to you. Life is too short to deal with things you really don't enjoy.</div><div><br></div><div>I would not
On Sun, Dec 18, 2022 at 8:13 AM Dale <rdalek1967@gmail.com> wrote:but the thought crossed my mind. lol Even tho I now have a 1GB network
Well, I finally got it so I could do a backup. I didn't need a hammer
277,193,507 100% 16.18MB/s 0:00:16
519,216,571 100% 18.86MB/s 0:00:26
738,078,565 100% 23.54MB/s 0:00:29
When I use iftop, it shows it isn't doing anywhere near the speed itAs you can see, the files sizes are large enough it should do better.
and 8GBs of memory. It should have enough horsepower under the hood.I can't figure out why it is so slow tho. The NAS rig is a 4 core CPU
done in about a week, maybe. o_OAnyway, it's progress for now at least. ;-) At this rate, it'll be
Dale
To what end Dale? Aren't you painting yourself into a corner with asystem you don't really want to run? Wipe the machine and start over from scratch with Gentoo.
From my vantage point you don't provide enough information for me to makean educated guess.
1) Is your data coming off the host machine able to transfer to othermachines at 1Gb/S type speeds?
2) Can data coming off of your NAS transfer to other machines at 1G/Stype speeds?
3) How are the two machines connected? If they are going through a routeror hub, do you know that hub doesn't limit throughput?
4) Is anything else happening on the network? Video flowing around whilepeople are watching TV or something?
5) CPU horsepower isn't the only potential bottleneck. Are your disks inthe NAS operating slowly? Are you running out of memory?
Have you considered running something like iperf?
Mark
rdalek1967@gmail.com</a>> wrote:<br>> ><br>> > Well, I finally got it so I could do a backup. I didn't need a hammer but the thought crossed my mind. lol Even tho I now have a 1GB network card, it's still really slow. Itshows up as a 1GB connection on both my Gentoo machine and the NAS machine. This is a example of the speeds I'm seeing. Just snippets.<br>> ><br>> ><br>> > 277,193,507 100% 16.18MB/s 0:00:16<br>> > 519,216,571 100%
<div>I would have used the Linux version if it had existed when I built the machine. I don't love BSD, but not because it doesn't work but because certain CLI tools have slightly different options.</div><div><br></div><div>Wipe the machineand start over with Gentoo from scratch...</div><div>Wipe the machine and start over with Gentoo from scratch...</div><div>Humm, I think you should...</div><div>Wipe the machine and start over with Gentoo from scratch...</div></div>
Mark Knecht wrote:
On Sat, Dec 17, 2022 at 4:42 PM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
<SNIP>
My reasoning is simple, I'm already familiar with LVM and how to
manage it.
<SNIP>
Take the machine, wipe it and build a NAS from scratch with Gentoo. If
all you want is an NFS mount that won't be difficult. Add an NFS
server, export your mount and you're done, right? Managing it over the
long term will be far more work than TrueNAS but you will be
comfortable with changing disks and adding network cards which
is important to you. Life is too short to deal with things you really
don't enjoy.
I would not suggest you look at Ubuntu Server because it's NGL. 10
minutes to install, 3 minutes to figure out how to add the NFS server. However it's a different package manager and truly not as nice as emerge/portage so you probably won't like that part of NGL either. I
truly don't like apt, but it works if I stay in my lane so I've
learned to do that, the advantage being I've never had to build a
package from scratch and I've never in 5 or 6 years had an update fail.
Wipe the machine. You'll be happier.
Best wishes,
Mark
Well, I finally got it so I could do a backup. I didn't need a hammer
but the thought crossed my mind. lol Even tho I now have a 1GB network card, it's still really slow. It shows up as a 1GB connection on both
my Gentoo machine and the NAS machine. This is a example of the speeds
I'm seeing. Just snippets.
277,193,507 100% 16.18MB/s 0:00:16
519,216,571 100% 18.86MB/s 0:00:26
738,078,565 100% 23.54MB/s 0:00:29
As you can see, the files sizes are large enough it should do better.
When I use iftop, it shows it isn't doing anywhere near the speed it
should, maybe 1/4th or so. I'd expect at least double or triple that
speed. In all honesty, I'd think the hard drive would be the limiting factor. Even on my Gentoo rig I only get about 50 to 60MBs/sec for
encrypted drives. I think the encryption slows that down. When copying
from a plain drive to a plain drive, I get 100MBs/sec or so.
I can't figure out why it is so slow tho. The NAS rig is a 4 core CPU
and 8GBs of memory. It should have enough horsepower under the hood.
Maybe it is something I'm not aware of. It is a older rig so maybe it
isn't SATA's fastest version, maybe even the original or something. I
can't find anything in lspci or dmesg so not real sure where to look on
BSD.
Anyway, it's progress for now at least. ;-) At this rate, it'll be
done in about a week, maybe. o_O
Dale
:-) :-)
On Sat, Dec 17, 2022 at 4:42 PM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
<SNIP>
My reasoning is simple, I'm already familiar with LVM and how tomanage it.
<SNIP>
[…]
Wipe the machine. You'll be happier.
Best wishes,
Mark
Well, I finally got it so I could do a backup. I didn't need a hammer
but the thought crossed my mind. lol Even tho I now have a 1GB network card, it's still really slow. It shows up as a 1GB connection on both
my Gentoo machine and the NAS machine. This is a example of the speeds
I'm seeing. Just snippets.
277,193,507 100% 16.18MB/s 0:00:16
519,216,571 100% 18.86MB/s 0:00:26
738,078,565 100% 23.54MB/s 0:00:29
As you can see, the files sizes are large enough it should do better.
When I use iftop, it shows it isn't doing anywhere near the speed it
should, maybe 1/4th or so. I'd expect at least double or triple that speed. In all honesty, I'd think the hard drive would be the limiting factor. Even on my Gentoo rig I only get about 50 to 60MBs/sec for encrypted drives. I think the encryption slows that down. When copying from a plain drive to a plain drive, I get 100MBs/sec or so.
I can't figure out why it is so slow tho. The NAS rig is a 4 core CPU
and 8GBs of memory.
It should have enough horsepower under the hood.
Maybe it is something I'm not aware of. It is a older rig so maybe it
isn't SATA's fastest version, maybe even the original or something. I
Well, I finally got it so I could do a backup. I didn't need a hammerbut the thought crossed my mind. lol Even tho I now have a 1GB network
277,193,507 100% 16.18MB/s 0:00:16
519,216,571 100% 18.86MB/s 0:00:26
738,078,565 100% 23.54MB/s 0:00:29
As you can see, the files sizes are large enough it should do better.When I use iftop, it shows it isn't doing anywhere near the speed it
I can't figure out why it is so slow tho. The NAS rig is a 4 core CPUand 8GBs of memory. It should have enough horsepower under the hood.
Anyway, it's progress for now at least. ;-) At this rate, it'll be donein about a week, maybe. o_O
Dale
I can't figure out why it is so slow tho. The NAS rig is a 4 core CPU
and 8GBs of memory. It should have enough horsepower under the hood.
Maybe it is something I'm not aware of. It is a older rig so maybe it
isn't SATA's fastest version, maybe even the original or something. I
can't find anything in lspci or dmesg so not real sure where to look on
BSD.
I can't figure out why it is so slow tho. The NAS rig is a 4 core CPUand 8GBs of memory. It should have enough horsepower under the hood.
I suspect it has something to do with this being a older system.
I wouldn't be surprised if the SATA was a older and slower version.
I guess I could google it.
I suspect it has something to do with this being a older system. I
wouldn't be surprised if the SATA was a older and slower version. I
guess I could google it.
On Sunday, 18 December 2022 15:12:37 GMT Dale wrote:
Mark Knecht wrote:Has it auto-negotiated a full-duplex connection at 1Gbps? Run ifconfig and check the output, it should say something like:
On Sat, Dec 17, 2022 at 4:42 PM Dale <rdalek1967@gmail.comWell, I finally got it so I could do a backup. I didn't need a hammer
<mailto:rdalek1967@gmail.com>> wrote:
<SNIP>
My reasoning is simple, I'm already familiar with LVM and how tomanage it.
<SNIP>
Take the machine, wipe it and build a NAS from scratch with Gentoo. If
all you want is an NFS mount that won't be difficult. Add an NFS
server, export your mount and you're done, right? Managing it over the
long term will be far more work than TrueNAS but you will be
comfortable with changing disks and adding network cards which
is important to you. Life is too short to deal with things you really
don't enjoy.
I would not suggest you look at Ubuntu Server because it's NGL. 10
minutes to install, 3 minutes to figure out how to add the NFS server.
However it's a different package manager and truly not as nice as
emerge/portage so you probably won't like that part of NGL either. I
truly don't like apt, but it works if I stay in my lane so I've
learned to do that, the advantage being I've never had to build a
package from scratch and I've never in 5 or 6 years had an update fail.
Wipe the machine. You'll be happier.
Best wishes,
Mark
but the thought crossed my mind. lol Even tho I now have a 1GB network
card, it's still really slow. It shows up as a 1GB connection on both
my Gentoo machine and the NAS machine. This is a example of the speeds
I'm seeing. Just snippets.
277,193,507 100% 16.18MB/s 0:00:16
519,216,571 100% 18.86MB/s 0:00:26
738,078,565 100% 23.54MB/s 0:00:29
As you can see, the files sizes are large enough it should do better.
When I use iftop, it shows it isn't doing anywhere near the speed it
should, maybe 1/4th or so. I'd expect at least double or triple that
speed. In all honesty, I'd think the hard drive would be the limiting
factor. Even on my Gentoo rig I only get about 50 to 60MBs/sec for
encrypted drives. I think the encryption slows that down. When copying
from a plain drive to a plain drive, I get 100MBs/sec or so.
I can't figure out why it is so slow tho. The NAS rig is a 4 core CPU
and 8GBs of memory. It should have enough horsepower under the hood.
Maybe it is something I'm not aware of. It is a older rig so maybe it
isn't SATA's fastest version, maybe even the original or something. I
can't find anything in lspci or dmesg so not real sure where to look on
BSD.
Anyway, it's progress for now at least. ;-) At this rate, it'll be
done in about a week, maybe. o_O
Dale
:-) :-)
media: Ethernet autoselect (1000baseT <full-duplex>)
If not, then you may need to set this up manually.
root@truenas[~]# iperf -s ------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default) ------------------------------------------------------------
And nothing. Several minutes later, still nothing. And it continues tosit there. I don't think it is working. :/
Still, odds are, whatever it is, I'm not likely going to be able tochange it. That poor old CPU just may not have the needed instruction set
And still nothing. It's been sitting there since I read the lastmessage. Still, nothing.
On Sun, Dec 18, 2022 at 11:39 AM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
<SNIP>
I suspect it has something to do with this being a older system. I wouldn't be surprised if the SATA was a older and slower version. I
guess I could google it.
You need to study your specs. Even the first version of SATA, SATA 1,
was capable of 150MB/S. SATA2 does 300MB/S. This is unlikely IMO
to be due to SATA specs.
Have you run iperf yet as I suggested? It will easily tell you what the network performance is and takes 5 seconds in NGL.
Since this is local, I just use rsync to do my backups. I did have to change the options a bit. It seems TrueNAS doesn't like some of the permissions or something.
Wol wrote:
On 18/12/2022 18:59, Dale wrote:
Since this is local, I just use rsync to do my backups. I did have to
change the options a bit. It seems TrueNAS doesn't like some of the
permissions or something.
Are you running the rsync daemon on the NAS? I'm probably teaching
grandma to suck eggs, but that massively reduces the need for network
traffic.
Cheers,
Wol
I mount the NAS on my Gentoo rig. I mount it under /mnt. Then I run
rsync and copy from the source to the mount point for the NAS. I may
could go the other way but never thought about doing it that way. Kinda sounds backwards to me but I dunno. ;-)
Mark Knecht wrote:
On Sun, Dec 18, 2022 at 12:20 PM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
root@truenas[~]# iperf -scontinues to sit there. I don't think it is working. :/
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
And nothing. Several minutes later, still nothing. And it
change it. That poor old CPU just may not have the needed
Still, odds are, whatever it is, I'm not likely going to be able to
instruction set to be really fast for this. Maybe a different
encryption would be better. I dunno. It's temporary anyway.
message. Still, nothing.
And still nothing. It's been sitting there since I read the last
Sadly, you didn't read all of my instructions or apparently read the
man page or help file
All you've done is start the server which listens for a connection.
Now go to your Gentoo Land machine that you want to backup and execute
iperf -c IP.ADDR.OF.SERVER
Wait 10 seconds and hit ctrl-C
Oh, I read that but didn't get that one worked with the other. Ooops.$ eix iperf
Thing is, I don't have the second command on my Gentoo install. :/
It sort of grumbles about that. May look into that later. Got other things in the air right now.
Dale
Eventually, I plan to build a Raspberry Pi NAS. When I do, I'll post everything major I needed, boards, case etc for everyone to look at.
I'll even try to upload some pics, or share as attachments if there is interest. Unless I find one heck of a deal on a used NAS that is.
Still may build one even then. ;-)
Am Thu, Dec 15, 2022 at 10:08:02PM -0600 schrieb Dale:
I just read a news story about a new NAS model from Terramaster. (Interestingly, they have their OS on an internal USB stick, so it’s easy to
swap it out for a standard Linux. And it uses a nice Celeron N5100 x86 processor.)
Eventually, I plan to build a Raspberry Pi NAS. When I do, I'll postIn the comments section of the article, there was a link to this: https://www.youtube.com/watch?v=R2S2RMNv7OU
everything major I needed, boards, case etc for everyone to look at.
I'll even try to upload some pics, or share as attachments if there is
interest. Unless I find one heck of a deal on a used NAS that is.
Still may build one even then. ;-)
A Raspberry Pi NAS with 1 Petabyte of storage. Enjoy. :)
I think if I can hold out a little while, something really nice is going
to come along. It seems there is a good bit of interest in having a Raspberry Pi NAS that gives really good performance. I'm talking a NAS
that is about the same speed as a internal drive. Plus the ability to
use RAID and such. I'd like to have a 6 bay with 6 drives setup in
pairs for redundancy. I can't recall what number RAID that is.
Basically, if one drive fails, another copy still exists. Of course,
two independent NASs would be better in my opinion. Still, any of this
is progress.
On 21/12/2022 02:47, Dale wrote:
I think if I can hold out a little while, something really nice is going
to come along. It seems there is a good bit of interest in having a
Raspberry Pi NAS that gives really good performance. I'm talking a NAS
that is about the same speed as a internal drive. Plus the ability to
use RAID and such. I'd like to have a 6 bay with 6 drives setup in
pairs for redundancy. I can't recall what number RAID that is.
Basically, if one drive fails, another copy still exists. Of course,
two independent NASs would be better in my opinion. Still, any of this
is progress.
That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note
that 1+0 is often called 10, but linux-10 is slightly different.
I'd personally be inclined to go for raid-6. That's 4 data drives, 2
parity (so you could have an "any two" drive failure and still recover).
A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A
three-copy is vulnerable to a three-drive failure.
In other words, a two-copy raid-10 might be taken out by a failure
that a raid-6 will survive. A three-copy raid-10 might be taken out by
a failure that will take out a raid-6. Choose your poison :-)
Cheers,
Wol
.
On 21/12/2022 02:47, Dale wrote:
I think if I can hold out a little while, something really nice is going
to come along. It seems there is a good bit of interest in having a Raspberry Pi NAS that gives really good performance. I'm talking a NAS that is about the same speed as a internal drive. Plus the ability to
use RAID and such. I'd like to have a 6 bay with 6 drives setup in
pairs for redundancy. I can't recall what number RAID that is.
Basically, if one drive fails, another copy still exists. Of course,
two independent NASs would be better in my opinion. Still, any of this
is progress.
That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0 is often called 10, but linux-10 is slightly different.
I'd personally be inclined to go for raid-6. That's 4 data drives, 2 parity (so you could have an "any two" drive failure and still recover).
A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A three-copy is vulnerable to a three-drive failure.
Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:
On 21/12/2022 02:47, Dale wrote:In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
...
pool. So mirror + pool = mirrorpool, hence the 1+0 → 10.
...
Fun, innit?
This is why at some point, I'd like to have two sets of backups. RAID
or not.
Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:
On 21/12/2022 02:47, Dale wrote:
I think if I can hold out a little while, something really nice is going >>> to come along. It seems there is a good bit of interest in having a
Raspberry Pi NAS that gives really good performance. I'm talking a NAS >>> that is about the same speed as a internal drive. Plus the ability to
use RAID and such. I'd like to have a 6 bay with 6 drives setup in
pairs for redundancy. I can't recall what number RAID that is.
Basically, if one drive fails, another copy still exists. Of course,
two independent NASs would be better in my opinion. Still, any of this >>> is progress.
That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0 >> is often called 10, but linux-10 is slightly different.
In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
pool. So mirror + pool = mirrorpool, hence the 1+0 → 10.
I'd personally be inclined to go for raid-6. That's 4 data drives, 2 parity >> (so you could have an "any two" drive failure and still recover).
A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A three-copy is >> vulnerable to a three-drive failure.
At first, I had only two drives in my 4-bay NAS, which were of course set up as a mirror. After a year, when it became full, I bought the second pair of drives and had long deliberations by then, what to choose. I went for raid-6 (or RaidZ2 in ZFS parlance). With only four disks, it has the same net capacity as a pair of mirrors, but at the advantage that *any* two drives
may fail, not just two particular ones. A raid of mirrors has performance benefits over a parity raid, but who cares for a simple Gbit storage device.
With increasing number of disks, a mirror setup is at a disadvantage with storage efficiency – it’s always 50 % or less, if you mirror over more than
two disks. But with only four disks, that was irrelevant in my case. On the plus-side, each mirror can have a different physical disk size, so you can more easily mix’n’match what you got lying around, or do upgrades in smaller
increments.
If I wanted to increase my capacity, I’d have to replace *all* drives with bigger ones. With a mirror, only the drives in one of the mirrors need replacing. And the rebuild process would be quicker and less painful, as
each drive will only be read once to rebuild its partner, and there is no parity calculation involved. In a RAID, each drive is replaced one by one, and each replacement requires a full read of all drives’ payload.
With older
drives, this is cause for some concern whether the disks may survive that. That’s why, with increasing disk capacities, raid-5 is said to be obsolete. Because if another drive fails during rebuild, you are officially screwed.
Fun, innit?
On 21/12/2022 06:19, Frank Steinmetzger wrote:
Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:
On 21/12/2022 02:47, Dale wrote:
I think if I can hold out a little while, something really nice is going
to come along. It seems there is a good bit of interest in having a Raspberry Pi NAS that gives really good performance. I'm talking a NAS
that is about the same speed as a internal drive. Plus the ability to use RAID and such. I'd like to have a 6 bay with 6 drives setup in pairs for redundancy. I can't recall what number RAID that is. Basically, if one drive fails, another copy still exists. Of course, two independent NASs would be better in my opinion. Still, any of this
is progress.
That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0
is often called 10, but linux-10 is slightly different.
In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
pool. So mirror + pool = mirrorpool, hence the 1+0 → 10.
Except raid-10 is not a stripe of mirrors.
It's each block is saved to two different drives. (Or 3, or more, so long
as you have more drives than mirrors.)
Linux will happily give you a 2-copy mirror across 3 drives - 3x6TB drives will give you 9TB useful storage ...
If I wanted to increase my capacity, I’d have to replace *all* drives with
bigger ones. With a mirror, only the drives in one of the mirrors need replacing. And the rebuild process would be quicker and less painful, as each drive will only be read once to rebuild its partner, and there is no parity calculation involved. In a RAID, each drive is replaced one by one, and each replacement requires a full read of all drives’ payload.
If you've got a spare SATA connection or whatever, each replacement does not need a full read of all drives. "mdadm --add /dev/sdx --replace /dev/sdy". That'll stream sdy on to sdx, and only hammer the other drives if sdy complains ...
With older
drives, this is cause for some concern whether the disks may survive that. That’s why, with increasing disk capacities, raid-5 is said to be obsolete.
Because if another drive fails during rebuild, you are officially screwed.
Fun, innit?
They've always said that. Just make sure you don't have multiple drives from the same batch, then they're less likely statistically to fail at the same time. I'm running raid-5 over 3TB partitions ...
Yes? In a mirror setup, all member drives of a mirror have the same content (at least in ZFS).
Raid 10 distributes its content across several mirrors. This is the cause
for its increased performance. So when one of the mirrors (not single drive, but a whole set of mirrored drives) fails, the pool is gone.
Linux will happily give you a 2-copy mirror across 3 drives - 3x6TB drives >> will give you 9TB useful storage ...
I admit, I’ve never head of that. (Though it sounds like raid-5 to me.)
On 18/12/2022 18:59, Dale wrote:
Since this is local, I just use rsync to do my backups. I did have to change the options a bit. It seems TrueNAS doesn't like some of the permissions or something.
Are you running the rsync daemon on the NAS? I'm probably teaching
grandma to suck eggs, but that massively reduces the need for network traffic.
Cheers,
Wol
Wols Lists wrote:
On 18/12/2022 22:11, Dale wrote:
Wol wrote:Sounds to me like you're doing it all wrong either way ...
On 18/12/2022 18:59, Dale wrote:
Since this is local, I just use rsync to do my backups. I did have to >>>> change the options a bit. It seems TrueNAS doesn't like some of the >>>> permissions or something.
Are you running the rsync daemon on the NAS? I'm probably teaching
grandma to suck eggs, but that massively reduces the need for network
traffic.
Cheers,
Wol
I mount the NAS on my Gentoo rig. I mount it under /mnt. Then I run
rsync and copy from the source to the mount point for the NAS. I may
could go the other way but never thought about doing it that way. Kinda >> sounds backwards to me but I dunno. ;-)
What is *supposed* to happen is that you have the daemon running on
one machine and the client on the other - doesn't matter which.
Then the client tells the daemon what files are to be copied, THE TWO COMPARE CHECKSUMS, and only the stuff that fails the checksum is
copied. So if you're doing an incremental backup, network usage and
writes are kept to a minimum.
I tell people to an in-place backup if they're running on a snapshot
setup, because again it only writes stuff that has actually changed.
Cheers,
Wol
Do you have a link to the proper way to do it? I don't copy to a
different machine often so my current method may be the problem. Maybe
the way you mention will work much better, even a little better would be nice. ;-)
Dale
:-) :-)
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 302 |
| Nodes: | 16 (0 / 16) |
| Uptime: | 96:58:24 |
| Calls: | 6,764 |
| Calls today: | 2 |
| Files: | 12,295 |
| Messages: | 5,376,370 |
| Posted today: | 1 |
![https://en.wikipedia.org/wiki/File:Informatik-Containerformate-Beispiele.svg"](https://en.wikipedia.org/wiki/File:Informatik-Containerformate-Beispiele.svg"){kind=link}
