1. Forum
  2. tqwNet
  3. TQW DOCKER

  • Re: FInally hit a wall...

    From poindexter FORTRAN@1337:3/178 to paulie420 on Thu Dec 19 09:15:24 2024
    paulie420 wrote to poindexter FORTRAN <=-

    I use my NFS for file storage; like on all my laptops ~/Downloads, ~/Documents, etc etc - I syncthing to the NAS... then I backup the
    entire NAS to my PBS. So I guess ONE of my backups gets sent to the
    NAS, but I try to keep a proper 3-2-1 backup system - its getting so
    big, however, that I'm gonna have to buy more hardware in order to get
    an offsite backup.

    I do something similar - store media locally on a spinning drive. Use
    Resilio Sync to back up to my Synology. Use Synology backup to back up
    to an external USB drive.

    One really cool thing I've been looking into - is linking up with
    someone else who has the same sized NAS as I do, and doing the remote backups with them. I give them access to some share on my NAS/PBS, and they do the same for me, and we send encrypted backups offsite; sharing the hardware we run so we both have a FINAL 3-2-*1* backup offsite... I just need to find someone else who runs a 100TB+ NAS. :P

    If you want to rent out space on your NAS, I might have a business
    model for you. :) I'm looking into AT&T fiber, and once I'm off of
    Comcast with their bandwidth caps, am looking at backing up to the
    cloud somewhere as another layer of protection. I've only got around
    3TB I'm backing up.




    --- MultiMail/Win v0.52
    * Origin: realitycheckBBS.org -- information is power. (1337:3/178)
  • From paulie420@1337:3/129 to deon on Thu Dec 19 19:08:54 2024
    so we both have a FINAL 3-2-*1* backup offsite... I just need to find someone else who runs a 100TB+ NAS. :P

    How much data is on that 100TB nas backup?

    Well; I'm currently @ 30TB or so, but for THAT offsite backup I wouldn't even mind finding a buddy for personal photos/stuff that would be much less. (Or, for the whole enchilada!!)

    Restoring from it would be a pain though, while I'm 1000MBs down, I'm
    only 50MBs up at the moment, hoping to change that to 250down/100up soon...

    Thought I'd mention it :)

    Thanks kindly; and I'm the same - I've been eying a much cheaper (so they say, anyway) fiber internet thats available here w/ 1Gbps/1Gbps - but it has that weird IP that changes and makes another step for running the public-facing services... (bbs, etc)



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A49 2024/05/29 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (1337:3/129)
  • From deon@1337:2/101 to poindexter FORTRAN on Fri Dec 20 15:40:53 2024
    Re: Re: FInally hit a wall...
    By: poindexter FORTRAN to paulie420 on Thu Dec 19 2024 09:15 am

    Howdy,

    If you want to rent out space on your NAS, I might have a business
    model for you. :) I'm looking into AT&T fiber, and once I'm off of
    Comcast with their bandwidth caps, am looking at backing up to the
    cloud somewhere as another layer of protection. I've only got around
    3TB I'm backing up.

    Have you heard of tahoe-lafs?

    I played with it a few years ago - and its design is perfect for this.

    It's effectively raid (or erasure coding) across unreliable targets. IE: Everybody provides capacity, and data stored is split, encrypted and erasure coded across everybody's drives (so no single person has a full file - and the data is encrypted anyway).

    If a node goes down, your data is still recoverable from other targets - upto the parity setting. IE: If you set parity to 3, then 3 systems can go offline and your data is still accessible.

    The trouble I had with it, is health check and repair, there didnt seem to be an easy way to manage, validate and repair as a hosting "node". IE: If I had two drives, one failed and I replaced it with a new drive, I couldnt recover all the "chunks" for data hosted there - only the data owner could.

    Development was slow as well...


    ...����
    --- SBBSecho 3.20-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (1337:2/101)
  • From poindexter FORTRAN@1337:3/178 to deon on Fri Dec 20 10:00:45 2024
    deon wrote to poindexter FORTRAN <=-

    Have you heard of tahoe-lafs?

    No, I haven't. Sounds interesting!

    I played with it a few years ago - and its design is perfect for this.

    It's effectively raid (or erasure coding) across unreliable targets.
    IE: Everybody provides capacity, and data stored is split, encrypted
    and erasure coded across everybody's drives (so no single person has a full file - and the data is encrypted anyway).

    I'm thinking Pied Piper, now. :)

    If a node goes down, your data is still recoverable from other targets
    - upto the parity setting. IE: If you set parity to 3, then 3 systems
    can go offline and your data is still accessible.

    That would be an interesting experiment, if a group of people got
    together and offered x amount of disk space and bandwidth into a storage
    pool.


    --- MultiMail/Win v0.52
    * Origin: realitycheckBBS.org -- information is power. (1337:3/178)
  • From deon@1337:2/101 to poindexter FORTRAN on Sat Dec 21 20:25:11 2024
    Re: Re: FInally hit a wall...
    By: poindexter FORTRAN to deon on Fri Dec 20 2024 10:00 am

    Howdy,

    That would be an interesting experiment, if a group of people got
    together and offered x amount of disk space and bandwidth into a storage pool.

    So I'm in - I've been meaning to take another look at tahoe to see how it has progressed...

    I'll look at making a docker image, so that's easier to install/setup. If others are interested (not necessarily using the docker image), then let me know - it'd be good to see how well it works - and performs.

    Ideally there should be 5 minimum in the cluster, but more is better ;)


    ...����
    --- SBBSecho 3.20-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (1337:2/101)