besides rsync, do you know any other software/service for increment backup?
On Fri, Mar 14, 2025 at 2:08?AM tim wade <[email protected]> wrote:
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
I use Duplicity to backup a webserver and MySQL database. The
directories are different, but the operations are the same.
Jeff
Hello
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
Thank you.
Hello
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
Thank you.
It's not truly "incremental", though combines compression,
deduplication and optional encryption, which may improve on that.
I use borg on rsync.net for remote backup. The only remote providers I know of are:
https://rsync.net
https://www.borgbase.com
tim wade (HE12025-03-14):
besides rsync, do you know any other software/service for increment backup?
Ahem. rsync is not a backup tool.
rsync can be *part of* a backup tool, for example in rsnapshot, or
manually with filesystem snapshots, but rsync alone does not allow you
to recover a file you damaged, which is part of the job description of a backup tool.
Hello
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
Hello
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment backup?
Thank you.
On 14/03/2025 08.07, tim wade wrote:
besides rsync, do you know any other software/service for increment backup?
Dirvish - "Disk based virtual image network backup system."
Dirvish can create user browseable (daily) backup directories.
tim wade <[email protected]> wrote:
HelloMost of the replies so far seem to have ignored 'increment[al]'. While complete backups are sometimes useful I find incremental ones much
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
better most of the time and they allow one to keep really old backups
without running out of space.
There are a couple of programs that use rsync to do incremental
backups, the rsync resources page (https://rsync.samba.org/resources.html) provides some possibilities.
I used to use rsyncbackup which worked OK but I decided I wanted more
ways to configure things so I wrote my own system which is simply a
bash script that uses rsync to do the actual file copying and making
hard links.
I have hourly backups of /home which are kept until they're five weeks
old, these often save me from myself when I delete or change files by
mistake or whatever. These are kept on the same machine but on a
separate drive.
Then I have long term backups which are made daily to a remote
machine, every day's backup is kept for a month and then they are
selectively thinned out so I have monthly backups for the past year
and then yearly back ups 'for ever'. They go back around twwnty years
or more!
I did actually dig around one of the very old backups (well over ten
years old) for some photos my daughter had lost - and I found them!
One of the big advantages of using rsync's hard-link for doing this is
that the backups all look like a complete /home directory and you can
dig around in them without any effort at all, no unpacking, no
decoding, nothing, it's all just the files as you know them.
I do use .rsync-filter to filter out things that either don't need
backup or that change so frequently that they make backups pointless,
things like .bash_history and .xsession-errors for example.
tim wade <[email protected]> wrote:
HelloMost of the replies so far seem to have ignored 'increment[al]'. While complete backups are sometimes useful I find incremental ones much
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
better most of the time and they allow one to keep really old backups
without running out of space.
Depends on, what you prefer: in console, or with GUIn as conjob or whatever
That sounds most interesting Chris. Something that can work as you work
w/o interfering with your work.
Can you write up some docs, pkg and publish it?
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment
backup?
A tool that cannot be automated is not a backup tool.
Corollary: a GUI-only tool is not a backup tool.
Regards,
besides rsync, do you know any other software/service for increment backup?I use borg. It stores files in its own archive format with
deduplication and compression. 4 backups of 32G /+/home of my old
netbook created every month stored in ~11GB backup directory.
Slower than rsync, eat more cpu, need borg binary at remote storage
(or mount storage local). Chosen backup archive (see borg list
$backup-path) may be r/o mounted for restoring some files if needed.
Hello
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment backup?
Thank you.
"Incremental backup" implies "full backup" -- e.g. make full backups on Sunday nights and make incremental backups Monday through Saturday nights, etc..
I use Bup, which provides a fairly similar featureset to Borg (tho
doesn't support encryption yet). AFAIK the main difference is that
instead of its own archive format, Bup uses the Git repository format.
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
On Fri Mar 14, 2025 at 9:14 PM GMT, Stefan Monnier wrote:
I use Bup, which provides a fairly similar featureset to Borg (thoCan bup purge old backups?
doesn't support encryption yet). AFAIK the main difference is that
instead of its own archive format, Bup uses the Git repository format.
I haven't actually tried Borg, but I'm surprised by "slower than rsync,
eat more cpu" because I'd expect it to perform similarly to Bup, and
(for my use case at least) Bup performs incremental backups
significantly faster and using less CPU (especially on the server to
which I send my backups) than the rsync system I used before.
Yes, it has `bup prune-older`.
I must admit I don't use it, but I'd expect it to be fairly
costly/slow.
On 2025-03-14, Jerome BENOIT <[email protected]> wrote:
Hi,
backup2l is simple and has been reliable for me for years.
Is that a Debian package?
On 2025-03-14 [FR], Russell Harris wrote:
I use git. I keep terminal open running a ssh connection open to the
backup system. Whenever I wish to save the state of the system, I
switch to the terminal and execute git commit. To check the previous
state of a file, Emacs provides git-timemachine.
Simple and miminal hassle.
Would you mind sharing with us some details about how you do this (and
how you decided to do so), or at least give some pointers to more
information about this from other sources?
= With git, I can do a hundred backups in a morning or afternoon
session without worry about consumption of disk space. I am not
concerned with an orderly and uncluttered commit record; my concern is
the ability to recover.
= The time required for each backup is negligible:
(1) switch to a virtual terminal
(2) type: git commit -a -m journal #050
(3) switch back to Emacs
On Fri, Mar 14, 2025 at 02:07:53PM +0800, tim wade wrote:
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
I use git. I keep terminal open running a ssh connection open to the
backup system. Whenever I wish to save the state of the system, I
switch to the terminal and execute git commit. To check the previous
state of a file, Emacs provides git-timemachine.
Simple and miminal hassle.
git fails to preserve ownership, permissions or timestamps. While this
may not be relevant to your usecase, for example backing up /etc would
be catastrophic (which is why we have etckeeper)
On 3/15/25 12:50, Tim Woodall wrote:
git fails to preserve ownership, permissions or timestamps. While this
may not be relevant to your usecase, for example backing up /etc would
be catastrophic (which is why we have etckeeper)
I am curious -- if I make my /etc directory tree into a version control system working directory (Git or otherwise), please explain how this would
be catastrophic.
On Sat, Mar 15, 2025 at 01:18:45PM -0700, David Christensen wrote:
I am curious -- if I make my /etc directory tree into a version control
system working directory (Git or otherwise), please explain how this would >> be catastrophic.
/etc has things in it which require specific ownership and permissions.
These aren't stored in git or in many other version control systems. It
will be fine until you actually restore anything, at which point the
files will get ownership/group of your user and mode 0644 or 0755
depending upon whether they were executable or not when committed.
Thanks,
Andy
Hello
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
besides rsync, do you know any other software/service for increment backup?
Thank you.
</blockquote></div></body></html>------05ARS48RL5XN4T0DYBBF8YWLCOJ9SJ--
Hi,
On Sat, Mar 15, 2025 at 01:18:45PM -0700, David Christensen wrote:
On 3/15/25 12:50, Tim Woodall wrote:
git fails to preserve ownership, permissions or timestamps. While this may not be relevant to your usecase, for example backing up /etc would
be catastrophic (which is why we have etckeeper)
I am curious -- if I make my /etc directory tree into a version control system working directory (Git or otherwise), please explain how this would be catastrophic.
/etc has things in it which require specific ownership and permissions.
On Fri, 14 Mar 2025, Russell L. Harris wrote:[...]
On Fri, Mar 14, 2025 at 02:07:53PM +0800, tim wade wrote:
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
I use git. I keep terminal open running a ssh connection open to the backup system. Whenever I wish to save the state of the system, I
switch to the terminal and execute git commit. To check the previous
state of a file, Emacs provides git-timemachine.
Simple and miminal hassle.
git fails to preserve ownership, permissions or timestamps. While this may
Git *does* preserve permissions [1]. For the ownerships (and more accurate mtime, atime and ctime) cf. etckeeper.
On Sun, 16 Mar 2025, [email protected] wrote:
Git *does* preserve permissions [1]. For the ownerships (and more accurate mtime, atime and ctime) cf. etckeeper.
Git only tracks the execute bit. And because it always writes a new file rather than truncate then write by most editors, the permissions get reset very easily:
On Sat, Mar 15, 2025 at 07:50:27PM +0000, Tim Woodall wrote:
git fails to preserve ownership, permissions or timestamps. While this may[...]
Half-wrong. Git doesn't preserve ownerships, but it does preserve
permissions (the POSIX things).
As for timestamps, it records the last commit timestamp,
which may be a "good enough" approximation of the mtime or
not.
Hi,
backup2l is simple and has been reliable for me for years.
Cheers,
Jerome
Git is an awesome tool that can be utilized with a wide variety of
files, not just software source code or text. However, it faces
scalability challenges with larger files, such as videos. While
extensions like git-annex can assist in managing these larger files,
they can struggle when dealing with tens of thousands or more of files.
Additionally, Git's core feature of maintaining immutable history is not well-suited for backing up large, dynamic datasets, where retaining the entire history indefinitely may be undesirable. There may also be
privacy and legal concerns related to maintaining permanent records of
all changes, since establishing a retention policy is difficult.
So, most people cant't have their whole $HOME in a Git repository.
Moreover, storing Git repositories on the same storage device as your original files only safeguards your data against your own mistakes. This
is analogous to the (opposite) misconception that RAID systems serve as
a backup solution; they only protect against hardware failures of one or
more disks, and happily mirror all your mistakes for you. Proper backup system need to protect your data from most (ideally all) data loss
scenarios.
On Fri, 14 Mar 2025, Russell L. Harris wrote:
On Fri, Mar 14, 2025 at 02:07:53PM +0800, tim wade wrote:git fails to preserve ownership, permissions or timestamps. While this
I plan to make increment backup for my home dir.
It's currently in the size of 1xx GB.
I use git. I keep terminal open running a ssh connection open to the
backup system. Whenever I wish to save the state of the system, I
switch to the terminal and execute git commit. To check the previous
state of a file, Emacs provides git-timemachine.
Simple and miminal hassle.
may not be relevant to your usecase, for example backing up /etc would
be catastrophic (which is why we have etckeeper)
For a homedir, ownership may not be a problem but permissions might be.
IIRC, for example, ssh (and sshd) will not work properly if the
permissions on ~/.ssh are wrong.
git will also not backup (or even see) extended attributes.
Jerome BENOIT wrote:
Hi,
backup2l is simple and has been reliable for me for years.
Cheers,
Jerome
interesting, at first glance it might help me out, but
i don't know for sure. i'm a bit worried though that the
debian package doesn't look like it is actively maintained.
i have old backups in tar format. will backup2l allow
those to be used as a source in such a way that each one
is not duplicated without having to unpack them first?
ages ago i used bup and it was ok, but i really did not
like the presentation for going back and finding things.
this was a long time ago though and now perhaps it isn't
so bad. doesn't matter now as i have no old bup anyplace.
ultimately i really need a way to do backups that will
deduplicate
and must be 100% bulletproof and stable.
songbird
songbird wrote:...
ultimately i really need a way to do backups that will
deduplicate
I do not see what you mean.
and must be 100% bulletproof and stable.
backup2l is stable (it uses stable tools (as tar) and it is written in an easy-to-read bash).
For the bulletproof part, the choice of the compressor might be crucial: backup2l allows to
choose your favorite one.
Jerome BENOIT wrote:
songbird wrote:...
ultimately i really need a way to do backups that will
deduplicate
I do not see what you mean.
my old backups are not incremental so they will contain a
lot of files that would be identical copies to other backups.
and must be 100% bulletproof and stable.
backup2l is stable (it uses stable tools (as tar) and it is written in an easy-to-read bash).
For the bulletproof part, the choice of the compressor might be crucial: backup2l allows to
choose your favorite one.
sounds ok to me, but i may also decide for some things to
not compress at all. very large archives of materials
already compressed would be one i'd like to skip messing
with at all.
songbird
Not compressing the tarballs is possible.
Jerome BENOIT wrote:
...
Not compressing the tarballs is possible.
i don't think it is really deduplicating but that is ok
for now. what i've been playing with this afternoon seems
to be going ok.
i'm not sure i have a daily run that is working since i
don't always leave my machine on, but that too is also ok
as there's a good chance i'll just create a script that i
can run manually on my own as i shutdown at night.
songbird
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 716 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 54:16:50 |
| Calls: | 12,117 |
| Calls today: | 8 |
| Files: | 15,010 |
| Messages: | 6,518,629 |
| Posted today: | 2 |