This article <https://www.zdnet.com/article/linux-pc-acting-up-how-to-check-for-bad-blocks-on-a-hard-drive-before-its-too-late/>
is, I would say, more of a starting point for thinking about the
issue, rather than the final word on what to do.
I used to use the badblocks utility a lot, too. However, the last I
checked, it had limitations on the size of disks it could cope with --
a 12-terabyte drive was beyond its capabilities. So I had to write my
own disk-scanning utility <https://gitlab.com/ldo/scan_disks/>. (I
don’t bother much with SMART -- I think it’s mostly a waste of time.)
Frankly, I would not like to continue using a disk that had bad blocks
on it. I did sort-of try that once -- I had a 3-terabyte drive that
developed errors within the warranty period. Before taking it back to
the shop, I wrote zeroes over the whole drive, and as a result of
that, the bad blocks disappeared. And the shop couldn’t find anything
wrong with it, so they refused a warranty return.
That drive did last a few more years in my backup machine, before
developing more bad sectors, after which I retired it.
This article <https://www.zdnet.com/article/linux-pc-acting-up-how-to-check-for-bad-blocks-on-a-hard-drive-before-its-too-late/>
is, I would say, more of a starting point for thinking about the
issue, rather than the final word on what to do.
I used to use the badblocks utility a lot, too. However, the last I
checked, it had limitations on the size of disks it could cope with --
a 12-terabyte drive was beyond its capabilities. So I had to write my
own disk-scanning utility <https://gitlab.com/ldo/scan_disks/>. (I
don’t bother much with SMART -- I think it’s mostly a waste of time.)
Frankly, I would not like to continue using a disk that had bad blocks
on it. I did sort-of try that once -- I had a 3-terabyte drive that
developed errors within the warranty period. Before taking it back to
the shop, I wrote zeroes over the whole drive, and as a result of
that, the bad blocks disappeared. And the shop couldn’t find anything
wrong with it, so they refused a warranty return.
That drive did last a few more years in my backup machine, before
developing more bad sectors, after which I retired it.
On 2025-07-31 02:26, Lawrence D'Oliveiro wrote:
This article
<https://www.zdnet.com/article/linux-pc-acting-up-how-to-check-for-bad-blocks-on-a-hard-drive-before-its-too-late/>
is, I would say, more of a starting point for thinking about the
issue, rather than the final word on what to do.
I used to use the badblocks utility a lot, too. However, the last I
checked, it had limitations on the size of disks it could cope with --
a 12-terabyte drive was beyond its capabilities. So I had to write my
own disk-scanning utility <https://gitlab.com/ldo/scan_disks/>. (I
don’t bother much with SMART -- I think it’s mostly a waste of time.)
Frankly, I would not like to continue using a disk that had bad blocks
on it. I did sort-of try that once -- I had a 3-terabyte drive that
developed errors within the warranty period. Before taking it back to
the shop, I wrote zeroes over the whole drive, and as a result of
that, the bad blocks disappeared. And the shop couldn’t find anything
wrong with it, so they refused a warranty return.
This is one of the SMART features.
Even showing the bad blocks is not enough to get a refund/return.
That drive did last a few more years in my backup machine, before
developing more bad sectors, after which I retired it.
I have a dead Seagate 3TB disk, lost all the data inside.
I have a dead Seagate 3TB disk, lost all the data inside.
I have a dead Seagate 3TB disk, lost all the data inside.
On Thu, 31 Jul 2025 15:21:19 +0200, Carlos E.R. wrote:
I have a dead Seagate 3TB disk, lost all the data inside.
This is why I have a backup machine.
"Carlos E.R." <[email protected]d> wrote:
I have a dead Seagate 3TB disk, lost all the data inside.
Probably a decade old?
On 2025-08-01 10:05, Marc Haber wrote:
"Carlos E.R." <[email protected]d> wrote:
I have a dead Seagate 3TB disk, lost all the data inside.
Probably a decade old?
Probably. I think that model was infamous.
"Carlos E.R." <[email protected]d> wrote:
On 2025-08-01 10:05, Marc Haber wrote:
"Carlos E.R." <[email protected]d> wrote:
I have a dead Seagate 3TB disk, lost all the data inside.
Probably a decade old?
Probably. I think that model was infamous.
I apologize. I thought that we were talking about practical relevant
cases.
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
I think that Seagate model had many disk failing early. Defective model.
But as mine was working fine, I continued using it, till one day it
failed completely. Probably a pcb replacement would work, but that's not >something I can do.
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
My backups are done with rsync, or script wrappers around rsync. They are effectively mirror filesystems, not in any special “backup” format. These are the easiest to verify, and restoration can be done with the same file- manipulation commands I use every day.
In particular, that means that, in a moment of crisis (which is what
happens when you find you’ve lost data), I am doing the restoration using familiar commands with which I am (hopefully) less likely to make
mistakes.
Once an rsync backup deleted the original.
I was using the --delete parameter, which is intended to delete files
that are gone from the source, and delete them in the destination (when
you overwrite a backup). Unfortunately, the paths were messed up, and it deleted the files on the source of another different backup.
On Sat, 2 Aug 2025 03:39:19 +0200, Carlos E.R. wrote:
Once an rsync backup deleted the original.
I was using the --delete parameter, which is intended to delete files
that are gone from the source, and delete them in the destination (when
you overwrite a backup). Unfortunately, the paths were messed up, and it
deleted the files on the source of another different backup.
A computer is a powerful tool. It will do exactly what you tell it to do,
no more, no less.
Try --delete-after, which postpones all the deletions until after the new
and updated files have been successfully copied -- and which doesn’t do
any deletions at all, if there were errors in those copies.
Le 01-08-2025, Marc Haber <[email protected]> a écrit :
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
When a backup is never tested, it's called a Schrödinger's backup.
On 2025-08-02 01:55, Lawrence D'Oliveiro wrote:
On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
My backups are done with rsync, or script wrappers around rsync. They are
effectively mirror filesystems, not in any special “backup” format. These
are the easiest to verify, and restoration can be done with the same
file-
manipulation commands I use every day.
In particular, that means that, in a moment of crisis (which is what
happens when you find you’ve lost data), I am doing the restoration using >> familiar commands with which I am (hopefully) less likely to make
mistakes.
Once an rsync backup deleted the original.
I was using the --delete parameter, which is intended to delete files
that are gone from the source, and delete them in the destination (when
you overwrite a backup). Unfortunately, the paths were messed up, and it deleted the files on the source of another different backup.
On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
My backups are done with rsync, or script wrappers around rsync.
Le 01-08-2025, Marc Haber <[email protected]> a écrit :
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
When a backup is never tested, it's called a Schrödinger's backup.
On Sat, 2 Aug 2025 03:39:19 +0200, Carlos E.R. wrote:
Once an rsync backup deleted the original.
I was using the --delete parameter, which is intended to delete files
that are gone from the source, and delete them in the destination (when
you overwrite a backup). Unfortunately, the paths were messed up, and it
deleted the files on the source of another different backup.
A computer is a powerful tool. It will do exactly what you tell it to do,
no more, no less.
Try --delete-after, which postpones all the deletions until after the new
and updated files have been successfully copied -- and which doesn’t do
any deletions at all, if there were errors in those copies.
Lawrence D'Oliveiro <[email protected]d> wrote:
On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
My backups are done with rsync, or script wrappers around rsync.
You're comparing 1998 with 2025. Do you think that is fair?
The OS in question wasn't even unix back then.
Btw, script wrappers around rsync do not scale to even a small fleet.
On Sat, 02 Aug 2025 10:07:02 +0200, Marc Haber wrote:
Lawrence D'Oliveiro <[email protected]d> wrote:
On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
My backups are done with rsync, or script wrappers around rsync.
You're comparing 1998 with 2025. Do you think that is fair?
Andrew Tridgell submitted his PhD thesis, in which he developed rsync’s algorithm for comparing files across the network without having to copy
the entirety of either side onto the other, in 1998, as I recall.
At that time (and for a few years after), people would have been doing backups using commands like “cp -u”. Which work well enough, as far as they go.
The OS in question wasn't even unix back then.
The clunkiness of such compared to *nix systems was already becoming apparent.
Btw, script wrappers around rsync do not scale to even a small fleet.
They do, actually. Remember, rsync itself is doing all the grunt work, and that can happily handle hundreds of gigabytes, or terabytes of data. (Just speaking from personal experience.)
And given that CERN, for example, is a heavy Linux user, I imagine they
use rsync to deal with their petabytes of data from the LHC and other projects as well.
How do I know this? Because being a bunch of scientists, if they had to develop a more robust tool for that purpose, they would have announced and open-sourced it by now.
On Tue, 5 Aug 2025 19:54:56 -0400, c186282 wrote:
Best modern plan = TWO local copies, on different machines/media, and
one dup (maybe just the 'important stuff') to 'cloud' storage. Not
too expensive. Just automate it all so it works overnight.
We used portable drives that were off premises. That was part of the
formal backups the build guy did. Informally everyone checked out the
entire source tree in Subversion and kept it updated so unless the
building was nuked there would be a backup.
For my works in progress I'd have the code on two or three machines plus
the corporate OneDrive.
My personal projects at home usually are copied to two or three machines.
I don't bother with the cloud since there's nothing I couldn't recreate.
Documentation, SDKs, IDEs and so forth are safely stored wherever I got
them from in the first place, and probably newer versions to boot.
I do have some mp3s on a WD Passport and assorted thumb drives because no
way in hell do I want to rip a shelf full of CDs again. At least they're easier than cassettes.
Note - ALWAYS pre-encrypted before sending to cloud.
Encrypt local, random name, copy to DBx, Rename
re-date and whatever else is possible afterwards.
OpenSSL can do most of the security stuff. I just
do NOT believe claims of their own 'security'.
Send 'em binary garbage.
c186282 <[email protected]> wrote:
Note - ALWAYS pre-encrypted before sending to cloud.
Encrypt local, random name, copy to DBx, Rename
re-date and whatever else is possible afterwards.
OpenSSL can do most of the security stuff. I just
do NOT believe claims of their own 'security'.
Send 'em binary garbage.
Sure, although keep in mind that one day in the future they might
be able to decrypt even your old self-encrypted files easily, like
encrypted files from the 1990s can be today. Even if they say that
data is deleted, there's no way to be sure that someone didn't
keep/steal a copy.
On 8/6/25 5:14 PM, Computer Nerd Kev wrote:
c186282 <[email protected]> wrote:
Note - ALWAYS pre-encrypted before sending to cloud.Sure, although keep in mind that one day in the future they might be
Encrypt local, random name, copy to DBx, Rename
re-date and whatever else is possible afterwards.
OpenSSL can do most of the security stuff. I just
do NOT believe claims of their own 'security'.
Send 'em binary garbage.
able to decrypt even your old self-encrypted files easily, like
encrypted files from the 1990s can be today. Even if they say that
data is deleted, there's no way to be sure that someone didn't
keep/steal a copy.
Well, 'relevance' tends to fall off over time. Ten or twenty year
old bank/personnel records and such ... not very interesting.
Even now I'm sure the spooks can crack AES256 and likely most of the
others. Quantum makes that much faster. Have seen allegedly
'quantum-resistant' ciphers, but not sure how well they live up to
the label yet. There's also the issue of govts demanding backdoors -
ciphers ARE still classed as 'munitions'.
A (currently hypothetical) quantum computer of sufficient size would in principle reduce AES-256 key search to 2^128 time complexity (with a
rather larger constant factor than a classical computer) - far beyond
what human civilization can achieve. Adding more quantum computers
doesn’t help as much as you might imagine since (AIUI) Grover’s
algorithm scales sublinearly.
On 8/6/25 5:14 PM, Computer Nerd Kev wrote:
c186282 <[email protected]> wrote:
Note - ALWAYS pre-encrypted before sending to cloud.
Encrypt local, random name, copy to DBx, Rename
re-date and whatever else is possible afterwards.
OpenSSL can do most of the security stuff. I just
do NOT believe claims of their own 'security'.
Send 'em binary garbage.
Sure, although keep in mind that one day in the future they might
be able to decrypt even your old self-encrypted files easily, like
encrypted files from the 1990s can be today. Even if they say that
data is deleted, there's no way to be sure that someone didn't
keep/steal a copy.
Well, 'relevance' tends to fall off over time.
Ten or twenty year old bank/personnel records
and such ... not very interesting.
On 2025-08-05, candycanearter07 wrote:
Carlos E.R. <[email protected]d> wrote at 01:39 this Saturday (GMT): >>> On 2025-08-02 01:55, Lawrence D'Oliveiro wrote:
On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
backup was invalid the hard way.
My backups are done with rsync, or script wrappers around rsync. They are >>>> effectively mirror filesystems, not in any special “backup” format. These
are the easiest to verify, and restoration can be done with the same file- >>>> manipulation commands I use every day.
In particular, that means that, in a moment of crisis (which is what
happens when you find you’ve lost data), I am doing the restoration using
familiar commands with which I am (hopefully) less likely to make
mistakes.
Once an rsync backup deleted the original.
I was using the --delete parameter, which is intended to delete files
that are gone from the source, and delete them in the destination (when
you overwrite a backup). Unfortunately, the paths were messed up, and it >>> deleted the files on the source of another different backup.
That's why you have backups of backups.
But... what if these fail too!?
10 PRINT "That's why you have"
20 PRINT "backups of backups of"
30 GOTO 20
(Ok, in reality, a second level probably already drives the likelihood
down a lot. But, of course, this gets to be an engineering problem, and
one gets to ask what could possibly go wrong outside of these
likelihoods.
And one could do things such as not using the same
approach/scripts/software for both levels. Just like NASA had BFS
developed separately from PASS.)
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 08:39:41 |
| Calls: | 12,100 |
| Files: | 15,003 |
| Messages: | 6,517,955 |