Summary:: Devices need just as much cache coherence as cores--maybe
more.
[email protected] (MitchAlsup1) writes:
Summary:: Devices need just as much cache coherence as cores--maybe
more.
Does a uart need cache coherence? How about a SPI or MMC controller?
With the narrow memory interfaces, are cores starved for memory
bandwidth?
On Sun, 11 May 2025 07:47:53 -0700, Al Kossow wrote:
With the narrow memory interfaces, are cores starved for memory
bandwidth?
Low memory bandwidth has been a chronic issue since the “wait states” of the 1980s.
That’s why we have memory caches.
On Mon, 12 May 2025 00:30:37 +0000, MitchAlsup1 wrote:
On Sun, 11 May 2025 23:46:17 +0000, Lawrence D'Oliveiro wrote:
That’s why we have memory caches.
Which architects tend to only understand what happens when these caches
are attached to CPUs and not "Joe Random Bus Master",
One of my pet peeves is disk drives with memory caches in them. Why?
On Sun, 11 May 2025 23:46:17 +0000, Lawrence D'Oliveiro wrote:
That’s why we have memory caches.
Which architects tend to only understand what happens when these caches
are attached to CPUs and not "Joe Random Bus Master",
On Mon, 12 May 2025 00:30:37 +0000, MitchAlsup1 wrote:
On Sun, 11 May 2025 23:46:17 +0000, Lawrence D'Oliveiro wrote:
That’s why we have memory caches.
Which architects tend to only understand what happens when these caches
are attached to CPUs and not "Joe Random Bus Master",
One of my pet peeves is disk drives with memory caches in them. Why?
Lawrence D'Oliveiro wrote:
One of my pet peeves is disk drives with memory caches in them. Why?
=20
For reads it allows the disk to always read full sets of sectors, the=20 >following blocks are likely to be needed soon anyway.
For writes, as long as the drive has enough energy (maybe in the form of =
spinning inertia, or a hefty cap?) the always be able to save the buffer =
cache to spinning rust, it can allow operations to complete immediately, =
or as soon as the data has been transferred into the disk cache.
Since all disks are using linear sector (or 4K block?) addressing these=20 >days, instead of head/cylinder/sector, a little bit of cache can help=20
hide the tiny time glitches when the disk has to reposition.
On Mon, 12 May 2025 1:19:44 +0000, Lawrence D'Oliveiro wrote:
One of my pet peeves is disk drives with memory caches in them. Why?
Makes the disk faster, decreasing pressure on the "Disk Cache" in DRAM.
Lawrence D'Oliveiro wrote:
For reads it allows the disk to always read full sets of sectors, the following blocks are likely to be needed soon anyway.
One of my pet peeves is disk drives with memory caches in them. Why?
For writes, as long as the drive has enough energy (maybe in the form of spinning inertia, or a hefty cap?) the always be able to save the buffer cache to spinning rust, it can allow operations to complete immediately,
or as soon as the data has been transferred into the disk cache.
On SSDs DRAM cache is also used for storing the logical-to-physical
sector mapping of the flash translation layer; accessing it on flash is apparently too slow.
Terje Mathisen <[email protected]> writes:
Lawrence D'Oliveiro wrote:
One of my pet peeves is disk drives with memory caches in them. Why?
=20
For reads it allows the disk to always read full sets of sectors, the=20
following blocks are likely to be needed soon anyway.
Yes.
For writes, as long as the drive has enough energy (maybe in the form of = >>
spinning inertia, or a hefty cap?) the always be able to save the buffer = >>
cache to spinning rust, it can allow operations to complete immediately, = >>
or as soon as the data has been transferred into the disk cache.
I used to think so, but someone (who appeared knowledgable) corrected
that view and told me that all that HDDs ever do on power loss is to
complete the current sector.
Since all disks are using linear sector (or 4K block?) addressing these=20 >> days, instead of head/cylinder/sector, a little bit of cache can help=20
hide the tiny time glitches when the disk has to reposition.
An alternative is to skew the start of the first sector on the next
track to take the repositioning time into account, and from what I
have read, that is what is done. It allows to complete a sequence of
sectors in one go, and then move on to the next sequence of sectors elsewhere, without having to wait for another disk rotation to be able
to write the sector that was missed in an unskewed drive.
On Mon, 12 May 2025 08:05:56 +0200, Terje Mathisen wrote:
For reads it allows the disk to always read full sets of sectors, theLeave that up to the OS I/O optimization algorithms. Because they know things about the data that the drive doesn’t.
following blocks are likely to be needed soon anyway.
On Mon, 12 May 2025 08:41:57 GMT, Anton Ertl wrote:
On SSDs DRAM cache is also used for storing the logical-to-physicalThere is a lot of complicated firmware in SSDs to make them look as
sector mapping of the flash translation layer; accessing it on flash is
apparently too slow.
much like a traditional hard drive as possible, so that traditional
hard drive filesystems can be used unchanged. This firmware has been
known to have bugs in it.
Whereas the Linux kernel includes a few filesystems purpose-designed
for operation on raw flash devices, that integrate wear-levelling etc
right into the block allocation algorithms. Wouldn’t it be much
better (more efficient and more reliable) to get rid of most of that
firmware layer, and use these sorts of filesystems directly?
But the drive also knows things about the data that the OS can't know
(things that have to do with the physical location of the data on the platters).
More reliable, I don't know: to get comparable performance, you'll need comparable complexity ...
- Those "few filesystems" aren't nearly good enough to compete with
a normal filesystem running on top of a typical SSD. Simply because
those filesystems have not been designed for those kinds of uses.
But the drive also knows things about the data that the OS can't knowNo it doesn’t.
(things that have to do with the physical location of the data on the
platters).
More reliable, I don't know: to get comparable performance, you'llFewer layers ⇒ less complexity.
need comparable complexity ...
- Those "few filesystems" aren't nearly good enough to compete withOf course they are.
a normal filesystem running on top of a typical SSD. Simply because
those filesystems have not been designed for those kinds of uses.
You don’t think there are filesystem experts working on the
Linux kernel?
On Mon, 12 May 2025 08:05:56 +0200, Terje Mathisen wrote:
Lawrence D'Oliveiro wrote:
For reads it allows the disk to always read full sets of sectors, the
One of my pet peeves is disk drives with memory caches in them. Why?
following blocks are likely to be needed soon anyway.
Leave that up to the OS I/O optimization algorithms. Because they know
things about the data that the drive doesn’t.
For writes, as long as the drive has enough energy (maybe in the form of
spinning inertia, or a hefty cap?) the always be able to save the buffer
cache to spinning rust, it can allow operations to complete immediately,
or as soon as the data has been transferred into the disk cache.
In other words, telling lies to the OS that the write has completed when
it hasn’t.
This kind of thing can really stuff up filesystem integrity
guarantees.
In this case the drive knows things that the OS does not: Consider that
the OS asked for sector N what happens when the arm finally has settled enough to read from the track, and the first sector it sees is sector
N+10.
On Mon, 12 May 2025 21:50:02 -0400, Stefan Monnier wrote:
But the drive also knows things about the data that the OS can't know
(things that have to do with the physical location of the data on the
platters).
No it doesn’t. Consider a journalling filesystem, where the journal entry must be written first before the actual filesystem update. If the drive reorders the writes to do the latter first, there go your filesystem integrity guarantees.
On Tue, 13 May 2025 07:40:35 GMT, Anton Ertl wrote:
In this case the drive knows things that the OS does not: Consider that
the OS asked for sector N what happens when the arm finally has settled
enough to read from the track, and the first sector it sees is sector
N+10.
The OS isn’t likely to ask for one sector at a time.
On Mon, 12 May 2025 21:50:02 -0400, Stefan Monnier wrote:
But the drive also knows things about the data that the OS can't know
(things that have to do with the physical location of the data on the
platters).
No it doesn’t.
Consider a journalling filesystem, where the journal entry
must be written first before the actual filesystem update. If the drive reorders the writes to do the latter first, there go your filesystem integrity guarantees.
More reliable, I don't know: to get comparable performance, you'll need
comparable complexity ...
Fewer layers ⇒ less complexity.
- Those "few filesystems" aren't nearly good enough to compete with
a normal filesystem running on top of a typical SSD. Simply because
those filesystems have not been designed for those kinds of uses.
Of course they are. You don’t think there are filesystem experts working
on the Linux kernel?
No one is arguing that host based file caches are bad. It is simply the
fact that there are situations where drive caches are a useful
addition ...
On Tue, 13 May 2025 08:33:15 -0700, Stephen Fuld wrote:
No one is arguing that host based file caches are bad. It is simply the
fact that there are situations where drive caches are a useful
addition ...
You can tell that’s wrong because the drive cache is slower than the OS filesystem cache.
waste of time ... unless the slower cache is much larger.
This is why, for example, we typically have 3 levels of RAM cache between
the CPU and main memory these days. There is a factor of about 100:1 in relative speeds, so to bridge the gap we need multiple caches of various intermediate speeds, and you will notice their sizes are inversely related
to their speeds.
A drive cache can never be as big as main RAM on a modern PC. That’s why the drive cache is useless.
On Tue, 13 May 2025 08:33:15 -0700, Stephen Fuld wrote:
No one is arguing that host based file caches are bad. It is simply the
fact that there are situations where drive caches are a useful
addition ...
You can tell that’s wrong because the drive cache is slower than the OS >filesystem cache. Putting a slower cache in series with a faster one is a >waste of time ... unless the slower cache is much larger.
According to Scott Lurndal <[email protected]>:
Lawrence D'Oliveiro <[email protected]d> writes:
[something]
You really have no clue what you're talking about. Disk drive caches were >>never intended to be extensions of the host cache structure.
Er, you do know who you're arguing with?
Lawrence D'Oliveiro <[email protected]d> writes:
[something]
You really have no clue what you're talking about. Disk drive caches were >never intended to be extensions of the host cache structure.
According to Scott Lurndal <[email protected]>:
Lawrence D'Oliveiro <[email protected]d> writes:
[something]
You really have no clue what you're talking about. Disk drive caches were >>never intended to be extensions of the host cache structure.
Er, you do know who you're arguing with?
In other words, telling lies to the OS that the write has completed
when it hasn’t. This kind of thing can really stuff up filesystem
integrity guarantees.
On 5/13/2025 5:18 PM, Lawrence D'Oliveiro wrote:
You can tell that’s wrong because the drive cache is slower than the OS
filesystem cache.
I don't think that proves anything. I gave you an example of where the
drive cache speeded up the sequence of two reads to where it was faster
than two reads into the file cache.
Think of what a cache is for in the first place. The only reason they work
is because of the “principle of locality”. This can also be expressed as saying that typical patterns of data access by application programs follow
a Pareto distribution, less formally known by monikers like the “80/20 rule” or the “90/10 rule”.
On Tue, 13 May 2025 17:49:17 -0700, Stephen Fuld wrote:
On 5/13/2025 5:18 PM, Lawrence D'Oliveiro wrote:
You can tell that’s wrong because the drive cache is slower than the OS >>> filesystem cache.
I don't think that proves anything. I gave you an example of where the
drive cache speeded up the sequence of two reads to where it was faster
than two reads into the file cache.
Think of what a cache is for in the first place. The only reason they
work
is because of the “principle of locality”. This can also be expressed as saying that typical patterns of data access by application programs
follow
a Pareto distribution, less formally known by monikers like the “80/20 rule” or the “90/10 rule”.
Let’s use the “90/10 rule” name just to put some concrete numbers on the
whole idea: this says that 90% of (recent) data accesses will be to just
10% of the data.
For “recent”, let’s say “within the last 10 seconds”.
So the OS cache
should be big enough to hold that 10% of the data that the app accessed
in
about the last 10 seconds. Of course the precise set of frequently-
accessed data (the “working set”) will drift over time. So anything beyond
this last-10-seconds’ worth will require hitting the actual disk device, which will happen eventually.
At that point, the cache on the disk device is going to come into play. Trouble is it’s nowhere near this kind of size: it can only hold enough data for, say, about the last 1 second’s worth of accesses. So by the
time
the OS cache experiences a miss like this, the disk cache isn’t going to
be able to help much, since it is more likely than not going to miss as
well. To be useful, it needs to be about 10 times the size of the OS
cache, so it can hold more of the 90% of the data that the app wants to access less frequently.
But you can’t get drives like that.
On Sun, 18 May 2025 1:35:54 +0000, Lawrence D'Oliveiro wrote:
For “recent”, let’s say “within the last 10 seconds”.
Principle is reasonable, specified time is not.
A 4MB cache can be filled in 1ms when memory performs at (only) 4GB/s
(and the memory systems are up in the 40GB/s peak territory while caches remain rather small (several GB at most).)
Lawrence D'Oliveiro <[email protected]d> writes:
Think of what a cache is for in the first place. The only reason they work >> is because of the “principle of locality”. This can also be expressed as >> saying that typical patterns of data access by application programs follow >> a Pareto distribution, less formally known by monikers like the “80/20
rule” or the “90/10 rule”.
IBM "added" full-track "-13" cache to 3880 dasd control for 3380 disk
(ten records/track) ... claiming 90% "hit rate". Issue was that there
was a lot of sequential file reading ... the 1st record read for track
would be a "miss" but bring in the whole track, resulting in the next
nine reads being "hits".
system services offered option for application doing sequential i/o to specify full-track i/o (into processor memory) ... which would result
in the zero hit rate for the controller cache (IBM standard batch
operating system did contiguous allocation on file creation).
About the same time, we did system mod. that did highly efficient trace/capture of every record operation which was deployed on numerous production systems. Then traces were fed to sophisticated simulator that could vary algorithms, caches, kinds of caches, sizes of caches,
distribution of caches, etc.
Given a fixed amount of cache storage, it was always better to have a
global system cache ... than partitioned/distributed; except a few edge cases. Example, if device track cache could be used to immediately
start transfering data, rather having to rotate to start of track before starting transfer.
It didn't make sense to, after executing the full track read, since the
disk was positioned at the end of the track, and you had a good
indication that it was a sequential file read, to start caching the next track in anticipation of the next full track read?
On 19/05/2025 02:41, Stephen Fuld wrote:
It didn't make sense to, after executing the full track read, since
the disk was positioned at the end of the track, and you had a good
indication that it was a sequential file read, to start caching the
next track in anticipation of the next full track read?
It might if the disk was idle. But if there is a queue from another
process it probably will be better to do that instead.
On Mon, 19 May 2025 00:18:23 +0000, MitchAlsup1 wrote:
On Sun, 18 May 2025 1:35:54 +0000, Lawrence D'Oliveiro wrote:
For “recent”, let’s say “within the last 10 seconds”.
Principle is reasonable, specified time is not.
A 4MB cache can be filled in 1ms when memory performs at (only) 4GB/s
(and the memory systems are up in the 40GB/s peak territory while caches
remain rather small (several GB at most).)
Filesystem cache turnover happens in times on the order of seconds, not milliseconds.
On Mon, 19 May 2025 1:13:01 +0000, Lawrence D'Oliveiro wrote:
Filesystem cache turnover happens in times on the order of seconds, not
milliseconds.
Yes, running at the latency and bandwidth of the disk/SSD subsystem not
of the DRAM system.
Parallel programming isn't "hard" or "easy".
On 5/19/2025 1:46 PM, Vir Campestris wrote:
On 19/05/2025 02:41, Stephen Fuld wrote:
It didn't make sense to, after executing the full track read, since
the disk was positioned at the end of the track, and you had a good
indication that it was a sequential file read, to start caching the
next track in anticipation of the next full track read?
It might if the disk was idle. But if there is a queue from another
process it probably will be better to do that instead.
I presume you know that the 3880 controller did not do what today we
call command queuing, so I think you were referring to a potential queue
in the host. That being the case, the controller doesn't know if there
is a queue or not. So given that, why not start reading record 1 on the next track. If a request comes in, you can abandon the read to service
the request - no harm, no foul. If there isn't, and you subsequently
get a request for that track, it's a big win. The only potential loss
is if you get a request for the track that was LRU and got pushed out of
the cache.
On Mon, 19 May 2025 21:33:11 +0000, quadibloc wrote:
Parallel programming isn't "hard" or "easy".
Concurrent processes/threads, running either fully concurrently on
separate processors, or with asynchronous preemption on shared processors, >are hard to program.
You need RAM to make this work, and a few MB of HDD side cache isn't a huge cost if one would have already needed the same stuff to make the HDD work.
On 5/20/2025 9:49 AM, Stefan Monnier wrote:
You need RAM to make this work, and a few MB of HDD side cache isn't
a huge
cost if one would have already needed the same stuff to make the HDD
work.
Indeed, AFAIK, what we call "HDD cache" is actually just the RAM used
by the embedded CPU inside the drive for its operation.
I expect this is used to store the information about in-flight requests
(e.g. most importantly the data about the write requests received but
that haven't yet reached the platters), but I also expects it holds data
that happened to fly recently by the read-head, just in case.
Probably.
As I understand it, it is this, along with a certain amount of "read prefetch", which is granted, typically the data for the rest of the
track as the drive spins around;
And, keeping some copies of previously read content around, which can be
read again from this cache if they happen to be requested.
As I understand it, also more modern HDDs tend to be "density per area" rather than angular slices (as it was on much older HDDs and floppies,
*), so there would be more sectors on outer tracks vs on inner tracks.
Personally, I rarely use multi-threading, and when I do, it is usually in
the form of using mutex locks over shared buffers.
You lock the mutex if needed to copy data from one thread to another; or
when doing a task that depends on the data being consistent.
Personally, I rarely use multi-threading, and when I do, it is usually
in
the form of using mutex locks over shared buffers.
You lock the mutex if needed to copy data from one thread to another; or
when doing a task that depends on the data being consistent.
FWIW, I think these kinds of things usually fall in the scope of
concurrency rather than parallelism.
Stefan
When I run 20-copies of a FEM CFD application, each uni-process::
am I running concurrently ?? or in parallel ?? or both ??
Indeed, AFAIK, what we call "HDD cache" is actually just the RAM used by
the embedded CPU inside the drive for its operation.
When I run 20-copies of a FEM CFD application, each uni-process::
am I running concurrently ?? or in parallel ?? or both ??
Also the SATA interface is technically faster and has a higher bandwidth
than it can actually read stuff from the HDD itself, so if the HDD's
cache can do *anything* (in terms of prefetch and allowing IO requests
to be completed more quickly) it is still a win.
Well, say, because the FS designers seem to casually assume smaller
numbers of multi-MB files, rather than people filling the drive with
millions of files most of which being kB sized.
The caches serve different purposes.
Giving make the option to run multiple processes improved the build
speed a lot.
FWIW, I think these kinds of things usually fall in the scope of
concurrency rather than parallelism.
On Tue, 20 May 2025 20:06:13 +0000, Stefan Monnier wrote:
Personally, I rarely use multi-threading, and when I do, it is usually
in
the form of using mutex locks over shared buffers.
You lock the mutex if needed to copy data from one thread to another; or >>> when doing a task that depends on the data being consistent.
FWIW, I think these kinds of things usually fall in the scope of
concurrency rather than parallelism.
When I run 20-copies of a FEM CFD application, each uni-process::
am I running concurrently ?? or in parallel ?? or both ??
Also note: I need to use the "affinity" service under taskMangler
so that only 4 processes share a core, performance goes up by roughly
20%.
The CFD simulations typically run for 10 hours (each, wall clock time),
and every second; dump out a dozen KB of compressed data (each). The
disk load is low enough I can do other "stuff" on the computer as
long as I affinitize the heavy loads to 5 of the 6 cores.
Stefan
I presume you know that the 3880 controller did not do what today we
call command queuing, so I think you were referring to a potential
queue in the host. That being the case, the controller doesn't know
if there is a queue or not. So given that, why not start reading
record 1 on the next track. If a request comes in, you can abandon
the read to service the request - no harm, no foul. If there isn't,
and you subsequently get a request for that track, it's a big win.
The only potential loss is if you get a request for the track that was
LRU and got pushed out of the cache.
Processes on the same core are concurrent - processes on different
cores are parallel.
Only if the cores and/or "hardware threads" do not interfere with one another?
On 5/20/2025 7:29 PM, Lawrence D'Oliveiro wrote:
If it were just I/O buffers for operations in progress, that would be
fine. The problem is when it keeps data around instead of immediately
writing it out, and what’s worse, lies about it, so it tells the OS
that the write has completed when it hasn’t.
Note that (with SATA and similar) the OS can request that the drive
flush its caches, and (in theory) drive should not respond to more
requests until everything has been fully written back to disk.
Stephen Fuld <[email protected]d> writes:
I presume you know that the 3880 controller did not do what today we
call command queuing, so I think you were referring to a potential
queue in the host. That being the case, the controller doesn't know
if there is a queue or not. So given that, why not start reading
record 1 on the next track. If a request comes in, you can abandon
the read to service the request - no harm, no foul. If there isn't,
and you subsequently get a request for that track, it's a big win.
The only potential loss is if you get a request for the track that was
LRU and got pushed out of the cache.
over optimizing full track read ahead could lock out other tasks that
had competing requirements for other parts of the disk.
On Tue, 20 May 2025 10:49:54 -0400, Stefan Monnier wrote:
Indeed, AFAIK, what we call "HDD cache" is actually just the RAM used by
the embedded CPU inside the drive for its operation.
If it were just I/O buffers for operations in progress, that would be
fine. The problem is when it keeps data around instead of immediately
writing it out, and what’s worse, lies about it, so it tells the OS that the write has completed when it hasn’t.
On Tue, 20 May 2025 20:08:28 -0500, BGB wrote:
On 5/20/2025 7:29 PM, Lawrence D'Oliveiro wrote:
If it were just I/O buffers for operations in progress, that would be
fine. The problem is when it keeps data around instead of immediately
writing it out, and what’s worse, lies about it, so it tells the OS
that the write has completed when it hasn’t.
Note that (with SATA and similar) the OS can request that the drive
flush its caches, and (in theory) drive should not respond to more
requests until everything has been fully written back to disk.
I mentioned elsewhere that a special function was added that was supposed
to mean “really flush your caches dammit”.
But there is still no way to tell that the drive really does what you
demand that it do, and isn’t still lying about it ...
And furthermore, you saw earlier that the host can turn on or off,
either for all writes or a particular write, the completion return
before the data is on the disk.
On Tue, 20 May 2025 20:54:08 -0700, Stephen Fuld wrote:
And furthermore, you saw earlier that the host can turn on or off,
either for all writes or a particular write, the completion return
before the data is on the disk.
The host can *request* such a thing, nothing more. And of course the drive can answer in the affirmative.
Whether a suitable action actually takes place is another matter entirely.
On Tue, 20 May 2025 20:06:13 +0000, Stefan Monnier wrote:
FWIW, I think these kinds of things usually fall in the scope of
concurrency rather than parallelism.
When I run 20-copies of a FEM CFD application, each uni-process::
am I running concurrently ?? or in parallel ?? or both ??
Personally, I rarely use multi-threading, and when I do, it is usually
in
the form of using mutex locks over shared buffers.
You lock the mutex if needed to copy data from one thread to another; or >>> when doing a task that depends on the data being consistent.
FWIW, I think these kinds of things usually fall in the scope of
concurrency rather than parallelism.
When I run 20-copies of a FEM CFD application, each uni-process::
am I running concurrently ?? or in parallel ?? or both ??
Personally, I rarely use multi-threading, and when I do, it is usually >>>> in
the form of using mutex locks over shared buffers.
You lock the mutex if needed to copy data from one thread to another; or >>>> when doing a task that depends on the data being consistent.
FWIW, I think these kinds of things usually fall in the scope of
concurrency rather than parallelism.
When I run 20-copies of a FEM CFD application, each uni-process::
am I running concurrently ?? or in parallel ?? or both ??
Both: AFAIK the choice of how to divide&spread the data and the work is
in the parallelism camp, while the choice of how to synchronize them is
in the concurrency camp.
Stefan
On Tue, 20 May 2025 20:54:08 -0700, Stephen Fuld wrote:
And furthermore, you saw earlier that the host can turn on or off,
either for all writes or a particular write, the completion return
before the data is on the disk.
The host can *request* such a thing, nothing more. And of course the drive >can answer in the affirmative.
Whether a suitable action actually takes place is another matter entirely.
On Tue, 20 May 2025 18:39:54 -0700, Chris M. Thomasson wrote:
Processes on the same core are concurrent - processes on different
cores are parallel.
Only if the cores and/or "hardware threads" do not interfere with one
another?
That’s why I think the distinction is meaningless.
Yes, there is a valid distinction to be made between truly concurrent/ >parallel processes/threads, and ones which are made to appear so by >preemptive scheduling on shared hardware.
Not sure if there is a good term for the latter: “timesliced”? >“timeshared”? “pseudoconcurrent”? “pseudoparallel”?
On 5/20/2025 8:46 PM, Lawrence D'Oliveiro wrote:
On Tue, 20 May 2025 20:08:28 -0500, BGB wrote:
On 5/20/2025 7:29 PM, Lawrence D'Oliveiro wrote:
If it were just I/O buffers for operations in progress, that would be
fine. The problem is when it keeps data around instead of immediately
writing it out, and what’s worse, lies about it, so it tells the OS
that the write has completed when it hasn’t.
Note that (with SATA and similar) the OS can request that the drive
flush its caches, and (in theory) drive should not respond to more
requests until everything has been fully written back to disk.
I mentioned elsewhere that a special function was added that was supposed
to mean “really flush your caches dammit”.
But there is still no way to tell that the drive really does what you
demand that it do, and isn’t still lying about it ...
Sure there is. Just do a small write to a random location and time it. repeat several times to assure consistent results.
Besides, if a disk vendor was foolish enough to not follow the spec and
not document that fact, customers would soon find out and that would
ruin that vendor's reputation. They wouldn't risk it.
Stephen Fuld wrote:
On 5/20/2025 8:46 PM, Lawrence D'Oliveiro wrote:
On Tue, 20 May 2025 20:08:28 -0500, BGB wrote:
On 5/20/2025 7:29 PM, Lawrence D'Oliveiro wrote:
If it were just I/O buffers for operations in progress, that would be >>>>> fine. The problem is when it keeps data around instead of immediately >>>>> writing it out, and what’s worse, lies about it, so it tells the OS >>>>> that the write has completed when it hasn’t.
Note that (with SATA and similar) the OS can request that the drive
flush its caches, and (in theory) drive should not respond to more
requests until everything has been fully written back to disk.
I mentioned elsewhere that a special function was added that was
supposed
to mean “really flush your caches dammit”.
But there is still no way to tell that the drive really does what you
demand that it do, and isn’t still lying about it ...
Sure there is. Just do a small write to a random location and time
it. repeat several times to assure consistent results.
Besides, if a disk vendor was foolish enough to not follow the spec
and not document that fact, customers would soon find out and that
would ruin that vendor's reputation. They wouldn't risk it.
I suspect some of Lawrence's concerns go back to the Win3.1 days,
when HDD could now afford to expand the read/write buffer and improve
their performance stats. Because the Parallel ATA interface was synchronous one way drives "improved" their performance stats was to lie and send back
a fake ACK to writes until they had enough sectors to make up a whole
track.
Yes it risked scrambling the file system if one powered off too quick.
But I gather DOS/Win3.1 FAT file system would do that enough on its own
that the drive wouldn't be blamed.
The PATA spec includes commands to enable/disable read look-ahead feature, and enable/disable write cache, but doesn't say what they actually do.
Also there are various commands for setting idle mode, standby mode,
power down mode, etc. but none say what happens to the cache.
I suspect that the HDD manufacturers looked at what drive commands Win3.1 issued when you exited and used those to trigger write back of any
pending cached data.
When I installed WinNT 3.1 (beta) in 1992 it came with explicit
instructions that HDD must use write-through caching, which was enabled/disabled by a jumper pin on the drive.
Only if the cores and/or "hardware threads" do not interfere with one another? Fwiw, an example of an embarrassingly parallel algorithm is computing the Mandelbrot set. Actually, this reminds me of the "alias" problem with Intel hyper threading in the past.
I suspect some of Lawrence's concerns go back to the Win3.1 days,
when HDD could now afford to expand the read/write buffer and improve
their performance stats. Because the Parallel ATA interface was synchronous >one way drives "improved" their performance stats was to lie and send back
a fake ACK to writes until they had enough sectors to make up a whole track.
Yes it risked scrambling the file system if one powered off too quick.
But I gather DOS/Win3.1 FAT file system would do that enough on its own
that the drive wouldn't be blamed.
When I installed WinNT 3.1 (beta) in 1992 it came with explicit
instructions that HDD must use write-through caching, which was >enabled/disabled by a jumper pin on the drive.
EricP <[email protected]> writes:
I suspect some of Lawrence's concerns go back to the Win3.1 days,
when HDD could now afford to expand the read/write buffer and improve
their performance stats. Because the Parallel ATA interface was synchronous >> one way drives "improved" their performance stats was to lie and send back >> a fake ACK to writes until they had enough sectors to make up a whole track.
AFAIK there was no tagged command queuing in the (P)ATA interface
until pretty late in the game (and then it was not used by Linux
AFAIK). So if the HDD had waited until the sector was on the platter
(and IIRC there was a way to switch the drive to such a synchronous
mode), anything that writes to the HDD would have been extremely slow:
Wait until the sector is found, write the sector, report success, get
the next command that writes the next sector, but now the head is
already past the sector and you have to wait for another rotation.
By contrast, if you return success as soon as the sector is in the
HDD's RAM, the OS can continue sending data, and the HDD can write the
data in any order that it deems appropriate; it does not have to wait
for a complete track or something, it can start writing right away,
and, of course, if there is a sequence of sectors in the cache, write
that sequence in one go.
I have written a test that checks how much HDDs can reorder writes,
and my results are that, for the pattern I tested, there is no bound on
the out-of-orderness that the drives exhibit if you don't ask for a
barrier or sync. You can find the test software and the results of my testing at
<http://www.complang.tuwien.ac.at/anton/hdtest/>
Yes it risked scrambling the file system if one powered off too quick.
But I gather DOS/Win3.1 FAT file system would do that enough on its own
that the drive wouldn't be blamed.
Actually on MS-DOS the usual way to shut down the system was to turn
the computer off, or in case of floppy disks to just take them out of
the drive after the light has gone out, and the amount of breakage on
the file system level was not big. I expect that the file system
synced pretty obsessively to achieve that.
When I installed WinNT 3.1 (beta) in 1992 it came with explicit
instructions that HDD must use write-through caching, which was
enabled/disabled by a jumper pin on the drive.
SCSI or PATA?
- anton
That’s why I think the distinction is meaningless.Processes on the same core are concurrent - processes on differentOnly if the cores and/or "hardware threads" do not interfere with one
cores are parallel.
another?
I get roped into helping with a 370 16-CPU multiprocessor design. It was going fine until somebody tells head of POK (high end 370 processors)
that it could be decades before POK's favorite son operating system (now "MVS") had ("effective") 16-cpu support (POK doesn't ship a 16-CPU
system until after the turn of the century) ... and some of us are
invited to never visit POK again.
In the case of concurrency the core question is: Given a set of somewhat independent tasks working on some chunks of data, make sure the computed result is correct, e.g. design tools like mutexes, memory barriers, transactional memory, static analysis, reasoning principles, etc...
whose core focus is on making sure there's no race conditions, dead
locks, ...
In the case of parallelism, the core question instead is: given a program/algorithm, restructure (or even completely replace) it so as to divide it into somewhat independent tasks that can take advantage of
multiple CPUs to finish the work faster.
Anton Ertl wrote:
EricP <[email protected]> writes:
When I installed WinNT 3.1 (beta) in 1992 it came with explicit
instructions that HDD must use write-through caching, which was
enabled/disabled by a jumper pin on the drive.
SCSI or PATA?
- anton
IIRC it needed a SCSI board for the CD-ROM drive (I don't remember why,
I just remember being miffed that I had to buy one) but could use IDE HDD.
As the SCSI HDD were the same drives but with a SCSI interface were
$100 more expensive, I would have used the IDE.
WinNT was developed on MIPS R4000 and ported to 80386/486 later.
Those MIPS systems used all SCSI so maybe they hadn't had time to port
the CD-ROM drivers to IDE but had ported the HDD drivers to IDE
before the beta release.
... when running back-to-back on the same core they are concurrent.
That leaves the question open what HPC applications are. It's somewhat cyclic: Those that can make good use of parallel (super-)computers.
I also remember from some early 90s PCs some CD-ROM drives that used
some sort of non-standard interface. Typically, they would plug into an
ISA card with a cable that was (IIRC) somewhat narrower than a normal
IDE cable (I remember it being around the width of a floppy cable; two
ends with a direct connection, and no twists, unlike a typical floppy
cable which usually has part of the cable twisted).
Not entirely sure what it was. Wasn't an IDE/ATAPI CD-ROM though, do
know this much at least...
Digging more, there was apparently a proprietary 34-pin Sony connector,
which does at least resemble what I remember (and some of the drives do
look like the drives I remember seeing).
*: Had once encountered a computer (that at the time was being
discarded, but I got it and had it for a while). It had SCSI drives, and
also the weirdness that rather than the CPU and RAM being on the
motherboard, it was on a riser card (the MOBO was IIRC effectively just
card slots, IIRC they resembled 16-bit ISA slots buts with a long
extended part on the front; similar to the "VESA Local Bus" IIRC).
IIRC, no connectors on the MOBO (apart from power IIRC), all of the
external connections (like VGA and mouse/keyboard) being on riser cards.
I remember the CPU riser card apparently having dual 486DX's, but from
what information I can gather, multi-socket systems weren't a thing with
the 486, so don't know what was going on there. Beyond this, was SIMM
RAM, and a lot of other stuff you would normally see on the MOBO
(excluding connectors).
It was a weird machine, I had managed to get Win NT4 installed on it,
but I remember I couldn't do much else with it at the time. Also for
some unknown reason, while normal 16-bit ISA cards would plug into the
slots, they did not seem to be recognized by the OS.
WinNT was developed on MIPS R4000 and ported to 80386/486 later.
On Wed, 21 May 2025 14:08:37 +0000, MitchAlsup1 wrote:
... when running back-to-back on the same core they are concurrent.
Consecutively, surely (on a time-slice basis).
When a judge sentences you to serve different jail terms concurrently,
they are not added together.
When two processes run on the same CPU, their execution times are added together.
On Thu, 22 May 2025 2:49:53 +0000, Lawrence D'Oliveiro wrote:
On Wed, 21 May 2025 14:08:37 +0000, MitchAlsup1 wrote:
... when running back-to-back on the same core they are concurrent.
Consecutively, surely (on a time-slice basis).
When a judge sentences you to serve different jail terms concurrently,
they are not added together.
When two processes run on the same CPU, their execution times are added
together.
Wall clock time adds.
process times do not.
But, yeah, this was apparently something that my mom grabbed from a
dumpster sometime around 20-25 years ago, and its fate was that parents
later returned it to a dumpster.
When two processes run on the same CPU, their execution times are added
together.
Wall clock time adds. process times do not.
On 5/13/2025 1:12 AM, Lawrence D'Oliveiro wrote:
On Tue, 13 May 2025 07:40:35 GMT, Anton Ertl wrote:
In this case the drive knows things that the OS does not: Consider that
the OS asked for sector N what happens when the arm finally has settled
enough to read from the track, and the first sector it sees is sector
N+10.
The OS isn’t likely to ask for one sector at a time.
Frequently true, so consider this related scenario. The host requests a
read of 10 sectors starting at sector N. When the head settles, the
next sector is N+6. Without any in drive buffering, it would wait
almost a full revolution till record N comes under the head.
With buffering, but no cache, the drive reads record N+5 to N+9 into the buffer, then waits until the drive rotates to record N and begins the
host transfer. This is an improvement because the transfer to the host
is faster than the transfer from the disk, and the last 3 sectors can be transferred out of the buffer without waiting for the disk, so the
transfer is completed faster.
Now consider with caching. Similar, but after record N+9, the drive continues reading into the cache. Lets say there are 30 records on this track. If it reads all of the data into the cache, then proceeds as
above once the disk rotates to record N, it has cost zero time, and if
the host then issues another 10 sector read sequential to the initial
one (or actually any sectors from N+10 to N+29). This can be satisfied
out of the cache without any drive delay, so much faster than without
the cache, and the heads can be moved away to start satisfying another unrelated request. There is minimal cost and substantial benefit.
Now you have argued that the file system cache should take care of that, presumably issuing prefetch reads for the next sectors. This will work,
of course, but has some disadvantages relative to using the drive cache.
Specifically,since it is unlikely the prefetch request will be
received by the drive before record N+10 has passed the heads, it will
incur additional most of a rotational delay, which will tie up the
drive, preventing it from responding to some other request.
No one is arguing that host based file caches are bad. It is simply the
fact that there are situations where drive caches are a useful addition,
and since the drive has to have some DRAM anyway for other reasons, the
cost is minimal. You can think of the drive cache as the "next level"
cache behind the host based cache.
[email protected] (MitchAlsup1) writes:
On Thu, 22 May 2025 2:49:53 +0000, Lawrence D'Oliveiro wrote:
On Wed, 21 May 2025 14:08:37 +0000, MitchAlsup1 wrote:
... when running back-to-back on the same core they are concurrent.
Consecutively, surely (on a time-slice basis).
When a judge sentences you to serve different jail terms concurrently,
they are not added together.
When two processes run on the same CPU, their execution times are added
together.
Wall clock time adds.
process times do not.
Angels.
Pins.
Given that the two processes are effectively interleaved, the two
problems
solved by the individual processes are solved concurrently.
If one ran to completion before the other started, then they'd be
considered
consecutive.
Hyperthreading allows instruction level concurrency between independent processes assigned to the same core.
On Wed, 21 May 2025 14:08:37 +0000, MitchAlsup1 wrote:
... when running back-to-back on the same core they are concurrent.
Consecutively, surely (on a time-slice basis).
When a judge sentences you to serve different jail terms concurrently,
they are not added together.
When two processes run on the same CPU, their execution times are added >together.
It is pretty clear that due to drive mechanics track cache/buffer is
useful.
However, the real question is about size: how big should it be.
On 5/23/2025 12:35 AM, Lawrence D'Oliveiro wrote:
On Fri, 23 May 2025 00:18:53 -0000 (UTC), Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/buffer is
useful.
Only if you don’t take the statistics of real-world cache behaviour into >> account.
However, the real question is about size: how big should it be.
It can never be big enough to make a difference.
Sometimes, it is not the size of the cache that matters.
Say, for example, a typical cache configuration in my core:
L1 D$: 32K, direct mapped
L1 I$: 16K, direct mapped
L2: 256K, direct mapped.
OK, so say I stick a 4K cache between the L1 and L2 caches.
Seems kinda useless just based on sizes.
Except: This small cache is 4-way set associative and so can absorb a
bunch of conflict misses, and notably reducing the number of cache
misses in the L2 cache. It can bring a performance benefit, despite
being small.
Why?
On 5/22/2025 5:41 PM, Lawrence D'Oliveiro wrote:
On Thu, 22 May 2025 12:39:00 -0500, BGB wrote:
But, yeah, this was apparently something that my mom grabbed from a
dumpster sometime around 20-25 years ago, and its fate was that parents
later returned it to a dumpster.
Don’t you have regulations, or at least discouragements, against e-waste >> going to landfill?
This is 'Murica, pretty much everything goes in the trash here...
Old computers, old electronics, batteries, ...
Paper, plastic, used motor oil, ...
It all goes in the trash...
BGB <[email protected]> writes:
On 5/22/2025 5:41 PM, Lawrence D'Oliveiro wrote:
On Thu, 22 May 2025 12:39:00 -0500, BGB wrote:
But, yeah, this was apparently something that my mom grabbed from a
dumpster sometime around 20-25 years ago, and its fate was that parents >>>> later returned it to a dumpster.
Don’t you have regulations, or at least discouragements, against e-waste >>> going to landfill?
Yes, in the last two decades. A quarter century ago, not so much.
This is 'Murica, pretty much everything goes in the trash here...
Nonsense.
Old computers, old electronics, batteries, ...
Paper, plastic, used motor oil, ...
It all goes in the trash...
Nonsense.
Stephen Fuld <[email protected]d> wrote:
On 5/13/2025 1:12 AM, Lawrence D'Oliveiro wrote:
On Tue, 13 May 2025 07:40:35 GMT, Anton Ertl wrote:
In this case the drive knows things that the OS does not: Consider that >>>> the OS asked for sector N what happens when the arm finally has settled >>>> enough to read from the track, and the first sector it sees is sector
N+10.
The OS isn’t likely to ask for one sector at a time.
Frequently true, so consider this related scenario. The host requests a
read of 10 sectors starting at sector N. When the head settles, the
next sector is N+6. Without any in drive buffering, it would wait
almost a full revolution till record N comes under the head.
With buffering, but no cache, the drive reads record N+5 to N+9 into the
buffer, then waits until the drive rotates to record N and begins the
host transfer. This is an improvement because the transfer to the host
is faster than the transfer from the disk, and the last 3 sectors can be
transferred out of the buffer without waiting for the disk, so the
transfer is completed faster.
Now consider with caching. Similar, but after record N+9, the drive
continues reading into the cache. Lets say there are 30 records on this
track. If it reads all of the data into the cache, then proceeds as
above once the disk rotates to record N, it has cost zero time, and if
the host then issues another 10 sector read sequential to the initial
one (or actually any sectors from N+10 to N+29). This can be satisfied
out of the cache without any drive delay, so much faster than without
the cache, and the heads can be moved away to start satisfying another
unrelated request. There is minimal cost and substantial benefit.
Now you have argued that the file system cache should take care of that,
presumably issuing prefetch reads for the next sectors. This will work,
of course, but has some disadvantages relative to using the drive cache.
Specifically,since it is unlikely the prefetch request will be
received by the drive before record N+10 has passed the heads, it will
incur additional most of a rotational delay, which will tie up the
drive, preventing it from responding to some other request.
No one is arguing that host based file caches are bad. It is simply the
fact that there are situations where drive caches are a useful addition,
and since the drive has to have some DRAM anyway for other reasons, the
cost is minimal. You can think of the drive cache as the "next level"
cache behind the host based cache.
It is pretty clear that due to drive mechanics track cache/buffer
is useful.
However, the real question is about size: how big
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
To elaborate. Batteries are collected by retailers (drop 'em off at Ace Hardware).
The solid waste company that picks up the trash provides 5gal jugs
for used motor oil and bags for used filters (which can also be dropped
off at any auto parts store). Paper, plastic, metals (other than styrofoam) are collected and recycled. Old electronics are recycled. Office supply stores accept used printer ink cartridges. Best buy accepts electronics
for recycling as do various recyclers that offer drop-off locations.
On 5/22/2025 5:18 PM, Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/bufferPretty clear to everyone except one person. :-)
is useful.
However, the real question is about size: how bigIt's not just the rotations, but the seek time. So your example is fewer "operations" than the 80-150 you get when just including rotations.
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
And if you are caching writes, more cache gives you more blocks to choose from when optimizing the write back order, which reduces the time to write them all back.
The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.
And if you are caching writes, more cache gives you more blocks to
choose from when optimizing the write back order ...
On Fri, 23 May 2025 08:28:44 -0700, Stephen Fuld wrote:
And if you are caching writes, more cache gives you more blocks to
choose from when optimizing the write back order ...
Fun fact: It seems to be part of the spec that write cache reordering is disabled if the driver is using NCQ.
Assuming drives pay attention to that part of the spec, of course.
See the description of the “wcreorder” setting here <https://manpages.debian.org/smartctl(8)>.
On 5/23/2025 7:36 AM, MitchAlsup1 wrote:
I had usually understood it that a "victim buffer" was typically glued directly to the L1 cache.
On Fri, 23 May 2025 6:09:37 +0000, BGB wrote:
OK, so say I stick a 4K cache between the L1 and L2 caches.
Seems kinda useless just based on sizes.
It is called a "victim buffer"
Stephen Fuld [2025-05-23 08:28:44] wrote:
On 5/22/2025 5:18 PM, Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/bufferPretty clear to everyone except one person. :-)
is useful.
🙂
However, the real question is about size: how bigIt's not just the rotations, but the seek time. So your example is fewer
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
"operations" than the 80-150 you get when just including rotations.
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.
And if you are caching writes, more cache gives you more blocks to choose
from when optimizing the write back order, which reduces the time to write >> them all back.
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so unless the drive is allowed to do write-back caching it seems the amount
of space used for write-buffering is likely small (compared to 256MB).
[ Unless it is common for individual write commands to cover multi-MB
chunks of data? ]
The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.
In some markets (e.g. home routers), the size of DRAM seems to be enough
of a cost factor that it took many years until reaching 256MBs, even
though those boxes *need* that RAM for all kinds of purposes (the 128MB
of my current home-router seems to be its main source of instability).
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?
On 5/23/2025 2:03 PM, Stefan Monnier wrote:
Stephen Fuld [2025-05-23 08:28:44] wrote:
On 5/22/2025 5:18 PM, Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/bufferPretty clear to everyone except one person. :-)
is useful.
🙂
However, the real question is about size: how bigIt's not just the rotations, but the seek time. So your example is fewer >>> "operations" than the 80-150 you get when just including rotations.
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.
Yes, I didn't express myself well. :-( And once again, I have to say
that my information may be obsolete.
I think it is useful to separate talking about read data from write
data. For read data, as with any cache, more is always better than
less, though with diminishing returns. Why pick 1.25 sec as the "cut
off point"? If the host re-references data that it hasn't read for say
3 seconds, having it in cache still saves, probably a seek time and on average 1/2 rotation time. Plus, it means the heads will be free to
handle other requests. All of this is standard cache benefits. I see
no reason to limit the cache size and reduce this benefit.
And if you are caching writes, more cache gives you more blocks to choose >>> from when optimizing the write back order, which reduces the time to write >>> them all back.
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so
unless the drive is allowed to do write-back caching it seems the amount
of space used for write-buffering is likely small (compared to 256MB).
[ Unless it is common for individual write commands to cover multi-MB
chunks of data? ]
For write data, I was unaware of the 32 operation limit. I was used to
SCSI, which, IIRC was larger, and for server type applications, where
some sort of UPS is more common, the site may choose to enable write
caching in the disk. For a disk vendor, given the small cost of the
DRAM, it is an easy choice.
The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.
In some markets (e.g. home routers), the size of DRAM seems to be enough
of a cost factor that it took many years until reaching 256MBs, even
though those boxes *need* that RAM for all kinds of purposes (the 128MB
of my current home-router seems to be its main source of instability).
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?
I can't comment on routers, but for disks, while the cost of the disk
may not have come down, increasing capacity allows reduced cost per
gigabyte. A substantial portion of the cost is not subject to Moore's
law (e.g. drive motor, magnets and arm assembly, etc.) and some capacity increasing technologies cost more (but not enough more to overwhelm the capacity advantage).
Stephen Fuld <[email protected]d> wrote:
On 5/23/2025 2:03 PM, Stefan Monnier wrote:
Stephen Fuld [2025-05-23 08:28:44] wrote:
On 5/22/2025 5:18 PM, Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/bufferPretty clear to everyone except one person. :-)
is useful.
🙂
However, the real question is about size: how bigIt's not just the rotations, but the seek time. So your example is fewer >>>> "operations" than the 80-150 you get when just including rotations.
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.
Yes, I didn't express myself well. :-( And once again, I have to say
that my information may be obsolete.
I think it is useful to separate talking about read data from write
data. For read data, as with any cache, more is always better than
less, though with diminishing returns. Why pick 1.25 sec as the "cut
off point"? If the host re-references data that it hasn't read for say
3 seconds, having it in cache still saves, probably a seek time and on
average 1/2 rotation time. Plus, it means the heads will be free to
handle other requests. All of this is standard cache benefits. I see
no reason to limit the cache size and reduce this benefit.
We are talking here about common case, that is when disc is accessed
via OS cache. OS cache is significantly larger than disc cache, so
hit ratio for data sent to host is going to be quite low. Disc
cache has an advantage: it gets "for free" some data that host did
not request. But it is rather unlikely that keeping such data
for long time has significant advantage.
And if you are caching writes, more cache gives you more blocks to choose >>>> from when optimizing the write back order, which reduces the time to write >>>> them all back.
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so >>> unless the drive is allowed to do write-back caching it seems the amount >>> of space used for write-buffering is likely small (compared to 256MB).
[ Unless it is common for individual write commands to cover multi-MB
chunks of data? ]
For write data, I was unaware of the 32 operation limit. I was used to
SCSI, which, IIRC was larger, and for server type applications, where
some sort of UPS is more common, the site may choose to enable write
caching in the disk. For a disk vendor, given the small cost of the
DRAM, it is an easy choice.
I do not look at details of disc protocol. But with protocal done
right host would first transfer commands and then deliver data
in order requested by the drive. So most buffering would be in
the host and disc would need just enough buffering to ensure
smooth transmission and low interrupt rate. 4 track looks like
plenty for this purpose.
The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.
In some markets (e.g. home routers), the size of DRAM seems to be enough >>> of a cost factor that it took many years until reaching 256MBs, even
though those boxes *need* that RAM for all kinds of purposes (the 128MB
of my current home-router seems to be its main source of instability).
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?
I can't comment on routers, but for disks, while the cost of the disk
may not have come down, increasing capacity allows reduced cost per
gigabyte. A substantial portion of the cost is not subject to Moore's
law (e.g. drive motor, magnets and arm assembly, etc.) and some capacity
increasing technologies cost more (but not enough more to overwhelm the
capacity advantage).
In nineties I read that for motherboard manufactures 1 cent was
"negligible", but 10 cents was significant: In volume transactions
margins were low and no party were willing to absorb 10 cents
per piece "loss". Discs probably are less competitive than
motherboards, but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
So IMO it is highly unclear why manufacturers use large caches.
One possible explanation could be benchmarketing and using
obsolete benchmarks.
Another could be inertia with customers
thinking that "larger cache is better".
Another things is fragmenting market into different "kinds" of
drives. Rationally, high performance drives should get
better mechanical parts. But in given performance area there
seem to be no reason for different mechanics, so I suspect
that they use the same. They may get different firmware.
"Green" consumer parts seem to be quite aggressive powering
down (IIUC on recent WD parts it is impossible to permanently
disable this), but beyond this it is not clear to me if there
are rational reasons for significantly different firmware.
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so unless the drive is allowed to do write-back caching it seems the amount
of space used for write-buffering is likely small (compared to 256MB).
[ Unless it is common for individual write commands to cover multi-MB
chunks of data? ]
...
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?
No, when the disk receives a write command, it accepts the write data immediately (up to some large limit). That way, when the heads settle on
the track, if the disk happens to be positioned in the middle of the transfer, it can write the last part of the data to the disk immediately, then wait for the disk to spin to where the transfer starts to finish the transferring the first part of the write data. This reduces average
latency, i.e. improves performance.
No, when the disk receives a write command, it accepts the write data
immediately (up to some large limit). That way, when the heads settle on
the track, if the disk happens to be positioned in the middle of the
transfer, it can write the last part of the data to the disk immediately,
then wait for the disk to spin to where the transfer starts to finish the
transferring the first part of the write data. This reduces average
latency, i.e. improves performance.
Really? I had the impression that it would be very hard to start writing from the middle of a sector because of the need to be sure exactly where
in the sector we are. IOW, the drive needs to see the inter-sector markers before it can start writing to a sector.
On 5/25/2025 4:16 PM, Stefan Monnier wrote:
No, when the disk receives a write command, it accepts the write data
immediately (up to some large limit). That way, when the heads settle
on
the track, if the disk happens to be positioned in the middle of the
transfer, it can write the last part of the data to the disk
immediately,
then wait for the disk to spin to where the transfer starts to finish
the
transferring the first part of the write data. This reduces average
latency, i.e. improves performance.
Really? I had the impression that it would be very hard to start
writing
from the middle of a sector because of the need to be sure exactly where
in the sector we are. IOW, the drive needs to see the inter-sector
markers
before it can start writing to a sector.
I am not sure what you mean by sector. If you mean a disk block,
typically used to be 512 bytes, now typically 4K bytes, then you are
right that you can't start writing in the middle of one. But a track typically has many blocks and you can start writing at any one of them.
So if a write is say 16K bytes and you have 4K blocks on the disk,
then the drive could start at say the second 4K block, complete the last
12K of the transfer, wait for the rotation then finish the first 4K
block.
I am not sure what you mean by sector. If you mean a disk block, typically used to be 512 bytes, now typically 4K bytes, then you are right that you can't start writing in the middle of one.
But a track typically has many blocks and you can start writing at any
one of them. So if a write is say 16K bytes and you have 4K blocks
on the disk, then the drive could start at say the second 4K block,
complete the last 12K of the transfer, wait for the rotation then
finish the first 4K block.
As I see it, with variable track length geometries, the OS file
system cannot make reasonable assumptions about track boundaries, so it cannot maintain track/cylinder caches, but the drive processor can.
...
but HDDs are pretty damn expensive beasts nowadays (because prices have
not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?
Actually, HDDs are still on a steep downward price curve.
No, when the disk receives a write command, it accepts the write data immediately (up to some large limit). That way, when the heads settle
on the track, if the disk happens to be positioned in the middle of the transfer, it can write the last part of the data to the disk
immediately, then wait for the disk to spin to where the transfer starts
to finish the transferring the first part of the write data. This
reduces average latency, i.e. improves performance.
Discs probably are less competitive than
motherboards,
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
Actually, HDDs are still on a steep downward price curve. The amount of
HDD space you get for USD 100 keeps going up, up, up.
Short term variations of growth are hard deal with in any sort of |>projection, but if you look at the time between 1TB and 10TB disks,|
January 2007 through December 2015, or 106 months, you get a doubling
once every 31.9 months (that wasn't actually the source for my 30 year |>estimate, but it happens to work out about the same way).
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at least in
the past. The fact that there are only 2-3 surviving HDD
manufacturers indicates intense competition in the past, possible less
today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold
512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB DDR3
DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3
DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is unlikely
to save even one cent.
- anton
On Sun, 25 May 2025 12:13:08 -0700, Stephen Fuld wrote:
No, when the disk receives a write command, it accepts the write data
immediately (up to some large limit). That way, when the heads settle
on the track, if the disk happens to be positioned in the middle of the
transfer, it can write the last part of the data to the disk
immediately, then wait for the disk to spin to where the transfer starts
to finish the transferring the first part of the write data. This
reduces average latency, i.e. improves performance.
But it requires reordering of writes.
You are projecting computer memory prices on very different market.
The memory used for HD cache is likely an individual memory chip or two
chips and likely several generation older than devices used in computer >DIMMs. I would expect something like x16 DDR3. Looking for price of
such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less than what
we see on Mauser, there exists measurable difference between 128, 256
and 512 MB.
On 5/25/2025 11:05 AM, Waldek Hebisch wrote:
Stephen Fuld <[email protected]d> wrote:
On 5/23/2025 2:03 PM, Stefan Monnier wrote:
Stephen Fuld [2025-05-23 08:28:44] wrote:
On 5/22/2025 5:18 PM, Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/bufferPretty clear to everyone except one person. :-)
is useful.
🙂
However, the real question is about size: how bigIt's not just the rotations, but the seek time. So your example is fewer >>>>> "operations" than the 80-150 you get when just including rotations.
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
I don't understand what you're getting at, here.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't
change that fact.
Yes, I didn't express myself well. :-( And once again, I have to say
that my information may be obsolete.
I think it is useful to separate talking about read data from write
data. For read data, as with any cache, more is always better than
less, though with diminishing returns. Why pick 1.25 sec as the "cut
off point"? If the host re-references data that it hasn't read for say
3 seconds, having it in cache still saves, probably a seek time and on
average 1/2 rotation time. Plus, it means the heads will be free to
handle other requests. All of this is standard cache benefits. I see
no reason to limit the cache size and reduce this benefit.
We are talking here about common case, that is when disc is accessed
via OS cache. OS cache is significantly larger than disc cache, so
hit ratio for data sent to host is going to be quite low. Disc
cache has an advantage: it gets "for free" some data that host did
not request. But it is rather unlikely that keeping such data
for long time has significant advantage.
And if you are caching writes, more cache gives you more blocks to choose >>>>> from when optimizing the write back order, which reduces the time to write
them all back.
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so >>>> unless the drive is allowed to do write-back caching it seems the amount >>>> of space used for write-buffering is likely small (compared to 256MB). >>>> [ Unless it is common for individual write commands to cover multi-MB
chunks of data? ]
For write data, I was unaware of the 32 operation limit. I was used to
SCSI, which, IIRC was larger, and for server type applications, where
some sort of UPS is more common, the site may choose to enable write
caching in the disk. For a disk vendor, given the small cost of the
DRAM, it is an easy choice.
I do not look at details of disc protocol. But with protocal done
right host would first transfer commands and then deliver data
in order requested by the drive. So most buffering would be in
the host and disc would need just enough buffering to ensure
smooth transmission and low interrupt rate. 4 track looks like
plenty for this purpose.
No, when the disk receives a write command, it accepts the write data immediately (up to some large limit). That way, when the heads settle
on the track, if the disk happens to be positioned in the middle of the transfer, it can write the last part of the data to the disk
immediately, then wait for the disk to spin to where the transfer starts
to finish the transferring the first part of the write data. This
reduces average latency, i.e. improves performance.
The larger DRAM is a small component of drive cost, so the
manufacturers think it is worth including more.
In some markets (e.g. home routers), the size of DRAM seems to be enough >>>> of a cost factor that it took many years until reaching 256MBs, even
though those boxes *need* that RAM for all kinds of purposes (the 128MB >>>> of my current home-router seems to be its main source of instability). >>>> but HDDs are pretty damn expensive beasts nowadays (because prices have >>>> not gone down for the last 10 years or so), so I guess that makes
the relative cost of 512MB of DRAM "negligible"?
I can't comment on routers, but for disks, while the cost of the disk
may not have come down, increasing capacity allows reduced cost per
gigabyte. A substantial portion of the cost is not subject to Moore's
law (e.g. drive motor, magnets and arm assembly, etc.) and some capacity >>> increasing technologies cost more (but not enough more to overwhelm the
capacity advantage).
In nineties I read that for motherboard manufactures 1 cent was
"negligible", but 10 cents was significant: In volume transactions
margins were low and no party were willing to absorb 10 cents
per piece "loss". Discs probably are less competitive than
motherboards, but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
I can't comment on your specific numbers, but assuming you are right,
adding $1 to the cost is is small, at least in the part of the market I
was familiar with. And remember, you are not "adding" 256MB, as some of
that is needed for various internal operations.
I had the impression that it would be very hard to start writing from the middle of a sector because of the need to be sure exactly where
in the sector we are. IOW, the drive needs to see the inter-sector markers before it can start writing to a sector.
Stephen Fuld <[email protected]d> wrote:
On 5/25/2025 11:05 AM, Waldek Hebisch wrote:
Stephen Fuld <[email protected]d> wrote:
On 5/23/2025 2:03 PM, Stefan Monnier wrote:
Stephen Fuld [2025-05-23 08:28:44] wrote:
On 5/22/2025 5:18 PM, Waldek Hebisch wrote:
It is pretty clear that due to drive mechanics track cache/buffer >>>>>>> is useful.Pretty clear to everyone except one person. :-)
🙂
I don't understand what you're getting at, here.However, the real question is about size: how bigIt's not just the rotations, but the seek time. So your example is fewer >>>>>> "operations" than the 80-150 you get when just including rotations. >>>>>
should it be. For "consumer" drives I see claims of 256 MB
cache. Given rather optimistic 200 MB/s transfer rate it is
about 1.25s of drive data, that is 80-150 rotations. I would
expect that say 4 tracks should be enough for reading. For
writing one could use few more tracks. Still, advertised cache
sizes seem to be much bigger than necessary.
I think Waldek's argument is that 256MB corresponds approximately
to the amount of data stored in 80-150 tracks, and seek time doesn't >>>>> change that fact.
Yes, I didn't express myself well. :-( And once again, I have to say >>>> that my information may be obsolete.
I think it is useful to separate talking about read data from write
data. For read data, as with any cache, more is always better than
less, though with diminishing returns. Why pick 1.25 sec as the "cut
off point"? If the host re-references data that it hasn't read for say >>>> 3 seconds, having it in cache still saves, probably a seek time and on >>>> average 1/2 rotation time. Plus, it means the heads will be free to
handle other requests. All of this is standard cache benefits. I see >>>> no reason to limit the cache size and reduce this benefit.
We are talking here about common case, that is when disc is accessed
via OS cache. OS cache is significantly larger than disc cache, so
hit ratio for data sent to host is going to be quite low. Disc
cache has an advantage: it gets "for free" some data that host did
not request. But it is rather unlikely that keeping such data
for long time has significant advantage.
And if you are caching writes, more cache gives you more blocks to choose
from when optimizing the write back order, which reduces the time to write
them all back.
IIUC, for SATA drives, NCQ is still limited to 32 in-flight commands, so >>>>> unless the drive is allowed to do write-back caching it seems the amount >>>>> of space used for write-buffering is likely small (compared to 256MB). >>>>> [ Unless it is common for individual write commands to cover multi-MB >>>>> chunks of data? ]
For write data, I was unaware of the 32 operation limit. I was used to >>>> SCSI, which, IIRC was larger, and for server type applications, where
some sort of UPS is more common, the site may choose to enable write
caching in the disk. For a disk vendor, given the small cost of the
DRAM, it is an easy choice.
I do not look at details of disc protocol. But with protocal done
right host would first transfer commands and then deliver data
in order requested by the drive. So most buffering would be in
the host and disc would need just enough buffering to ensure
smooth transmission and low interrupt rate. 4 track looks like
plenty for this purpose.
No, when the disk receives a write command, it accepts the write data
immediately (up to some large limit). That way, when the heads settle
on the track, if the disk happens to be positioned in the middle of the
transfer, it can write the last part of the data to the disk
immediately, then wait for the disk to spin to where the transfer starts
to finish the transferring the first part of the write data. This
reduces average latency, i.e. improves performance.
Latency of writes typically is of little importance, host buffers
several seconds of of write data and writes only after delay
(ratinale for this is that data may be overwritten, by dealying
host may avoid actual disc operation).
To allow scheduling in
the drive one wants commands to be sent as fast as possible.
Sending possibly bulky write data before sending next command
looks counterpoductive.
There could be some data that host
wants to be in persistent storage as fast as possible, but
making it the only option clearly would be a design error in
the disc protocol.
I can't comment on your specific numbers, but assuming you are right,
adding $1 to the cost is is small, at least in the part of the market I
was familiar with. And remember, you are not "adding" 256MB, as some of
that is needed for various internal operations.
Generously, buffering could do with about 8MB. I am not sure
how drives handle bad sector map, that potentially could be
quite large.
But in principle drive could read infor about bad
sectors from the track, keeping in RAM only info about say bad
tracks and current track. In such case I see no reason for
large internal RAM.
On 5/26/2025 1:00 PM, Chris M. Thomasson wrote:
On 5/26/2025 6:06 AM, Michael S wrote:
On Mon, 26 May 2025 06:46:01 GMT
[email protected] (Anton Ertl) wrote:
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at least in
the past. The fact that there are only 2-3 surviving HDD
manufacturers indicates intense competition in the past, possible less >>>> today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold
512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB DDR3
DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3
DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is unlikely
to save even one cent.
- anton
You are projecting computer memory prices on very different market.
The memory used for HD cache is likely an individual memory chip or two
chips and likely several generation older than devices used in computer
DIMMs. I would expect something like x16 DDR3. Looking for price of
such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less than what
we see on Mauser, there exists measurable difference between 128, 256
and 512 MB. The difference is smaller than suggested by Waldek but much
bigger than suggested by yourself.
I wonder if a HDD can have several multi-core processors on them,
where adding a new drive means you get not only more storage, but more
CPU's in the system like a NUMA machine? For some reason, the AI's
have a lot of "fun" when I talk to it about such things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a new
drive gives you more storage, more memory, and more cpus?
On 5/26/2025 2:15 PM, Stephen Fuld wrote:
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
On 5/26/2025 1:00 PM, Chris M. Thomasson wrote:
On 5/26/2025 6:06 AM, Michael S wrote:
On Mon, 26 May 2025 06:46:01 GMT
[email protected] (Anton Ertl) wrote:
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at least in >>>>>> the past. The fact that there are only 2-3 surviving HDD
manufacturers indicates intense competition in the past, possible
less
today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold >>>>>> 512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB DDR3 >>>>>> DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3 >>>>>> DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is unlikely >>>>>> to save even one cent.
- anton
You are projecting computer memory prices on very different market.
The memory used for HD cache is likely an individual memory chip or
two
chips and likely several generation older than devices used in
computer
DIMMs. I would expect something like x16 DDR3. Looking for price of >>>>> such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less than
what
we see on Mauser, there exists measurable difference between 128, 256 >>>>> and 512 MB. The difference is smaller than suggested by Waldek but
much
bigger than suggested by yourself.
I wonder if a HDD can have several multi-core processors on them,
where adding a new drive means you get not only more storage, but
more CPU's in the system like a NUMA machine? For some reason, the
AI's have a lot of "fun" when I talk to it about such things... ;^)
lol.
Or even, have some ram on there, say a couple of gigs. So, adding a
new drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable, and is almost certainly
not the same ISA as the "main" CPU in the system.
Can it be?
hard drive, many cpus, cores, threads, with dram and al that fun shit?
Or too wacky? Or stupid in a sense? Well, how stupid on a scale from 0
to 1? ;^)
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
On 5/26/2025 1:00 PM, Chris M. Thomasson wrote:
On 5/26/2025 6:06 AM, Michael S wrote:
On Mon, 26 May 2025 06:46:01 GMT
[email protected] (Anton Ertl) wrote:
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at least in >>>>> the past. The fact that there are only 2-3 surviving HDD
manufacturers indicates intense competition in the past, possible less >>>>> today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold
512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB DDR3 >>>>> DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3
DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is unlikely >>>>> to save even one cent.
- anton
You are projecting computer memory prices on very different market.
The memory used for HD cache is likely an individual memory chip or two >>>> chips and likely several generation older than devices used in computer >>>> DIMMs. I would expect something like x16 DDR3. Looking for price of >>>> such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less than what >>>> we see on Mauser, there exists measurable difference between 128, 256
and 512 MB. The difference is smaller than suggested by Waldek but much >>>> bigger than suggested by yourself.
I wonder if a HDD can have several multi-core processors on them,
where adding a new drive means you get not only more storage, but more
CPU's in the system like a NUMA machine? For some reason, the AI's
have a lot of "fun" when I talk to it about such things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a new
drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable,
and is almost certainly
not the same ISA as the "main" CPU in the system.
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
have a lot of "fun" when I talk to it about such things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a new
drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable, and is almost certainly
not the same ISA as the "main" CPU in the system.
Stephen Fuld <[email protected]d> writes:
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
have a lot of "fun" when I talk to it about such things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a new
drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable, and is almost certainly
not the same ISA as the "main" CPU in the system.
I think you might be surprised by the computing power of the
SoC on a modern enterprise grade hard disk drive, including both R
and M class ARM cores.
Yup. And IIRC the IBM 3380 had a linear actuator with two heads per
arm, one covering the outer cylinders, the other the inner
cylinders. The tradeoff was shorter seeks, thus smaller seek time but
higher cost due to more heads.
Stephen Fuld <[email protected]d> writes:
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
have a lot of "fun" when I talk to it about such things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a new
drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable, and is almost certainly
not the same ISA as the "main" CPU in the system.
I think you might be surprised by the computing power of the
SoC on a modern enterprise grade hard disk drive, including both R
and M class ARM cores. SSDs generally have more powerful
cores than HDDs.
On 5/26/2025 4:02 PM, Chris M. Thomasson wrote:
On 5/26/2025 3:40 PM, Stephen Fuld wrote:
On 5/26/2025 3:20 PM, Chris M. Thomasson wrote:
On 5/26/2025 2:15 PM, Stephen Fuld wrote:
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
On 5/26/2025 1:00 PM, Chris M. Thomasson wrote:
On 5/26/2025 6:06 AM, Michael S wrote:
On Mon, 26 May 2025 06:46:01 GMT
[email protected] (Anton Ertl) wrote:
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at
least in
the past. The fact that there are only 2-3 surviving HDD
manufacturers indicates intense competition in the past,
possible less
today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold >>>>>>>>> 512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB >>>>>>>>> DDR3
DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3 >>>>>>>>> DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is >>>>>>>>> unlikely
to save even one cent.
- anton
You are projecting computer memory prices on very different market. >>>>>>>> The memory used for HD cache is likely an individual memory chip >>>>>>>> or two
chips and likely several generation older than devices used in >>>>>>>> computer
DIMMs. I would expect something like x16 DDR3. Looking for >>>>>>>> price of
such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less
than what
we see on Mauser, there exists measurable difference between 128, >>>>>>>> 256
and 512 MB. The difference is smaller than suggested by Waldek >>>>>>>> but much
bigger than suggested by yourself.
I wonder if a HDD can have several multi-core processors on them, >>>>>>> where adding a new drive means you get not only more storage, but >>>>>>> more CPU's in the system like a NUMA machine? For some reason, the >>>>>>> AI's have a lot of "fun" when I talk to it about such
things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a >>>>>> new drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable, and is almost
certainly not the same ISA as the "main" CPU in the system.
Can it be?
Which, user programmable or same ISA?
That's a hard one. Perhaps a special ISA for the bridge, but then again,
it might be the same ISA as multiple multi-core chips on the HDD.
No disk vendor would allow user programmability as it would void any
warranty, possibly corrupt data, etc. As for same ISA, many years
ago, one vendor used an 80186, but today, any current X86 would be
overkill and too expensive for the job.
Ahh, well, then the same company that made the CPU's, GPU's, would be
the same company that made these "special" HDD's? Pie in the sky,
version 42... ;^) lol. Sorry.
Almost akin to the SPU on playstation. Except, each SPU ha a
hard drive, many cpus, cores, threads, with dram and al that fun
shit? Or too wacky? Or stupid in a sense? Well, how stupid on a scale
from 0 to 1? ;^)
No comment. :-)
LOL!!! However, this had, iirc, PPC for motherboard and a special vector
language for the SPU's?
I think some games for the older Playstation did not even use the SPU's?
They just ran on the PPC's? Too hard to program for? DMA for feeding the SPU's data?
On 5/27/2025 11:40 AM, Brett wrote:
Chris M. Thomasson <[email protected]> wrote:
On 5/26/2025 4:02 PM, Chris M. Thomasson wrote:
On 5/26/2025 3:40 PM, Stephen Fuld wrote:
On 5/26/2025 3:20 PM, Chris M. Thomasson wrote:
On 5/26/2025 2:15 PM, Stephen Fuld wrote:
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
On 5/26/2025 1:00 PM, Chris M. Thomasson wrote:
On 5/26/2025 6:06 AM, Michael S wrote:
On Mon, 26 May 2025 06:46:01 GMT
[email protected] (Anton Ertl) wrote:
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at >>>>>>>>>>> least in
the past. The fact that there are only 2-3 surviving HDD >>>>>>>>>>> manufacturers indicates intense competition in the past, >>>>>>>>>>> possible less
today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold >>>>>>>>>>> 512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB >>>>>>>>>>> DDR3
DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3 >>>>>>>>>>> DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is >>>>>>>>>>> unlikely
to save even one cent.
- anton
You are projecting computer memory prices on very different market. >>>>>>>>>> The memory used for HD cache is likely an individual memory chip >>>>>>>>>> or two
chips and likely several generation older than devices used in >>>>>>>>>> computer
DIMMs. I would expect something like x16 DDR3. Looking for >>>>>>>>>> price of
such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less >>>>>>>>>> than what
we see on Mauser, there exists measurable difference between 128, >>>>>>>>>> 256
and 512 MB. The difference is smaller than suggested by Waldek >>>>>>>>>> but much
bigger than suggested by yourself.
I wonder if a HDD can have several multi-core processors on them, >>>>>>>>> where adding a new drive means you get not only more storage, but >>>>>>>>> more CPU's in the system like a NUMA machine? For some reason, the >>>>>>>>> AI's have a lot of "fun" when I talk to it about such
things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a >>>>>>>> new drive gives you more storage, more memory, and more cpus?
The CPU on the drive is not user programmable, and is almost
certainly not the same ISA as the "main" CPU in the system.
Can it be?
Which, user programmable or same ISA?
That's a hard one. Perhaps a special ISA for the bridge, but then again, >>>> it might be the same ISA as multiple multi-core chips on the HDD.
No disk vendor would allow user programmability as it would void any >>>>> warranty, possibly corrupt data, etc. As for same ISA, many years
ago, one vendor used an 80186, but today, any current X86 would be
overkill and too expensive for the job.
Ahh, well, then the same company that made the CPU's, GPU's, would be
the same company that made these "special" HDD's? Pie in the sky,
version 42... ;^) lol. Sorry.
Almost akin to the SPU on playstation. Except, each SPU ha a
hard drive, many cpus, cores, threads, with dram and al that fun
shit? Or too wacky? Or stupid in a sense? Well, how stupid on a scale >>>>>> from 0 to 1? ;^)
No comment. :-)
LOL!!! However, this had, iirc, PPC for motherboard and a special vector >>>> language for the SPU's?
I think some games for the older Playstation did not even use the SPU's? >>> They just ran on the PPC's? Too hard to program for? DMA for feeding the >>> SPU's data?
The SPU’s were vector engines used for 3D transforms, the GPU was just a a >> blit engine with no transform capabilities.
The main CPU was too slow for to do all the transformations and ship a
usable game that would pass Sony certification.
You could do a lot more with the SPU vector engines if you rewrote your
code, cloth physics, particles, etc, this is what the developers were
talking about.
Right, but some "indy" "non-professional non AAA titles?"
games could be
played on the PPC? Now, some people grumbled about how it was hard to "program" for sending commands, data, to the SPU's, and get the data
back. Iirc, send vertices, part of a mesh, think if SPU A worked on
animating the left leg, some uniform variables (aka bones, instance
index, ect), SPU B worked on animating the right leg, and so on. Well, without proper sync, the leg animations, say they were indexed as say, left_leg_move_0, left_leg_move_1 for SPU A, and right_leg_move_0, ...,
until the all of them are complete and we can use it to actually render
a damn frame. Think of animating bones where we need to send multiple
mat4's to the shaders that are used to animate are completed., in the
vertex shader, perhaps even in the fragment shader. Fwiw, here is an
older try of mine using modern Opengl and my own engine, own shaders,
all in C++, and GLSL:
(Fractal Boom Box)
https://youtu.be/n13GHyYEfLA
I made the music as well.
That makes sense, but even if the advantage is small, the cost is
essentially zero, so why not do it.
On 5/28/2025 10:10 PM, Brett wrote:
Chris M. Thomasson <[email protected]> wrote:
On 5/27/2025 11:40 AM, Brett wrote:
Chris M. Thomasson <[email protected]> wrote:
On 5/26/2025 4:02 PM, Chris M. Thomasson wrote:
On 5/26/2025 3:40 PM, Stephen Fuld wrote:
On 5/26/2025 3:20 PM, Chris M. Thomasson wrote:
On 5/26/2025 2:15 PM, Stephen Fuld wrote:
On 5/26/2025 1:04 PM, Chris M. Thomasson wrote:
On 5/26/2025 1:00 PM, Chris M. Thomasson wrote:The CPU on the drive is not user programmable, and is almost >>>>>>>>> certainly not the same ISA as the "main" CPU in the system.
On 5/26/2025 6:06 AM, Michael S wrote:
On Mon, 26 May 2025 06:46:01 GMT
[email protected] (Anton Ertl) wrote:
[email protected] (Waldek Hebisch) writes:
Discs probably are less competitive than
motherboards,
I expect them to be just as competetive as motherboards, at >>>>>>>>>>>>> least in
the past. The fact that there are only 2-3 surviving HDD >>>>>>>>>>>>> manufacturers indicates intense competition in the past, >>>>>>>>>>>>> possible less
today.
but I would expect adding 256 MB to lead to 1
dollar or more increase of cost.
What makes you think so. The DRAM chips on DDR4 DIMMs today hold
512MB (x8->4GB DIMM) up to 2GB (x16->32GB DIMM). There are 2GB >>>>>>>>>>>>> DDR3
DIMMs (using 256MB chips), but they do not cost less than 4GB DDR3
DIMMs. Choosing a DRAM cache of 256MB rather than 512MB is >>>>>>>>>>>>> unlikely
to save even one cent.
- anton
You are projecting computer memory prices on very different market.
The memory used for HD cache is likely an individual memory chip >>>>>>>>>>>> or two
chips and likely several generation older than devices used in >>>>>>>>>>>> computer
DIMMs. I would expect something like x16 DDR3. Looking for >>>>>>>>>>>> price of
such devices on Mauser I see following figures:
1 Gbit - $2.10
2 Gbit - $2.60
4 Gbit - $2.90
So, even assuming that disk manufacturer pays 1.5x to 2x less >>>>>>>>>>>> than what
we see on Mauser, there exists measurable difference between 128, >>>>>>>>>>>> 256
and 512 MB. The difference is smaller than suggested by Waldek >>>>>>>>>>>> but much
bigger than suggested by yourself.
I wonder if a HDD can have several multi-core processors on them, >>>>>>>>>>> where adding a new drive means you get not only more storage, but >>>>>>>>>>> more CPU's in the system like a NUMA machine? For some reason, the >>>>>>>>>>> AI's have a lot of "fun" when I talk to it about such
things... ;^) lol.
Or even, have some ram on there, say a couple of gigs. So, adding a >>>>>>>>>> new drive gives you more storage, more memory, and more cpus? >>>>>>>>>
Can it be?
Which, user programmable or same ISA?
That's a hard one. Perhaps a special ISA for the bridge, but then again, >>>>>> it might be the same ISA as multiple multi-core chips on the HDD.
No disk vendor would allow user programmability as it would void any >>>>>>> warranty, possibly corrupt data, etc. As for same ISA, many years >>>>>>> ago, one vendor used an 80186, but today, any current X86 would be >>>>>>> overkill and too expensive for the job.
Ahh, well, then the same company that made the CPU's, GPU's, would be >>>>>> the same company that made these "special" HDD's? Pie in the sky,
version 42... ;^) lol. Sorry.
Almost akin to the SPU on playstation. Except, each SPU ha a
hard drive, many cpus, cores, threads, with dram and al that fun >>>>>>>> shit? Or too wacky? Or stupid in a sense? Well, how stupid on a scale >>>>>>>> from 0 to 1? ;^)
No comment. :-)
LOL!!! However, this had, iirc, PPC for motherboard and a special vector >>>>>> language for the SPU's?
I think some games for the older Playstation did not even use the SPU's? >>>>> They just ran on the PPC's? Too hard to program for? DMA for feeding the >>>>> SPU's data?
The SPU’s were vector engines used for 3D transforms, the GPU was just a a
blit engine with no transform capabilities.
The main CPU was too slow for to do all the transformations and ship a >>>> usable game that would pass Sony certification.
You could do a lot more with the SPU vector engines if you rewrote your >>>> code, cloth physics, particles, etc, this is what the developers were
talking about.
Right, but some "indy" "non-professional non AAA titles?"
The Sony console platforms are closed to Indy and non-AAA titles.
Only an AI could make such a blunder.
Are you sure that no titles did not make use of all the SPU's, some of
them, any of them?
games could be
played on the PPC? Now, some people grumbled about how it was hard to
"program" for sending commands, data, to the SPU's, and get the data
back. Iirc, send vertices, part of a mesh, think if SPU A worked on
animating the left leg, some uniform variables (aka bones, instance
index, ect), SPU B worked on animating the right leg, and so on. Well,
without proper sync, the leg animations, say they were indexed as say,
left_leg_move_0, left_leg_move_1 for SPU A, and right_leg_move_0, ...,
until the all of them are complete and we can use it to actually render
a damn frame. Think of animating bones where we need to send multiple
mat4's to the shaders that are used to animate are completed., in the
vertex shader, perhaps even in the fragment shader. Fwiw, here is an
older try of mine using modern Opengl and my own engine, own shaders,
all in C++, and GLSL:
(Fractal Boom Box)
https://youtu.be/n13GHyYEfLA
I made the music as
On Mon, 26 May 2025 14:12:09 -0700, Stephen Fuld wrote:
That makes sense, but even if the advantage is small, the cost is
essentially zero, so why not do it.
Additional complexity, leading to additional points of failure.
On 5/29/2025 6:42 PM, Lawrence D'Oliveiro wrote:
On Mon, 26 May 2025 14:12:09 -0700, Stephen Fuld wrote:
That makes sense, but even if the advantage is small, the cost is
essentially zero, so why not do it.
Additional complexity, leading to additional points of failure.
The "it" to which we are referring here is caching of write data.
I was too flip in my answer, so here is, I think, a better one. The
"it" to which we are referring here is caching of write data.
So let's look at a possible scenario. Let's say the heads are at
cylinder 100. A write comes in for data that is at cylinder
300. Without write caching, the disk will move the heads to cylinder
300. Now lets say the next request is a read for data at cylinder 150.
If the write had been cached, the disk can handle the read with only a
50 cylinder move, then the write with a 150 cylinder move for a total
of 200 cylinders. Without write caching, the first move is 200
cylinders for the write, followed by 150 back for the read for a total
of 350. Thus the read data, which is presumably more time critical, is delayed.
Overall, write caching improves performance, but if you don't want it,
then you can essentially not use it, either by forcing the writes to
go to the media, or not using command queuing at all.
Early 70s, as mainstream IBM was converting everything to virtual
memory, I got into a battle. Somebody came up with a (LRU?) page
replacement algorithm that would replace non-changed pages (didn't
require a write before the read replacement) before changed pages (which >needed a write before being able to fetch the needed page). Nearly a
decade later, they finally realized that they were replacing highly
used, highly shared RO/non-changed pages ... before replacing, private, >single-task, changed data page (before they realized it was possible to
keep a pool of immediately available, changed pages that had been >pre-written).
since sometime in the 80s, (at least) RDBMS have been using "write
caching" (write behind) where the sequential log/journal of "committed" transactions is made and actual RDBMS writes happen in the background. Failure recovery requires rereading the log and forcing pending writes
for committed transactions.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 11:09:20 |
| Calls: | 12,100 |
| Files: | 15,003 |
| Messages: | 6,517,992 |