George Neuner <
[email protected]> writes:
2: many programmers are much better at figuring out what CAN be
done in parallel than they are at figuring out what SHOULD be
done in parallel. The result often is too many threads each
making very little progress.
Yes, this is a hard problem in general. Compilers and OSs are pretty
bad at this, too. In theory, with a referentially transparent
programming language, there is a lot of parallelism that could be
exploited by the compiler, and that has been touted as one advantage
of such programming languages decades ago. In practice, that has not
happened.
You could even consider every operation in a language like, say, C or
Java, to be a parallelizable task that only has to obey the data-flow dependencies, and you would have a lot of parallelism. Yes, without referential transparency you would have to worry about dependencies
through aliases.
But let's assume for the moment that aliasing is not a problem: In
that setting, treating every operation or instruction as a
parallelizable task obviously has too much task creation and
destruction and scheduling overhead, and of course also communication
overhead; even if the task creation, destruction and scheduling
overhead is zero, there still remains the communication overhead:
Your sequential programs usually have good temporal and spatial
locality, that's why registers, caches and hardware prefetchers work
so well; but the splitting into tasks has destroyed the sequential
structure, so your scheduler would have to perform the tasks in an
order that has similarly good temporal and spatial locality (your
"scheduler" could consist of a dynamic scheduler and a static
(compiler) scheduler that combines a bunch of tasks into one). But
will automatic techniques do any better at auto-sequentialization than
they do at auto-parallelization? I doubt it.
3: the skill level of the average programmer now is only slightly
above "novice". More software is being written now than ever
before, but the vast majority of it is poor quality.
Even if you assume master programmers, they have their hands full with satisfying the requirements, and then on to the next project. How
much of their time do you want to spend on parallelizing the program
rather than working on the next project?
Parallelizing for close to maximum performance is only affordable if
the hardware is more expensive than the software, i.e., in
supercomputing, and in recent years I have seen some indications that
that is changing even there (i.e., the software crisis has reached supercomputing); I guess it's not very effective to get a new
supercomputer and then have people rewriting software for three years
in order to extract another factor of 2. Better design the hardware
such that the existing software works on it well from the start.
My favorite model is CSP (ala Hoare) with no shared memory. Which is
not to say I don't use threads, but I try to design programs such that >threads (mostly) are not sharing writable data structures.
That's certainly along the lines of my thinking. What's your
experience with that?
- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <
[email protected]>
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)