several decades now (since Gordon More conjectured his Moore’s
“Law”), the computing industry has benefited from ever-increasing
processor speed. Application developers, therefore, had the
privilege of planning ahead for more powerful application
software with the assurance that the computing power required
will be available. Processor manufacturers could also plan
to keep increasing processor speeds with the assurance that
application developers were ready to use them. In terms of
programmer productivity, the most important implication was
that most programmers could write sequential code; the only
exceptions being those who wrote operating systems or application
software for “supercomputers”.
few years ago, this convenient symbiotic relationship began
to unravel. The main “culprit” was limitations of technology.
It was no longer feasible to make processors faster because
it was not possible to handle the heat generated by the faster
Moore’s Law continues to hold, hence it is possible to keep
packing an increasing number of transistors on a chip. Given
the continued opportunity provided by Moore’s Law and faced
with the heat dissipation problem, the industry stopped increasing
the speed of processors and, instead, started designing chips
with multiple processors. This meant that the industry could
still bring out chips which had increasing computing power
albeit with each processor “core” not getting any faster.
downside of this development is that all programmers — and
not just a few select ones — need to write parallel programs
to be able to actually use the computing power of multi-processor
or multi-core chips. Parallel Programming has to go mainstream!
are two challenges a programmer faces when designing parallel
programs: ensuring correctness and extracting the maximum
the last few decades, the Computer Science community has addressed
the challenge of correctness fairly well. Besides developing
alternative programming paradigms — shared-memory and message-passing
— programming languages have been designed with appropriate
constructs to help (but not necessarily ensure) absence of
typical parallel-programming bugbears such as deadlocks and
is still sorely lacking, however, is any systematic methodology
to improve the performance of programs. Getting the best possible
performance for a given parallel program from the underlying
hardware is still an art, bordering on “black magic”. The
industry is in a dire need to create such a systematic methodology.
The primary obstacle to creating such a systematic methodology
is that we do not yet have a programming model of hardware
at the right level of abstraction. We have the ISA (Instruction
Set Architecture) that is excellent for addressing functionality
but has no information about hardware performance. At the
other end of the spectrum, we have the RTL (Register-Transfer
Level) model of hardware that provides information about hardware
performance but is too detailed (not abstract enough) to be
useful to programmers (at least those who are not experts).
industry today is witnessing acceleration in the rate of reduction
of hardware costs, an example being the one teraflops double-precision
performance of Intel’s KNC chip. This has created the exciting
opportunity to bring high-performance computing into the mainstream.
However, to make this happen needs a large number of engineers
who can design correct and efficient parallel programs.
Today we have a fairly good systematic methodology to design
parallel programs that are functionally correct. Programmer
productivity is being further enhanced with the development
of Domain-Specific Languages (DSL’s). This needs to be urgently
complemented with a systematic methodology to enhance performance
on a given target hardware platform. The development of such
a methodology must form one of the core themes of research
for the computing community.