Parallelism
Subject: Computer and Technology noun /ˈpær.əˌlɛl.ɪz.əm/ 1 definition
- computer-architecture
- parallel-computing
- distributed-systems
- software-engineering
-
Definition (noun)
Entry #183 · UpdatedParallelism is the simultaneous execution of multiple instructions, tasks, or data operations at the same instant across separate hardware resources, such as multi-core processors, vector units, graphics accelerators, or distributed cluster nodes. Unlike concurrency—which structures and interleaves overlapping tasks that may share a single core through time-slicing—parallelism requires physical hardware simultaneity to reduce wall-clock latency or scale total throughput. Its achievable speedup is bounded by inherently sequential portions of a workload and the overhead of synchronization and inter-unit communication.
Common Types
Instruction-Level Parallelism: Hardware techniques such as pipelining and superscalar execution that issue multiple independent instructions from a single program stream simultaneously within one core.
Data Parallelism: Applying the same operation simultaneously across distinct partitions of a dataset, commonly using CPU vector registers or massively parallel GPU compute arrays.
Task Parallelism: Running distinct functions or independent threads concurrently across multiple processor cores, coordinating execution through shared memory, queues, or synchronization primitives.
Distributed Parallelism: Partitioning large jobs across separate networked machines that share no physical memory and communicate via message passing or remote procedure calls.
Example
During 4K video transcoding, a media pipeline assigns independent 10-second chunks to eight CPU cores while executing pixel-level color grading across thousands of GPU compute threads simultaneously. By processing these operations in parallel rather than sequentially on a single core, total export time drops from 24 minutes to under 4 minutes.