Skip to main content
Definition Website Computer and Technology

Parallelism

Subject: Computer and Technology noun /ˈpær.əˌlɛl.ɪz.əm/ 1 definition

  1. Definition (noun)

    Entry #183 · Updated

    Parallelism is the simultaneous execution of multiple instructions, tasks, or data operations at the same instant across separate hardware resources, such as multi-core processors, vector units, graphics accelerators, or distributed cluster nodes. Unlike concurrency—which structures and interleaves overlapping tasks that may share a single core through time-slicing—parallelism requires physical hardware simultaneity to reduce wall-clock latency or scale total throughput. Its achievable speedup is bounded by inherently sequential portions of a workload and the overhead of synchronization and inter-unit communication.

    Common Types

    • Instruction-Level Parallelism: Hardware techniques such as pipelining and superscalar execution that issue multiple independent instructions from a single program stream simultaneously within one core.

    • Data Parallelism: Applying the same operation simultaneously across distinct partitions of a dataset, commonly using CPU vector registers or massively parallel GPU compute arrays.

    • Task Parallelism: Running distinct functions or independent threads concurrently across multiple processor cores, coordinating execution through shared memory, queues, or synchronization primitives.

    • Distributed Parallelism: Partitioning large jobs across separate networked machines that share no physical memory and communicate via message passing or remote procedure calls.

    Example

    During 4K video transcoding, a media pipeline assigns independent 10-second chunks to eight CPU cores while executing pixel-level color grading across thousands of GPU compute threads simultaneously. By processing these operations in parallel rather than sequentially on a single core, total export time drops from 24 minutes to under 4 minutes.