Skip to main content
Definition Website Computer and Technology

Recurrent Neural Network

Subject: Computer and Technology noun /rɪˈkɜːr.ənt ˈnʊr.əl ˈnɛtˌwɜːrk/ 1 definition

  1. Definition (noun)

    Entry #289 · Updated

    A Recurrent Neural Network (RNN) is a deep learning architecture designed for sequential data—such as time series, audio streams, and natural-language text—where the order and temporal context of inputs matter. Unlike feedforward networks that evaluate each input independently, an RNN processes elements one time step at a time and feeds its internal hidden state back into the next computation using shared weights across the sequence. During training, algorithms such as backpropagation through time (BPTT) unroll the recurrent loop across steps to update parameters, though standard RNNs suffer from vanishing or exploding gradients over long dependencies. While gated variants extend memory capacity across wider spans, step-by-step recurrence inherently limits training parallelization compared to attention-based architectures like Transformers.

    Common Types

    • Vanilla RNN: Applies a simple recurrent activation (typically tanh or ReLU) to combine the current input with the previous hidden state; lightweight to implement, though ineffective at retaining context over long sequences.

    • Long Short-Term Memory (LSTM): Adds a dedicated memory cell regulated by input, forget, and output gates to selectively retain or discard information across hundreds of time steps, mitigating the vanishing gradient problem.

    • Gated Recurrent Unit (GRU): Simplifies the LSTM design into reset and update gates without a separate cell state, delivering comparable sequence-modeling accuracy with fewer parameters and faster training.

    • Bidirectional RNN (BiRNN): Combines two independent recurrent layers that traverse a sequence forward and backward, allowing each output step to draw on both past and future context in tasks such as speech transcription and sequence labeling.

    Example

    An industrial monitoring service runs a two-layer LSTM model exported via ONNX Runtime on an edge gateway to forecast turbine bearing temperatures from 100 Hz MQTT sensor streams. At each 10-millisecond tick, the network ingests a 6-feature telemetry vector—including vibration RMS, shaft RPM, and ambient temperature—and updates its 128-dimensional hidden state to flag anomalous thermal drift before thresholds are breached.