Reinforcement Learning
Subject: Computer and Technology noun /ˌriː.ɪnˈfɔːrs.mənt ˈlɜːr.nɪŋ/ 1 definition
- machine-learning
- artificial-intelligence
- computer-science
- decision-making
-
Definition (noun)
Entry #293 · UpdatedReinforcement Learning (RL) is a machine learning paradigm in which an autonomous agent learns a decision-making policy through sequential interaction with an environment to maximize expected cumulative reward. Typically formalized as a Markov Decision Process (MDP), each step requires the agent to observe the current state, choose an action, and receive a scalar reward alongside the next state, using those transitions to update its value functions or policy parameters. Unlike supervised learning—which relies on static labeled examples—RL learns from evaluative and often delayed feedback, balancing exploration of untried actions against exploitation of high-reward behaviors while addressing temporal credit assignment. Although well suited to sequential control and multi-step optimization, RL is bounded by high sample complexity, sensitivity to poorly specified reward functions, and safety constraints when interacting directly with physical or production systems.
Common Forms
Value-Based Methods: Estimate the expected cumulative return of states or state-action pairs—using algorithms such as Q-learning and Deep Q-Networks (DQN)—and derive a policy implicitly by choosing the highest-valued action in discrete spaces.
Policy-Gradient Methods: Parameterize the policy directly and optimize its weights via gradient ascent on expected return, as in REINFORCE, supporting stochastic policies and continuous action spaces.
Actor-Critic Methods: Pair a policy network (the actor) that selects actions with a value network (the critic) that estimates baseline returns to reduce gradient variance, forming the foundation of algorithms like PPO and SAC.
Model-Based Methods: Learn or incorporate an explicit model of environment transition dynamics and rewards—as in Dyna-Q or MuZero—to simulate rollouts and plan trajectories internally, improving sample efficiency over model-free approaches.
Example
A robotics team trains a quadruped locomotion policy in MuJoCo through the
gymnasium.EnvAPI using Proximal Policy Optimization (PPO), where each 20-millisecond control step passes a 48-dimensional joint-state vector and outputs 12 actuator commands. The environment'sstep()method returns the next state and a scalar reward combining forward velocity minus torque penalties and a-10.0fall penalty, enabling the agent to iteratively update its actor-critic weights over millions of rollouts until it achieves a stable gait.