1 Introduction

Simultaneous localization and mapping (SLAM) is a fundamental computational problem in robotics and autonomous systems wherein a moving agent must construct a map of an unknown environment while simultaneously keeping track of its own location within that map. The core difficulty arises from a circular dependency: accurate localization requires a consistent map, but building the map depends on knowing the agent’s pose. SLAM algorithms typically employ sensor data—such as lidar, cameras, or inertial measurement units (IMUs)—alongside probabilistic estimation methods to jointly solve for both unknowns. Since its formalization in the 1980s, SLAM has become a foundational technology in autonomous navigation, augmented reality, and drone guidance, enabling real‑time, self‑contained spatial reasoning.

1.1 Historical background

The origins of SLAM trace to early work on probabilistic robotics in the 1980s. Researchers such as Hugh Durrant‑Whyte and John J. Leonard formulated the problem as a stochastic estimation task, initially using Kalman filters to manage uncertainty. In the 1990s, the introduction of the extended Kalman filter (EKF) provided a practical framework for real‑time SLAM with range sensors. A landmark achievement was the 1999 publication of “A solution to the simultaneous localization and map building problem” by Dissanayake et al., which proved theoretical convergence conditions. The 2000s saw the rise of particle‑filter‑based methods (FastSLAM) and graph‑based optimization, dramatically improving scalability and accuracy. The convergence of affordable sensors and increased computing power in the 2010s propelled SLAM into consumer applications.

1.2 Motivation and importance

SLAM is motivated by the need for autonomous agents to operate in unknown, unstructured environments without relying on external positioning infrastructure (e.g., GPS). Importance spans multiple domains: self‑driving cars depend on SLAM for real‑time localization and obstacle avoidance; mobile robots in warehouses use SLAM to navigate dynamic aisles; augmented‑reality devices require low‑latency sensor fusion to overlay virtual objects on the real world; and drones rely on SLAM for GPS‑denied flight. The ability to jointly solve mapping and localization without prior knowledge is considered a key enabler of full autonomy.

2 Problem formulation

The SLAM problem is formally defined as estimating the pose of a moving agent and the positions of static landmarks in an unknown environment, given only noisy sensor measurements and control inputs over time.

2.1 Mathematical model

2.1.1 State vector and observations

At time \(t\), the agent’s full state includes its pose (position and orientation) \(\mathbf{x}_t\) and the set of all landmark positions \(\mathbf{m}_i\). The combined state vector is \(\mathbf{X}_t = \{\mathbf{x}_t, \mathbf{m}_1, \mathbf{m}_2, \dots, \mathbf{m}_N\}\). The agent receives control inputs \(\mathbf{u}_t\) (e.g., odometry) that predict how the pose evolves, and observations \(\mathbf{z}_t\) that relate the sensor reading to a subset of landmarks. The motion model is \(\mathbf{x}_t = f(\mathbf{x}_{t-1}, \mathbf{u}_t) + \mathbf{w}_t\) and the observation model is \(\mathbf{z}_t = h(\mathbf{x}_t, \mathbf{m}_j) + \mathbf{v}_t\), where \(\mathbf{w}_t\) and \(\mathbf{v}_t\) represent process and measurement noise, respectively.

2.1.2 Probabilistic interpretation (Bayes filter)

SLAM is naturally cast as a Bayesian filtering problem: estimate the posterior probability \(P(\mathbf{x}_t, \mathbf{m} \mid \mathbf{z}_{1:t}, \mathbf{u}_{1:t})\). The Bayes filter recursively updates this distribution via a prediction step (using the motion model) and a correction step (using observations). For nonlinear models, the extended Kalman filter (EKF) approximates the posterior as a Gaussian, while particle filters represent it as a set of weighted samples.

2.2 Loop closure and data association

A critical subproblem is loop closure—recognizing when the agent revisits a previously mapped area. Accurate loop closure can correct accumulated drift in the pose estimates. Data association determines which observation corresponds to which existing landmark (or to a new landmark). Incorrect associations (e.g., confusing two similar features) can cause catastrophic failure; robust data association algorithms (e.g., joint compatibility branch and bound, RANSAC) are therefore central to SLAM.

3 Sensor modalities

Different sensors provide complementary information for SLAM. The choice of modality affects the algorithm’s accuracy, complexity, and environmental suitability.

3.1 Lidar‑based SLAM

Lidar (light detection and ranging) sensors emit laser pulses and measure time‑of‑flight to produce high‑resolution 3D point clouds. Lidar‑based SLAM is common in autonomous vehicles and outdoor robotics due to its long range and robustness to lighting conditions. Algorithms such as LOAM (Lidar Odometry and Mapping) and its variants (e.g., A‑LOAM, LeGO‑LOAM) use scan‑matching techniques (iterative closest point, feature extraction) to estimate motion and build consistent maps.

3.2 Visual SLAM

Visual SLAM uses cameras as the primary sensor. It is cost‑effective, lightweight, and provides rich texture information. However, it is sensitive to lighting, motion blur, and low‑texture surfaces.

3.2.1 Monocular SLAM

A single camera yields 2D images; monocular SLAM recovers 3D structure up to an unknown scale. Landmarks are usually keypoints (e.g., ORB, SIFT) extracted from frames. Pioneering systems include PTAM (Parallel Tracking and Mapping) and ORB‑SLAM. Scale ambiguity can be resolved through sensor fusion (e.g., with an IMU) or by assuming a known initial size.

3.2.2 Stereo and RGB‑D SLAM

Stereo cameras provide depth from disparity, giving direct metric scale. RGB‑D cameras (e.g., Microsoft Kinect, Intel RealSense) produce registered color and depth images. Both modalities eliminate scale ambiguity and simplify depth estimation. Systems like ORB‑SLAM2, DVO‑SLAM, and ElasticFusion operate in real time, especially in indoor environments.

3.3 Inertial SLAM and sensor fusion

Inertial measurement units (IMUs) measure angular velocity and linear acceleration. While IMUs suffer from drift over long periods, they offer high‑frequency motion estimates and are unaffected by lighting or texture. Sensor fusion (e.g., with a camera or lidar) combines the strengths: the IMU bridges gaps during fast motion or featureless regions, while the exteroceptive sensor corrects drift. Visual‑inertial SLAM (VINS) and lidar‑inertial odometry (LIO) are mature approaches.

4 Core algorithmic approaches

Three dominant families of SLAM algorithms have evolved: filter‑based, particle‑filter, and graph‑based methods.

4.1 Extended Kalman Filter (EKF) SLAM

EKF SLAM was one of the earliest formulations and remains pedagogically important.

4.1.1 State representation and prediction

The state vector includes the robot’s pose and all landmark positions, with a joint covariance matrix representing uncertainty. The prediction step propagates the mean and covariance through the motion model, adding process noise. The computational cost grows quadratically with the number of landmarks due to covariance updates (O(N²)).

4.1.2 Update and landmark management

When an observation is made, the EKF update corrects the entire state estimate and its covariance. New landmarks are initialized by augmenting the state and covariance matrix. Landmark management includes strategies to reduce complexity, such as submapping or sparsification.

4.2 Particle filter (FastSLAM)

FastSLAM factorizes the SLAM posterior into a product of a robot path distribution (represented by particles) and landmark positions conditioned on each particle.

4.2.1 Rao‑Blackwellization

Rao‑Blackwellization decomposes the estimation: the robot path is sampled using a particle filter, while each particle maintains independent Kalman filters for each landmark. This reduces the complexity for a fixed number of particles and allows real‑time performance with large maps.

4.2.2 Resampling strategies

Particle degeneracy (where only a few particles have significant weight) is a common issue. Resampling methods (systematic, multinomial, stratified) periodically replicate high‑weight particles and discard low‑weight ones. Adaptive resampling (e.g., based on effective sample size) helps maintain diversity.

4.3 Graph‑based SLAM

Graph‑based SLAM frames the problem as a sparse graph optimization, which is now the most widely adopted approach.

4.3.1 Factor graphs and constraints

Nodes represent robot poses and landmark positions. Edges (factors) encode constraints from odometry, observations, and loop closures. The graph’s sparsity enables efficient optimization of the entire trajectory and map.

4.3.2 Nonlinear least squares optimization (e.g., iSAM, g2o)

The graph defines an error function—the sum of squared errors over all constraints—which is minimized using nonlinear least squares solvers. Incremental smoothing and mapping (iSAM) and its successor iSAM2 perform incremental updates, while g2o is a general‑purpose library. These methods achieve real‑time performance for large maps.

4.4 Visual‑inertial SLAM (VINS)

VINS integrates visual features with IMU measurements in a tightly coupled manner. The IMU provides short‑term motion estimates; the camera corrects drift and provides loop‑closure constraints. Systems like VINS‑Mono and VINS‑Fusion operate on both monocular and stereo cameras, achieving robust performance in challenging conditions (e.g., rapid motion, low texture).

5 Key challenges

Despite decades of progress, several challenges remain active research topics.

5.1 Computational complexity

Maintaining a consistent global map, especially with thousands of landmarks or long trajectories, can be computationally demanding. Real‑time operation requires careful algorithm design, sparsification, and efficient data structures (e.g., KD‑trees for landmark searches).

5.2 Scalability and long‑term operation

As the environment grows, the state dimension increases, leading to larger covariance matrices or graphs. Techniques like submapping, keyframe selection, and marginalization help limit computational growth. Long‑term operation also requires dealing with resource exhaustion (e.g., memory) and environment changes.

5.3 Dynamic environments and moving objects

Most SLAM algorithms assume a static world. Moving objects (people, vehicles) violate this assumption, corrupting landmark estimates. Dynamic SLAM approaches detect and exclude moving points or treat them as temporary features. Some systems incorporate semantic segmentation to classify objects.

5.4 Perceptual aliasing and false loops

Similar‑looking places (e.g., hallways, forests) can cause false loop closures, corrupting the map. Robust place recognition (e.g., using bag‑of‑words or deep features) and outlier rejection (e.g., RANSAC, switchable constraints) mitigate this risk.

6 Evaluation and benchmarks

Standardized datasets and metrics enable fair comparison of SLAM systems.

6.1 Standard datasets (KITTI, EuRoC, TUM)

  • KITTI: A large‑scale outdoor dataset with lidar, cameras, and GPS ground truth, primarily for autonomous driving.
  • EuRoC MAV: Micro aerial vehicle dataset with stereo cameras and IMU, recorded in indoor and outdoor scenes.
  • TUM RGB‑D: Indoor sequences with RGB‑D cameras, providing ground truth from a motion‑capture system.
  • Other datasets: MIT Stata Center, ETH‑MSI, Newer College, etc., cover varied environments and sensor configurations.

6.2 Performance metrics (ATE, RPE, map consistency)

  • Absolute Trajectory Error (ATE): Root‑mean‑square difference between estimated and ground‑truth poses after alignment.
  • Relative Pose Error (RPE): Drift over fixed time intervals, capturing local accuracy.
  • Map consistency: Measures such as point‑cloud alignment error, number of landmarks, or loop‑closure success rate.
  • Computational metrics: CPU/GPU usage, memory footprint, and frame rate.

7 Applications

SLAM has been deployed in a wide range of real‑world systems.

7.1 Autonomous vehicles

Self‑driving cars use SLAM (often lidar‑based) for real‑time localization and mapping in urban environments. Systems like Autoware and Apollo integrate SLAM with GPS, HD maps, and perception for safe navigation.

7.2 Mobile robots and warehouse automation

In warehouses, robots such as those from Amazon Robotics use SLAM to navigate aisles, avoid obstacles, and pick items. SLAM enables operation without external beacons, simplifying deployment.

7.3 Augmented and virtual reality

AR/VR headsets (e.g., Microsoft HoloLens, Apple Vision Pro) rely on visual‑inertial SLAM to track head motion and map the user’s surroundings for placing virtual content. Low‑latency SLAM is critical for user comfort.

7.4 Aerial drones and UAVs

Drones use SLAM for autonomous flight in GPS‑denied environments (e.g., indoors, caves, under bridges). Systems like the Parrot Bebop and DJI’s “Return to Home” utilize visual or lidar SLAM for stabilization and path planning.

Research continues to push SLAM beyond its traditional boundaries.

8.1 Deep learning for SLAM (e.g., end‑to‑end, learned features)

Deep learning has been applied to every component: learned feature descriptors (SuperPoint, D2‑Net), depth estimation (monodepth, DORN), and even direct end‑to‑end SLAM via recurrent neural networks (e.g., DeepVO, DROID‑SLAM). Learned systems can be more robust to lighting and viewpoint changes but often require extensive training data.

8.2 Semantic SLAM and scene understanding

Semantic SLAM augments geometric maps with object labels and scene categories. Systems like SemanticFusion, Panoptic‑SLAM, and Kimera integrate semantic segmentation to improve data association, enable place recognition based on objects, and produce human‑interpretable maps.

8.3 Collaborative multi‑robot SLAM

Multiple agents can share maps and pose estimates to cover larger areas faster. Challenges include maintaining consistent coordinate frames, handling stale information, and merging overlapping maps. Algorithms such as CCM‑SLAM and DCL‑SLAM achieve centralized or distributed fusion.

8.4 Resilient SLAM under extreme conditions

SLAM in challenging environments—darkness, underwater, underground, low‑texture scenes—remains an open problem. Solutions include using novel sensors (event cameras, sonar), robust estimation (e.g., M‑estimation), and learning‑based depth priors. Resilience to sensor failure and communication loss is also a growing focus for safety‑critical applications.