AlphaZero is a reinforcement learning algorithm developed by DeepMind (a subsidiary of Google) that achieved superhuman performance in the board games of Go, chess, and shogi (Japanese chess) without any prior human knowledge beyond the game rules. It uses a combination of deep neural networks and Monte Carlo tree search, learning entirely through self-play. The algorithm's success demonstrated the power of tabula rasa learning in complex strategic domains and sparked further research into general‑purpose game‑playing systems.
1 Algorithm overview
AlphaZero combines a deep neural network with Monte Carlo tree search (MCTS) to evaluate positions and select moves. The neural network is trained exclusively from data generated during self‑play, without any human‑generated games or expert knowledge. During play, MCTS uses the network’s predictions to guide its search, and the search results in turn refine the network.
1.1 Neural network architecture
The neural network in AlphaZero is a single deep convolutional network that takes as input a representation of the board state (e.g., piece positions, whose turn, castling rights). It has two output heads: a policy head that outputs a probability distribution over legal moves, and a value head that outputs a scalar estimate of the game outcome (win/loss/draw) from the current player’s perspective. The network is trained to minimise the combined loss of policy and value predictions.
1.2 Monte Carlo tree search integration
During play, AlphaZero uses MCTS to simulate many game trajectories from the current state. Each simulation uses the neural network to evaluate leaf nodes and guide selection. The search accumulates visit counts and value estimates, and the final move is chosen proportionally to the visit counts. This integration allows the algorithm to combine deep learning with look‑ahead search, achieving stronger performance than either component alone.
1.3 Self‑play training loop
AlphaZero learns by playing millions of games against itself. In each game, the current network generates move probabilities and values via MCTS. After the game ends, every move is labelled with the final outcome, and these data are used to update the network via gradient descent. The network weights are periodically evaluated against the best‑known version; if the new version wins by a sufficient margin, it becomes the reference for future self‑play.
2 Training methodology
The training process is designed to be stable and scalable, leveraging large‑scale distributed computing and careful handling of the reward structure.
2.1 Reward shaping and terminal conditions
AlphaZero uses a sparse reward: the only signal is the final outcome (win = +1, loss = –1, draw = 0). No intermediate rewards or heuristics are used. This forces the network to learn long‑term planning purely from the end‑game result. Terminal conditions are simply the standard rules of each game (checkmate, resignation, repetition, etc.).
2.2 Data generation and replay buffer
Self‑play games are generated in parallel on many worker threads. Each move’s search statistics (visit counts, value estimates) and the final outcome are stored. These data are fed into a large replay buffer, from which mini‑batches are sampled uniformly. The buffer retains the most recent 500,000 games to ensure diversity while avoiding outdated strategies.
2.3 Distributed training and synchronization
DeepMind used a distributed architecture with a central parameter server. Multiple self‑play workers run independently, periodically sending their generated data to the server. The server updates the network weights using asynchronous gradient descent and then pushes the updated weights to the workers. This parallelization enabled AlphaZero to train on 5,000 first‑generation Tensor Processing Units (TPUs) and achieve mastery in a few hours for chess and shogi, and a few days for Go.
3 Application to chess
AlphaZero was applied to chess in December 2017, quickly surpassing the strongest traditional engines at the time.
3.1 Performance against traditional engines (e.g., Stockfish)
In a 100‑game match against Stockfish 8 (rated 3400+ Elo), AlphaZero won 25 games, drew 72, and lost 3 – a decisive victory. Unlike Stockfish, which uses hand‑crafted evaluation functions and extensive opening books, AlphaZero learned all its chess knowledge from self‑play. The match used a fixed time control with no opening book or endgame tablebases for either side, highlighting the raw learning capability.
3.2 Opening and endgame strategies learned
AlphaZero independently rediscovered many classical openings (e.g., the Italian Game, King’s Indian), but also developed novel variations not found in standard human practice. In the endgame, it demonstrated deep understanding of piece activity and king safety, sometimes sacrificing material for long‑term positional pressure. Its play often resembled human intuition rather than the brute‑force calculations typical of traditional engines.
3.3 Impact on computer chess research
AlphaZero’s success spurred a shift in computer chess away from hand‑crafted evaluation functions and toward neural network‑guided search. Several open‑source projects (e.g., Leela Chess Zero, Stockfish NNUE) adopted similar techniques, leading to rapid improvement in engine strength. The result was a new generation of chess engines that combine the flexibility of deep learning with the speed of traditional search.
4 Application to Go
On the 19×19 Go board, AlphaZero demonstrated performance far beyond its predecessor AlphaGo, using a simpler and more general approach.
4.1 Comparison with earlier AlphaGo versions
The original AlphaGo (2016) used supervised learning from human games and separate policy and value networks, plus hand‑crafted features for tree search. AlphaZero replaced all of that with a single network trained solely from self‑play, and still defeated the previously strongest version, AlphaGo Zero (2017), which itself already surpassed all human and previous artificial intelligence (AI) players. AlphaZero achieved a 100–0 record against the original AlphaGo.
4.2 Novel move discoveries (e.g., shoulder hit)
Because AlphaZero was not biased by human games, it independently discovered many non‑traditional moves. Notably, it frequently played the “shoulder hit” (a contact play at the shoulder of an opponent’s stone) in ways that differed from standard professional practice. Some of these moves were later adopted by top Go players, who found them strategically sound. AlphaZero also revived ancient Joseki (corner patterns) that had fallen out of favour.
4.3 Influence on professional Go play
AlphaZero’s games were published and studied extensively by Go professionals. Its style emphasised flexible, territory‑oriented play with fierce fighting when advantageous. Many pros incorporated AlphaZero’s ideas into their own repertoires, particularly in the opening and in complex tactical positions. The algorithm contributed to a broader acceptance of AI‑inspired strategies in human Go.
5 Application to shogi
Shogi (Japanese chess) presents unique challenges due to the “drop” rule, where captured pieces can be reintroduced as the capturing player’s own pieces.
5.1 Adaptation to captured‑piece mechanics
AlphaZero’s architecture required minimal changes to handle shogi. The board representation included an additional section for each player’s captured pieces (the “hand”). The policy head predicted over both standard moves and drop moves (placing a captured piece on an empty square). The MCTS handled the larger branching factor (about 400 legal moves per position on average) without modification.
5.2 Evaluation against top shogi engines
In a 100‑game match against Elmo, the strongest shogi engine at the time (winner of the 2017 World Computer Shogi Championship), AlphaZero achieved 76 wins, 2 draws, and 22 losses. Like in chess, AlphaZero played without an opening book, while Elmo used one. The result confirmed that the same tabula rasa learning approach could master a game with fundamentally different mechanics.
5.3 Cultural and strategic implications
Shogi has a strong professional tradition in Japan. AlphaZero’s success prompted Japanese players and researchers to reconsider the game’s strategic foundations. Its style often favoured placing pieces in seemingly awkward positions that later proved highly effective, challenging human‑centric heuristics. The algorithm’s victories also accelerated the adoption of neural networks in Japanese shogi software.
6 Generalization and limitations
Although AlphaZero set new standards, it is not a universal game‑playing system and has clear constraints.
6.1 Transfer learning and multi‑game variants
AlphaZero was trained from scratch for each game separately. It does not transfer knowledge between games, even though the underlying neural architecture is identical. Subsequent work, such as DeepMind’s MuZero, extended the approach to handle multiple games without knowing the rules. Some open‑source projects (e.g., Polygames) attempted to create a single network playing multiple board games, but with mixed success.
6.2 Resource requirements and training time
The full AlphaZero system required thousands of TPUs for millions of self‑play games. A single training run for chess took about nine hours on Google’s infrastructure; for Go it took several days. This computational cost is far beyond what most individuals or small research groups can afford, limiting the algorithm’s accessibility.
6.3 Constraints in non‑deterministic or imperfect‑information games
AlphaZero operates under perfect information (players see the entire board) and deterministic rules (no randomness). It cannot be applied directly to games like poker (hidden cards) or backgammon (dice). Extending the approach to stochastic or hidden‑information domains remains an open challenge, though later algorithms (e.g., ReBeL) have taken steps in that direction.
7 Legacy and influence
AlphaZero’s impact extends beyond board games, influencing reinforcement learning research and public perception of artificial intelligence.
7.1 Subsequent research (e.g., MuZero, open‑source implementations)
AlphaZero directly inspired MuZero, a DeepMind algorithm that learns the environment model (transition dynamics) alongside the policy and value, enabling it to work in domains where the rules are not known in advance. Open‑source implementations such as Leela Zero (Go), Leela Chess Zero, and KataGo made AlphaZero‑style training accessible to the community, fostering rapid improvements in both chess and Go engines.
7.2 Broader applications in combinatorial optimization
Researchers have adapted AlphaZero’s self‑play framework to combinatorial optimisation problems, such as the travelling salesman problem, graph colouring, and SAT solving. These applications treat the optimisation task as a game where the algorithm learns to search efficiently. While results are promising, the approach is still less competitive than specialised solvers for many problem classes.
7.3 Public and media reception
AlphaZero was widely covered in the press as an example of artificial intelligence achieving superhuman skill without human guidance. The publication of its games in chess and Go attracted amateur and professional players alike, who marveled at its creative and sometimes “alien” style. The algorithm became a symbol of the potential for AI to learn complex tasks from first principles, and its name entered popular culture as a benchmark for general‑purpose game learning.