1 Architecture Design
ConvNeXt is a modernized convolutional neural network that systematically incorporates design elements from Vision Transformers while retaining pure convolution operations. The architecture is built upon a series of macro and micro design choices, each validated through ablation studies.
1.1 Macro Design
The macro architecture follows a ResNet-like staged structure but introduces key modifications at the global level, including a patchify stem and adjusted stage ratios.
1.1.1 Stem: Patchify Layer with Overlapping Convolutions
The initial stem of ConvNeXt replaces the traditional ResNet-style 7×7 convolution with stride 2 (followed by max pooling) with a non-overlapping 4×4 convolution with stride 4. This operation "patchifies" the input image into non-overlapping patches of size 4×4, directly mirroring the patch embedding used in Vision Transformers. Each patch becomes a token in the subsequent convolutional blocks. The stem reduces the spatial resolution from 224×224 to 56×56 and increases the channel dimension to 96 (for ConvNeXt-T).
1.1.2 Stage Composition: 4 Stages (ResNet-like Configuration)
ConvNeXt contains four stages, where each stage consists of a series of identical blocks. The stage configuration is inspired by ResNet, with block counts varying by model size.
1.1.2.1 Stage Ratio (e.g., 3:3:9:3 for ConvNeXt-T)
The stage ratio – the number of blocks in each stage – is one of the key modifications. For the Tiny variant (ConvNeXt-T), the ratio is 3:3:9:3 (total 18 blocks). This differs from ResNet's typical 3:4:6:3 (ResNet-50) or 3:4:23:3 (ResNet-101). The increase in stage 3 blocks (from 6 to 9) is motivated by the observation that Vision Transformers concentrate processing in the middle layers, and the adjustment improves performance on ImageNet without significantly increasing FLOPs.
1.2 Micro Design
The micro design of individual blocks introduces several transformer-inspired changes while maintaining the convolution-based computation.
1.2.1 Inverted Bottleneck (Expand-Reduce Pattern)
Each ConvNeXt block uses an inverted bottleneck structure, common in MobileNetV2 and later adopted in Transformers. In a regular bottleneck, the channel dimension is first reduced (1×1 conv), then processed (3×3 conv), then expanded; in the inverted version, the channel dimension is first expanded (1×1 conv, e.g., 4×), then processed by a depthwise convolution, and finally reduced by a 1×1 conv with a residual connection. This design increases the representational capacity in the middle layer while keeping the skip connection narrow.
1.2.2 Depthwise Convolution with Large Kernel Sizes (7×7)
The spatial mixing is performed by a 7×7 depthwise convolution, applied after the first 1×1 expansion. Depthwise convolution is a per-channel operation that drastically reduces computational cost compared to standard convolution, allowing the use of larger kernel sizes. The choice of 7×7 is motivated by the self-attention window size in Swin Transformers (7×7). Larger kernels (e.g., 9×9 or 11×11) yield marginal gains at greater cost; 7×7 is a balanced trade-off.
1.2.3 Layer Normalization Instead of Batch Normalization
ConvNeXt replaces batch normalization (BN) with layer normalization (LN). In each block, layer normalization is applied after each of the three 1×1 convolutions (i.e., before the depthwise conv and after each pointwise conv). This change aligns with transformer practice and avoids the batch-dependent training behavior of BN. Layer normalization is computationally efficient and works well with small batch sizes, simplifying distributed training.
1.2.4 Activation Functions (GELU) and Downsampling Strategy
The activation function used throughout ConvNeXt is Gaussian Error Linear Unit (GELU), chosen for its smooth behavior and slight improvement over ReLU. Downsampling between stages is performed by a separate 2×2 convolution with stride 2, applied on the output of the last block of each stage (except the final stage). This differs from ResNet's stride-2 conv in the first block of a stage. The downsampling layer increases the channel dimension (e.g., from 96 to 192 between stages 1 and 2) and halves the spatial resolution.
1.3 Training Recipe Improvements
Beyond architectural changes, ConvNeXt adopts a modernized training recipe that significantly boosts performance.
1.3.1 Data Augmentation (Mixup, CutMix, RandAugment)
The training pipeline incorporates several strong data augmentation techniques: Mixup (mixing two images linearly), CutMix (cutting and pasting patches between images), and RandAugment (randomly applying a set of augmentation operations with controlled magnitude). These augmentations increase diversity and reduce overfitting, particularly on large-scale datasets.
1.3.2 Optimizer (AdamW) and Learning Rate Schedule
ConvNeXt is trained using the AdamW optimizer (Adam with decoupled weight decay) instead of traditional SGD. The learning rate follows a cosine decay schedule with a warm-up phase (typically 20 epochs). The maximum learning rate, weight decay, and momentum parameters are tuned per model size. This optimizer combination is standard in transformer training and yields faster convergence and better final accuracy.
1.3.3 Stochastic Depth and Label Smoothing
Stochastic depth (dropping entire blocks at each stage during training with a linear decay probability) is applied to all model variants. Drop rates range from 0.1 (Tiny) to 0.4 (XL). Label smoothing (with epsilon 0.1) is also used as a regularization technique, reducing overconfidence in predictions. Both techniques are common in state-of-the-art vision models.
2 Model Variants
ConvNeXt is available in five sizes, each scaling the channel widths and block counts while maintaining the same architectural principles.
2.1 ConvNeXt-T (Tiny)
Base channel width: 96. Stage channel dimensions: [96, 192, 384, 768]. Block counts: [3, 3, 9, 3]. Total parameters: approximately 28 million. FLOPs: ~4.5 G (on 224×224 input). Designed as a lightweight model suitable for resource-constrained scenarios.
2.2 ConvNeXt-S (Small)
Base channel width: 96. Stage channel dimensions: [96, 192, 384, 768]. Block counts: [3, 3, 27, 3]. Parameters: ~50 million. FLOPs: ~8.7 G. Offers a balanced trade-off between performance and computational cost.
2.3 ConvNeXt-B (Base)
Base channel width: 128. Stage channel dimensions: [128, 256, 512, 1024]. Block counts: [3, 3, 27, 3]. Parameters: ~89 million. FLOPs: ~15.4 G. Comparable to ResNet-152 and Swin-B in scale.
2.4 ConvNeXt-L (Large)
Base channel width: 192. Stage channel dimensions: [192, 384, 768, 1536]. Block counts: [3, 3, 27, 3]. Parameters: ~198 million. FLOPs: ~34.5 G. Targets high-performance demands on large datasets and downstream tasks.
2.5 ConvNeXt-XL (Extra Large)
Base channel width: 256. Stage channel dimensions: [256, 512, 1024, 2048]. Block counts: [3, 3, 27, 3]. Parameters: ~350 million. FLOPs: ~60.3 G. The largest variant, delivering top-tier results on benchmarks where compute is not a limiting factor.
3 Performance and Benchmarks
ConvNeXt was evaluated on standard computer vision benchmarks, demonstrating state-of-the-art or competitive performance at each model size.
3.1 ImageNet-1K Classification
On the ImageNet-1K dataset (1.28 million training images, 1,000 classes), ConvNeXt models achieve high top-1 accuracy.
3.1.1 Top-1 Accuracy and FLOPs Comparison
- ConvNeXt-T: 82.1% top-1 accuracy with 4.5 GFLOPs (single crop 224×224). Outperforms ResNet-50 (76.1% / 4.1 GFLOPs) and EfficientNet-B4 (82.6% / 9.6 GFLOPs) with fewer operations.
- ConvNeXt-S: 83.1% / 8.7 GFLOPs.
- ConvNeXt-B: 83.8% / 15.4 GFLOPs. Slightly below Swin-B (83.8% / 15.4 GFLOPs) but similar.
- ConvNeXt-L: 84.3% / 34.5 GFLOPs.
- ConvNeXt-XL: 84.6% / 60.3 GFLOPs. These results demonstrate that pure ConvNets can match or exceed transformer-based models on a standard classification benchmark.
3.2 Downstream Tasks
ConvNeXt backbones were transferred to object detection, instance segmentation, and semantic segmentation tasks, often serving as drop-in replacements for ResNet or Swin.
3.2.1 COCO Object Detection and Instance Segmentation
Using a Mask R-CNN framework (with ConvNeXt backbone and FPN), performance on the COCO dataset (2017) was evaluated. ConvNeXt-L achieved approximately 54.0 AP (box) and 46.9 AP (mask), outperforming ResNeXt-101 and competing with Swin-L. The improvement was attributed to the larger receptive field from 7×7 depthwise convolutions and the modern training recipe.
3.2.2 ADE20K Semantic Segmentation
For semantic segmentation, ConvNeXt backbones were integrated into UperNet. ConvNeXt-L achieved 52.6 mIoU on the ADE20K validation set (with single-scale testing), surpassing Swin-L (52.1 mIoU). This indicates that convolutional architectures can excel in dense prediction tasks.
3.3 Ablation Studies
A series of controlled experiments isolated the contribution of each design component.
3.3.1 Impact of Each Design Component
Starting from a ResNet-50 baseline (79.0% top-1 on ImageNet), the authors sequentially added modifications:
- Training recipe (AdamW, augmentation, etc.): +2.6% → 81.6%
- Macro design (stem, stage ratio): +0.5% → 82.1%
- Micro design (inverted bottleneck, depthwise 7×7, LN, GELU): +2.5% → 84.6%
- Additional scaling and refinements: final 82.0% (Tiny) to 84.6% (XL). Each step contributed positively, with micro design providing the largest gain.
3.3.2 Effect of Kernel Size and Normalization
Replacing 7×7 depthwise convolution with smaller kernels (e.g., 3×3) reduced accuracy by ~0.5%. Larger kernels (9×9, 11×11) gave marginal improvements (0.1–0.2%) at noticeably higher FLOPs. Replacing layer normalization with batch normalization dropped accuracy by ~0.3% and introduced training instability at small batch sizes. These ablations validated the design choices.
4 Relationship to Other Architectures
ConvNeXt bridges the gap between classical ConvNets and modern Vision Transformers by re-engineering convolutional blocks to incorporate transformer principles while retaining convolution's efficiency.
4.1 Comparison with ResNet
Compared to ResNet-50/101, ConvNeXt uses depthwise separable convolutions (large kernel) instead of standard 3×3 convolutions, layer normalization instead of batch normalization, inverted bottlenecks instead of standard bottlenecks, and a modified stage ratio. ConvNeXt achieves significantly higher accuracy at similar or lower FLOPs per parameter count. The training recipe also differs, as ResNet is typically trained with SGD and weaker augmentations.
4.2 Comparison with Vision Transformer (ViT) and Swin Transformer
ViT processes images as sequences of patches using global self-attention, requiring large-scale pretraining (e.g., JFT-300M). ConvNeXt, like Swin, operates on local windows, but uses convolution instead of attention. ConvNeXt matches Swin's performance on ImageNet and downstream tasks without relying on attention mechanisms, showing that the inductive biases of convolutions (locality, translation equivariance) are not limiting when modernized appropriately. Pure ViT (without locality) still requires more data to match ConvNeXt at smaller scales.
4.3 Integration into Modern Frameworks (e.g., MMClassification, Detectron2)
ConvNeXt has been integrated into open-source toolkits such as MMClassification (now part of MMSelfSup), Detectron2, and the official timm library. Users can instantiate pretrained checkpoints with a single function call and substitute ConvNeXt for existing backbones (e.g., ResNet, Swin) in task-specific pipelines (detection, segmentation, etc.). The model's compatibility with these frameworks accelerated adoption.
5 Applications and Extensions
Beyond standard benchmarks, ConvNeXt has been applied and extended to diverse domains.
5.1 Image Recognition and Transfer Learning
ConvNeXt is widely used as a backbone for transfer learning on fine-grained classification tasks (e.g., CIFAR-100, Stanford Cars, Food-101). Its strong feature representation and moderate computational cost make it suitable for both academia and industry.
5.2 Video Understanding (Video ConvNeXt)
The ConvNeXt design was adapted to video by incorporating 3D convolution. Video ConvNeXt replaces the 2D depthwise convolution with a 3D depthwise convolution (e.g., 3×7×7), adding temporal modeling. This variant achieved competitive results on action recognition benchmarks (e.g., Kinetics-400) and video object detection, outperforming 3D ResNet and offering an alternative to transformer-based video models.
5.3 Medical Image Analysis
ConvNeXt has been applied to medical imaging tasks such as segmentation of CT and MRI scans, tumor detection, and histopathology classification. Its ability to capture both global context (via large kernels) and local details (via depthwise conv) aligns well with the multi-scale nature of medical images. Pretrained models transferred from ImageNet generalize effectively to medical datasets with limited annotations.
5.4 Efficient Deployment on Edge Devices
Despite its modernized design, ConvNeXt retains the fixed computational graph of a ConvNet, making it amenable to optimization (e.g., ONNX export, quantization, TensorRT). The Tiny variant is particularly suited for mobile devices, achieving high accuracy at low latency. Researchers have further compressed ConvNeXt with techniques like knowledge distillation and channel pruning for edge deployment.
6 Future Directions and Impact
ConvNeXt has influenced subsequent research in both convolutional and hybrid architectures.
6.1 Influence on Hybrid Models
The success of ConvNeXt encouraged the development of hybrid models that combine convolution and attention. Examples include ConvNeXtV2 (adding global response normalization), ConvNeXt with attention (e.g., ConvNeXt + cross-attention), and architectures like FastViT that use ConvNeXt-like blocks as the convolutional component in a hybrid pipeline. ConvNeXt also provided a strong baseline for architecture search.
6.2 Potential for Self-Supervised Learning (SimCLR, MAE)
ConvNeXt backbones are effective for self-supervised learning. Using contrastive learning frameworks (SimCLR, MoCo) or masked image modeling (MAE), pre-trained ConvNeXt encoders achieve high linear probing accuracy on ImageNet. MAE with a ConvNeXt decoder (or full ConvNeXt encoder) has been explored, demonstrating that the architecture benefits from generative pre-training without attention.
6.3 Open Source Implementations and Community Adoption
The official ConvNeXt implementation, released under the MIT license, has been widely cloned and forked. Community implementations exist in PyTorch, TensorFlow, JAX, and ONNX. The model is a default backbone in popular repositories (timm, OpenMMLab) and has been used in many research projects. Its straightforward architecture and strong performance have made it a staple in modern computer vision.