Zero‑shot classification is a machine learning paradigm that enables a model to classify instances into categories not seen during training, without requiring any labeled examples of those unseen classes. Instead, it leverages auxiliary information—such as semantic attributes, textual descriptions, or word embeddings—to bridge the gap between seen and unseen classes. This approach is particularly valuable in information technology domains where the set of possible classes is large, dynamic, or expensive to annotate, such as in natural language processing, computer vision, and recommendation systems.
1 Introduction
1.1 Definition and core idea
Zero-shot classification refers to the task of assigning class labels to test instances whose categories were never present in the training set. The core idea is to exploit a shared semantic space that encodes both seen and unseen classes. During training, the model learns a mapping from input features (e.g., image pixels, text tokens) to this semantic space using only seen classes. At inference, unseen class descriptions (attributes, word vectors, or natural language) are projected into the same space, and the class whose representation best matches the input is chosen. This enables generalization to novel categories without any additional labeled examples.
1.2 Motivation and significance in information technology
In many IT applications, the universe of possible categories is vast and frequently changing. For instance, e‑commerce platforms must classify thousands of new products daily, news aggregators need to adapt to emerging topics, and medical imaging systems encounter previously unseen diseases. Traditional supervised learning would require manual annotation of every new category, which is costly and time‑consuming. Zero‑shot classification alleviates this bottleneck by reusing existing knowledge, thus reducing annotation effort, enabling rapid deployment, and supporting scalable, dynamic classification pipelines.
2 Technical foundations
2.1 Semantic embedding spaces
Semantic embedding spaces are numerical representations that capture the meaning or attributes of class labels. They serve as the bridge between seen and unseen classes. The quality of these embeddings directly influences zero‑shot performance.
2.1.1 Attribute‑based representations
Attributes are human‑defined, high‑level properties shared across classes, such as “has stripes”, “is furry”, or “can fly”. Each class is described by a binary or continuous attribute vector. During training, the model learns to predict these attributes from input instances; at test time, an unseen class is recognized by comparing the predicted attribute vector to the pre‑stored attribute vectors of unseen classes. Attribute‑based methods are interpretable but require manual engineering.
2.1.2 Text‑based embeddings (e.g., Word2Vec, GloVe)
Text‑based embeddings are distributed representations of words (like Word2Vec, GloVe, or fastText) that capture semantic similarities from large text corpora. For zero‑shot classification, the class name or a short description is converted into a fixed‑dimensional vector. These vectors often encode fine‑grained relationships (e.g., “dog” is closer to “cat” than to “car”) and can be obtained automatically, making them widely used in combination with deep learning.
2.2 Learning paradigms
2.2.1 Inductive zero‑shot learning
Inductive zero‑shot learning assumes that at test time only unlabeled instances from unseen classes are available, without any access to the unlabeled data from unseen classes during training. The model is trained solely on seen classes and then directly applied to unseen ones. This is the most common setting, but it suffers from a domain shift because the model never observes the distribution of unseen class features.
2.2.2 Transductive zero‑shot learning
Transductive zero‑shot learning relaxes the inductive assumption by allowing the model to use unlabeled test data (from unseen classes) during the inference phase, although still without labeled examples. The unlabeled data help to adapt the learned mapping to the target domain, reducing the bias toward seen classes and improving accuracy.
2.3 Compatibility functions
A compatibility function measures how well an input instance matches a class label in the semantic space. It is a learned scoring mechanism that assigns higher values to correct (instance, class) pairs.
2.3.1 Linear scoring functions
A linear scoring function computes the dot product (or bilinear form) between the input embedding and the class semantic embedding. The parameters are a weight matrix or vector learned via a ranking objective (e.g., hinge loss). Linear models are efficient and provide a simple baseline, but their expressive power is limited.
2.3.2 Nonlinear (neural) compatibility
Nonlinear compatibility functions use neural networks (e.g., multilayer perceptrons or deep bilinear projections) to map input and class embeddings into a common space where similarity is computed. These models can capture complex interactions and generally outperform linear functions, at the cost of increased training complexity and risk of overfitting.
3 Major techniques
3.1 Attribute‑based methods
3.1.1 Direct attribute prediction
In direct attribute prediction, the model is trained to predict the attribute vector for each input instance (e.g., an image). During inference, the predicted attribute vector is compared to the attribute vectors of unseen classes using a distance metric (e.g., cosine distance). The class with the closest attribute vector is selected. This method is straightforward but relies on precise attribute definitions and often suffers from noisy predictions.
3.1.2 Bilinear models
Bilinear models learn a compatibility function in the form of \( f(x, y) = x^T W \phi(y) \), where \(x\) is the input feature, \(\phi(y)\) is the attribute or embedding of class \(y\), and \(W\) is a learned bilinear matrix. This formulation captures interactions between input features and class attributes more flexibly than a simple dot product. Bilinear models have been successful in early zero‑shot image classification.
3.2 Embedding‑based methods
3.2.1 Visual‑semantic mapping
These methods learn a projection from visual features (e.g., CNN activations) to a semantic embedding space (e.g., word2vec). The goal is that the projected visual features of an instance should be close to the semantic embedding of its true class.
3.2.1.1 Deep visual‑semantic embedding (DeViSE)
DeViSE (2013) was a pioneering work that trained a deep neural network to map image features into a text embedding space by minimizing a ranking loss. It used a pre‑trained language model (Word2Vec) to define the semantic space and a pre‑trained image classifier (e.g., AlexNet) for visual features. At test time, the model ranks unseen classes by the cosine similarity between the predicted embedding and each class vector.
3.2.2 Multimodal embedding alignment
These approaches treat zero‑shot classification as a cross‑modal retrieval problem. They align visual and textual representations by learning a joint embedding space, often using contrastive losses.
3.2.2.1 Cross‑modal retrieval style
In this style, the model is trained on paired (image, text description) data from seen classes. A dual‑encoder architecture (e.g., a visual encoder and a text encoder) is optimized so that matching pairs have similar embeddings. At inference, a text description of an unseen class (e.g., “a small brown dog with floppy ears”) is embedded, and the input image is classified by retrieving the most similar text embedding.
3.3 Generative approaches
Generative models attempt to synthesize feature representations for unseen classes, thereby converting zero‑shot learning into a standard supervised learning problem on generated data.
3.3.1 Variational autoencoders (VAEs)
VAEs learn a probabilistic latent space that generates class‑conditional features. During training, seen class features are encoded to latent variables and reconstructed. At test time, a VAE conditioned on the semantic embedding of an unseen class generates synthetic features for that class. A classifier is then trained on real seen features and synthetic unseen features. This reduces the domain shift problem.
3.3.2 Generative adversarial networks (GANs)
GAN‑based zero‑shot methods (e.g., f‑GAN, CVAE‑GAN) use a generator that produces realistic feature vectors from random noise conditioned on class embeddings. A discriminator distinguishes real features (from seen classes) from generated ones, and the generator learns to produce features that fool the discriminator. Once trained, the generator creates features for unseen classes, and a classifier is trained on the augmented dataset.
3.4 Large language model based methods
Recent advances in large language models (LLMs) have enabled new zero‑shot classification paradigms that bypass the need for explicit embedding spaces.
3.4.1 Prompt‑based zero‑shot classification
In prompt‑based methods, an input instance is converted into a natural language prompt (e.g., “The sentiment of this review is ___.”). A pre‑trained LLM (e.g., GPT‑3, BERT) then fills in the blank by ranking possible label words (e.g., “positive”, “negative”). This approach requires no task‑specific training, only careful prompt engineering and a vocabulary of candidate labels.
3.4.2 LLM‑as‑classifier (e.g., zero‑shot with instruction tuning)
Instruction‑tuned LLMs (e.g., InstructGPT, T0) can directly perform classification by following a natural language instruction. The model receives a query like “Classify the following tweet as sports, politics, or technology: [tweet text]”. The LLM outputs the predicted class name. This method is flexible and can handle an arbitrary number of classes by simply listing them in the prompt, but it relies on the LLM’s ability to generalize and may be computationally expensive.
4 Applications in information technology
4.1 Text classification
4.1.1 Sentiment analysis on emerging domains
As new products, services, or social media trends appear, sentiment classifiers must adapt to domain‑specific language (e.g., slang, product names). Zero‑shot classification allows a model trained on a few review domains to be applied to an entirely new product category without retraining, by using semantic descriptions of sentiment classes (e.g., “positive” vs. “negative”) and domain‑agnostic word embeddings.
4.1.2 Topic labeling with evolving taxonomies
News aggregators and content management systems often need to assign topics that are updated or expanded over time (e.g., “COVID‑19”, “metaverse”). Zero‑shot methods can label documents with new topics by providing a short textual definition of each topic (e.g., “articles about the second life of virtual worlds”). This enables topic models to remain current without manual annotation of every new category.
4.2 Image classification
4.2.1 Natural image recognition
In general object recognition, zero‑shot learning allows a system to recognize rare or novel object categories (e.g., specific animal species, brand logos) that were not present in the training set. For instance, a model trained on 1,000 common objects can be extended to recognize a new breed of dog if its name and attribute description (e.g., “large, black, curly tail”) are provided.
4.2.2 Medical image diagnosis (unseen diseases)
Medical imaging datasets often lack examples of rare diseases. Zero‑shot classification can assist in diagnosing these conditions by leveraging clinical descriptions (e.g., “shows ground‑glass opacities with reticulation”) as auxiliary information. A model trained on common pathological findings can be applied to identify an unseen disease based solely on its textual description, aiding in early detection and screening.
4.3 Multimodal tasks
4.3.1 Visual question answering with novel objects
In visual question answering (VQA), the system must answer questions about images. Traditional VQA models struggle with questions about objects not seen during training. Zero‑shot VQA uses a semantic embedding of the object’s description to map the novel concept into the existing representation space, allowing the model to answer queries like “What color is the zib?” after only seen objects.
4.3.2 Video action recognition for new activities
Recognizing new human activities in videos (e.g., “parkour”, “whip dancing”) is challenging because collecting labeled clips is labor‑intensive. Zero‑shot action recognition uses attributes (e.g., “jumping”, “spinning”, “using hands”) or textual descriptions of the action to classify unseen activities. This enables surveillance, sports analytics, and user interface gesture recognition systems to adapt to new motions quickly.
4.4 Recommendation systems
4.4.1 New item cold start
Recommendation systems face a cold‑start problem when a new item (e.g., a new movie or product) lacks any user interaction history. Zero‑shot techniques can handle this by representing the new item via its metadata (e.g., title, category, tags) and mapping it to a semantic space shared with existing items. The system then recommends the new item to users whose profile embeddings are similar to that semantic representation.
4.4.2 Cross‑domain recommendation
Users often have preferences across different domains (e.g., books and movies). A zero‑shot cross‑domain recommender learns a common embedding space for items from multiple domains using auxiliary textual descriptions. When a user interacts with items in one domain, the system can recommend items from a previously unseen domain (e.g., a music album) by matching the user’s profile to the semantic embedding of the unseen item.
5 Evaluation and metrics
5.1 Top‑k accuracy (e.g., ZSL top‑1)
Top‑k accuracy measures whether the correct unseen class appears among the top‑k predictions. The most common metric is top‑1 accuracy, which checks if the predicted class with the highest compatibility score matches the ground truth. For zero‑shot, this is computed only on test instances from unseen classes.
5.2 Harmonic mean or area under the curve
Because zero‑shot models often trade off between seen‑class and unseen‑class accuracy, a single metric like harmonic mean of seen and unseen accuracies (H) is used: \( H = 2 \cdot (\text{acc}_{\text{seen}} \cdot \text{acc}_{\text{unseen}}) / (\text{acc}_{\text{seen}} + \text{acc}_{\text{unseen}}) \). Alternatively, area under the curve (AUC) plots the trade‑off across various thresholds. These metrics provide a balanced view of model performance.
5.3 Generalized zero‑shot evaluation
In generalized zero‑shot learning (GZSL), the test set includes both seen and unseen classes, and the model must classify instances from either set. Evaluation reports the average per‑class accuracy for seen and unseen classes separately, often together with the harmonic mean. This setting better reflects real‑world scenarios where seen and unseen categories coexist.
6 Challenges and open issues
6.1 Domain shift and bias toward seen classes
Models trained only on seen classes tend to predict seen classes with high confidence, even when the input belongs to an unseen class. This bias arises because the model has never observed the feature distribution of unseen classes. Transductive methods and generative approaches partially mitigate this, but the problem remains a core difficulty.
6.2 Hubness problem in high‑dimensional embeddings
In high‑dimensional semantic spaces, some points (hubs) become nearest neighbors to many other points, causing many unseen classes to be assigned to the same label. This hubness problem degrades zero‑shot accuracy, especially with simple nearest‑neighbor classifiers. Techniques such as centering, normalization, and reranking have been proposed, but no universal solution exists.
6.3 Label noise in auxiliary information
Auxiliary information (attributes, text embeddings) is often collected from noisy sources—e.g., crowd‑sourced attributes or automatically generated text. Incorrect or ambiguous descriptions lead to incorrect label‑semantic associations, harming classification. Robustness to such noise is an active research area.
6.4 Semantic gap between visual and textual spaces
Visual features (e.g., pixels) and textual embeddings (e.g., word vectors) reside in different statistical spaces. Even after alignment, the mapping may not capture fine‑grained nuances (e.g., the difference between “red apple” and “tomato”). Bridging this gap requires better multimodal representations and more expressive compatibility functions.
7 Future directions
7.1 Integration with few‑shot and continual learning
Zero‑shot classification is complementary to few‑shot learning (which uses a few labeled examples of unseen classes) and continual learning (which adapts to new classes over time). Future systems may combine these paradigms: start with zero‑shot, then refine with a few examples, and finally incrementally incorporate new classes without forgetting old ones.
7.2 Self‑supervised auxiliary knowledge acquisition
Manually designing attributes or collecting class descriptions is labor‑intensive. Future methods may use self‑supervised learning to automatically derive auxiliary knowledge from large unlabeled corpora—for instance, by mining visual‑textual relationships from web‑scale data or by leveraging multimodal pre‑training (e.g., CLIP, ALIGN). This would reduce human effort and expand the scope of zero‑shot classification.
7.3 Real‑time zero‑shot systems
Deploying zero‑shot models in latency‑sensitive applications (e.g., real‑time video surveillance, online content moderation) requires efficient inference. Future research will focus on lightweight architectures, approximate nearest‑neighbor search, and on‑device deployment. Prompt‑based LLM methods, while powerful, are currently too slow for many real‑time contexts; optimizing their inference is a key direction.