Natural Language Inference
Natural Language Inference (NLI), also known as Textual Entailment, is a task concerned with determining whether a given text statement (the hypothesis) follows logically from another (the premise). The goal is to classify a hypothesis as:
- Contradicting the premise
- Supporting the premise
- Neither supporting nor contradicting the premise
Core Process
- Premise and Hypothesis Encoding: Convert text premises and hypotheses into vectors.
- Similarity Computation: Calculate distances between each premise and its candidate hypotheses.
- Evaluation: Find the best obtainable F1, Accuracy and Average Precision (AP) values, when iterating over similarity thresholds.
Data Schema Specifications
| Column | Type | Description | Required |
|---|---|---|---|
premise |
string | Original text statement | ✓ |
hypothesis |
string | Statement to be compared with the premise for entailment | ✓ |
label |
string | Label denoting premise contradiction (0), support (1) or neither (2) | ✓ |
Relevant Metrics
| Metric | Description |
|---|---|
| F1 | Maximum obtainable F1 score across all decision thresholds |
| Accuracy | Maximum obtainable accuracy across all decision thresholds |
| AP | Average precision (only relevant for models which return a score) |
Supported Models
Only HuggingFace embedding-based models compatible with SentenceTransformer (e.g., SentenceTransformer, BERT, RoBERTa) are supported.