Datasets Q&A

SSL Errors When Loading Datasets

SSL errors may occur when downloading datasets from Hugging Face.

Root Cause

The root cause of these errors is that the Requests library serves as the HTTPS backend of the Datasets Library. You need to configure Requests to use system certificates.

Error Examples

You may encounter errors such as:

Solutions

Solution 1: Set Environment Variable (Recommended)

You can set the REQUESTS_CA_BUNDLE environment variable using either method:

Option A: Via Bash/Shell

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

(Optional) Add this to your .bashrc or .zshrc for permanent configuration.

Option B: Via .env File

add to .env file

# for Linux
REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
# for Mac, #!todo need to find a Mac user to validate
REQUESTS_CA_BUNDLE=

Solution 2: Disable SSL Verification (Not Recommended)

For development environments only, you can disable SSL verification by using the parameter in data loader:

from evals_hub.data_loader import get_dataset_loader

dataset_loader = get_dataset_loader(
    dataset_name="nfcorpus_gold",
    split="test",
    ssl_verify=False  # Not recommended for production
)

⚠️ Warning: This method is not recommended as it disables security verification and should only be used in controlled development environments.