Datasets

language-modeling pretraining ~19GB Apache 2.0

Amharic Hausa Igbo Kinyarwanda +7 more

image-classification computer-vision 27,558 images (~339MB) Public Domain (NIH)

English (Metadata)

language-modeling pretraining Terabytes ODC-BY

Afrikaans Amharic Hausa Igbo +8 more

topic-classification language-modeling ~20,000 articles CC-BY-4.0

Amharic Hausa Igbo Luganda +5 more

object-detection image-classification 16,552 images (~1.5GB) CC0 1.0

English (Metadata)

language-modeling asr 1,800 hours of speech CC BY-NC-SA 4.0

Igbo Hausa Yoruba

machine-translation alignment ~5k-30k sentences per language CC-BY-4.0-NC

Bambara Ewe Fon Hausa +5 more

tokenization pretraining Massive (varies by language) Common Crawl Terms

Amharic Hausa Igbo Somali +4 more

translation-evaluation embeddings 3,001 sentences per language CC-BY-SA 4.0

20+ African languages including Dinka, Nuer, Kanuri

pos-tagging dependency-parsing Small, high-quality treebanks CC-BY-SA 4.0

Amharic Bambara Coptic Hausa +3 more

named-entity-recognition sequence-labeling ~9,000-10,000 sentences per language CC-BY-4.0-NC

Bambara Ewe Fon Hausa +12 more

sentiment-analysis text-classification 110,000+ tweets CC-BY-4.0

Amharic Hausa Igbo Kinyarwanda +5 more

hate-speech-detection text-classification ~45,000 examples Apache 2.0

Algerian Arabic Amharic Hausa Igbo +8 more

sentiment-analysis text-classification ~3,000 sentences MIT

Tunisian Arabizi

question-answering cross-lingual 12,000+ QA pairs CC-BY-4.0-NC

Bemba Fon Hausa Igbo +6 more

summarization text-generation 6k-7k pairs per language CC-BY-NC-SA 4.0

Amharic Hausa Igbo Kirundi +6 more

reading-comprehension multiple-choice 900 questions per language CC-BY-SA 4.0

Bambara Wolof Yoruba Zulu +1 more

question-answering 7,526 QA pairs CC-BY 4.0

Swahili

instruction-tuning fine-tuning Millions of tokens CC-BY-4.0

Hausa Igbo Yoruba Swahili +2 more

instruction-tuning multi-task 62M Swahili, 6M Yoruba samples CC BY-NC 4.0

Swahili Yoruba IsiXhosa Hausa +1 more

medical-qa domain-adaptation 15,000+ QA pairs CC BY-NC-SA 4.0

English (African context)

instruction-following multilingual 204k human annotations, millions translated Apache 2.0

65 languages including African LRLs

bias-evaluation stereotypes ~5,000 pairs Open Source

English (African context) French

rlhf dialogue 161k messages Apache 2.0

35 languages (Multilingual)

topic-classification benchmarking Based on FLORES-200 CC-BY-SA 4.0

200+ including many African LRLs

asr speech-recognition 200 hours CC BY-NC-SA 4.0

English (120 African accents)

classification tabular ~33,000 rows Public (Zindi Terms)

English (Metadata)

time-series forecasting Daily records (1980-2023) Open

Numeric

audio-classification healthcare 6.5 hours, 786 infants CC BY-NC-ND 4.0

Infant Cry (Audio)

regression yield-prediction 500,000+ observations Open Access

Numeric

sign-language-recognition computer-vision 20,000 videos Open Access (Research)

Kenyan Sign Language

data-analysis regression Tabular (Country-level) Apache 2.0

Numeric

language-modeling pretraining Millions of sentences CC0 1.0 (Metadata)

Amharic Somali Yoruba Malagasy +2 more

machine-translation alignment Millions of sentence pairs Varies by sub-corpus (often open)

Xhosa Zulu Tswana Sotho +1 more

reading-comprehension multiple-choice ~358 questions (Yoruba subset) CC-BY-NC 4.0

Hausa Igbo Yoruba

benchmarking evaluation Multiple evaluation sets CC-BY-4.0

Hausa Igbo Yoruba Swahili +2 more

semantic-segmentation data-fusion ~10GB CC BY 4.0

English (Metadata)

regression yield-prediction <100MB CC BY 4.0

English (Metadata)

ocr computer-vision <1K records CC-BY 4.0

English (Nigeria)

image-classification computer-aided-diagnosis 3,356 images CC BY-NC 4.0

English (Metadata)

asr speech-recognition 4.2 hours audio CC BY-NC-SA 4.0

English (Nigerian accent)

image-classification object-detection ~5TB full (use a subset) CDLA-Permissive-1.0

English (Metadata)

asr speech-recognition 3,000+ hours CC BY 4.0

isiZulu isiXhosa Sesotho Setswana +3 more

machine-translation pos-tagging 5.6M words / 177 hours CC BY 4.0

Swahili Dholuo Luhya

asr speech-recognition Swahili ~20GB; Hausa smaller CC0 1.0

Swahili Hausa

Dataset Catalog