Browse Datasets

WURA

Web-crawled Unsupervised Representation for Africa

language-modeling pretraining masked-language-modeling ~19GB Apache 2.0
Access Dataset Read Paper

Description

A large-scale, document-level dataset covering 16 African languages and 4 high-resource languages widely spoken in Africa. Created by auditing mC4 and crawling verified news sources to address quality issues in web-crawled data for low-resource languages.

Quick Start

Python 
from datasets import load_dataset

# Load Yoruba document-level data
dataset = load_dataset("castorini/wura", "yor", split="train")
print(dataset['text'][:5])

Project Ideas

Train a Small LM from Scratch

Train a unidirectional LSTM or Transformer from scratch on Yoruba text

Intermediate

Compare Data Quality

Compare n-gram perplexity between WURA (curated) and mC4 (raw) for Hausa

Intermediate

Predictive Keyboard

Build a predictive text keyboard for Nigerian Pidgin

Beginner

Ethical Considerations

  • While curated to remove noise, web-crawled data may still contain biases
  • Religious text inclusion can skew domain distribution