Browse Datasets

mC4 (African Subset)

Multilingual Colossal Cleaned Crawled Corpus

language-modeling pretraining Terabytes ODC-BY
Access Dataset Read Paper

Description

A multilingual variant of the C4 dataset, mC4 is a colossal, cleaned version of Common Crawl. Standard for training massive multilingual models like mT5, though requires significant filtering for low-resource languages.

Quick Start

Python 
from datasets import load_dataset

# Streaming is recommended due to size
dataset = load_dataset("allenai/c4", "sw", split="train", streaming=True)
print(next(iter(dataset)))

Project Ideas

Data Cleaning Pipeline

Create a cleaning pipeline to filter noise from the Somali subset of mC4

Advanced

Word2Vec Training

Train a Word2Vec model on the Swahili subset

Intermediate

Ethical Considerations

  • Contains toxic content, hate speech, and significant noise
  • African language representation is often lower quality than high-resource languages