Browse Datasets

OPUS Parallel Corpora

Open Parallel Corpus Collection

machine-translation alignment embeddings Millions of sentence pairs Varies by sub-corpus (often open)
Access Dataset

Description

A collection of translated texts from the web. For African languages, the Ubuntu (localization files) and Bible subsets are particularly rich sources of parallel data for training alignment and embeddings.

Quick Start

Python 
from datasets import load_dataset

# Load Ubuntu localization corpus
dataset = load_dataset("opus_ubuntu", "en-xh", split="train")
print(dataset['translation'][:3])

Project Ideas

Word2Vec from Bible

Train static word embeddings (Word2Vec) on the Bible corpus in Xhosa

Intermediate

Dictionary Mapping

Create a dictionary mapping between English and Tswana using alignment tools

Intermediate

Ethical Considerations

  • Religious texts have specific domain bias and archaic vocabulary
  • Ubuntu corpus is technical domain-specific