Represent Your Language Data
Data is the fuel of AI. This course focuses on the critical pre-processing stage, teaching you how to convert human language into a format that machines can process mathematically. Learn about tokenization, embeddings, and responsible data documentation.
Key Topics
- Tokenization: Breaking text into smaller units (tokens)
- Embeddings: Representing tokens as vectors in high-dimensional space
- Vector and matrix operations relevant to NLP
- Data Cards: A framework for transparent dataset documentation
Learning Outcomes
- Investigate tools for structuring text data
- Analyze how meaning is represented geometrically in language models
- Think critically about bias introduced during data preparation
- Implement tokenizers and create word embeddings
- Design datasets with focus on transparency and accountability
Practical Projects
- Hands-on tokenization and embedding generation
- Ethics exercise: Designing a dataset to mitigate algorithmic bias
- Train a BPE tokenizer on African languages and compare vocabulary sizes
Prerequisites
- Completion of Course 01
- Familiarity with linear algebra (vectors/matrices)
Recommended Datasets
6 datasets are aligned with this course for hands-on exercises and projects.
MAFAND-MT
Machine Translation for 21 African Languages
CC-100
Common Crawl Corpus for 100+ Languages
FLORES-200
Professional Translations for 200 Languages
Universal Dependencies
Grammar Annotation for African Languages
OPUS Parallel Corpora
Open Parallel Corpus Collection
Kencorpus
Kenyan Languages Corpus: Swahili, Dholuo, Luhya
About This Course
This course explores how to represent language data in ways that machines can understand. You’ll learn about vector representations, embeddings, and how to work with multilingual data.
Key Topics
- Word Embeddings: Learning dense vector representations of words
- Word2Vec and GloVe: Classic embedding algorithms
- Multilingual Embeddings: Working with African language representations
- Contextual Representations: Introduction to modern embedding techniques
- Embedding Evaluation: Measuring embedding quality
Prerequisites
- Completion of Course 01 or equivalent knowledge
- Linear algebra basics (vectors, matrices)
- Basic neural network concepts