Browse Datasets
Course 04

Assemble Your Transformer Architecture

This is the theoretical centerpiece of the curriculum. Explore the architecture that underpins modern Generative AI like Gemini and GPT. Understand attention mechanisms, positional encoding, and build your own transformer model.

Duration: 4 hours 8 Datasets

Key Topics

  • The Transformer: History and architecture
  • Self-Attention Mechanisms: How models weigh word importance
  • Masked Attention: Preventing the model from seeing the future
  • Multi-Head Attention: Capturing different relationship types
  • Context-sensitive next-token prediction

Learning Outcomes

  • Investigate how Transformers process prompts
  • Understand the mathematical mechanism of attention
  • Visualize how a model 'thinks' by examining attention weights
  • Differentiate between Transformer components (encoders/decoders)
  • Build retrieval-based QA systems

Practical Projects

  • Visualize attention weights to see model focus during generation
  • Fine-tune mBERT for answer span extraction
  • Build a cross-lingual QA system using African language documents

Recommended Datasets

8 datasets are aligned with this course for hands-on exercises and projects.

MAFAND-MT

Machine Translation for 21 African Languages

machine-translation alignment ~5k-30k sentences per language
Bambara Ewe Fon Hausa +5 more
PAN-AF
View Details

AfriQA

Cross-lingual Question Answering for 10 African Languages

question-answering cross-lingual 12,000+ QA pairs
Bemba Fon Hausa Igbo +6 more
NG RW
View Details

XL-Sum

Abstractive Summarization for African Languages

summarization text-generation 6k-7k pairs per language
Amharic Hausa Igbo Kirundi +6 more
PAN-AF
View Details

Belebele

Reading Comprehension for 122 Language Variants

reading-comprehension multiple-choice 900 questions per language
Bambara Wolof Yoruba Zulu +1 more
PAN-AF
View Details

KenSwQuAD

Swahili Question Answering Dataset

question-answering 7,526 QA pairs
Swahili
KE TZ
View Details

NaijaRC

Reading Comprehension for Nigerian Languages

reading-comprehension multiple-choice ~358 questions (Yoruba subset)
Hausa Igbo Yoruba
NG
View Details

African Medical Records (OCR)

Nigerian Handwritten Clinical Records for OCR

ocr computer-vision <1K records
English (Nigeria)
NG
View Details

Kencorpus

Kenyan Languages Corpus: Swahili, Dholuo, Luhya

machine-translation pos-tagging 5.6M words / 177 hours
Swahili Dholuo Luhya
KE
View Details

About This Course

The transformer architecture revolutionized NLP. In this course, you’ll understand every component of transformers and build one from scratch for machine translation and other sequence-to-sequence tasks.

Key Topics

  • Self-Attention: The core mechanism of transformers
  • Multi-Head Attention: Parallel attention for richer representations
  • Positional Encoding: Adding position information
  • Encoder-Decoder Architecture: Building complete transformer models
  • Training Transformers: Efficient training strategies

Prerequisites

  • Completion of Courses 01-03 or equivalent knowledge
  • Strong understanding of matrix operations
  • Experience with PyTorch