Finetune Your Model
Moving from theory to application, this course addresses the reality that most researchers adapt existing foundation models rather than training from scratch. Learn transfer learning, fine-tuning techniques, and domain adaptation for African contexts.
Key Topics
- Transfer Learning: Principles of adapting pre-trained models
- Fine-tuning Techniques: Full fine-tuning vs Parameter-Efficient (PEFT)
- Domain adaptation for medical, legal, agricultural contexts
- Memory bottlenecks and efficiency in training
- Instruction-following datasets
Learning Outcomes
- Learn the vital skill of adapting pre-trained models
- Understand the computational trade-offs of fine-tuning
- Apply fine-tuning to solve specific African problems
- Execute fine-tuning workflows efficiently
- Specialize general-purpose models for niche tasks
Practical Projects
- Fine-tune a medical LLM for African healthcare questions
- Adapt a model for local language translation
- Fine-tune Llama to follow instructions in African languages
Recommended Datasets
7 datasets are aligned with this course for hands-on exercises and projects.
AfriInstruct
Instruction Tuning for African Languages
Inkuba Instruct
Multi-task Instruction Data by Lelapa AI
AfriMed-QA
Pan-African Medical Question Answering
Aya Dataset
Multilingual Instruction Dataset from Cohere
Med-Convo-Nig
Simulated Nigerian-Accented Medical Consultations
African Next Voices
Multilingual South African Speech (3000+ hrs)
Common Voice (Swahili & Hausa)
Crowdsourced Read Speech for African Languages
About This Course
Fine-tuning allows you to leverage powerful pre-trained models for your specific tasks. This course covers techniques for adapting models to African language tasks with limited data.
Key Topics
- Transfer Learning: Leveraging pre-trained representations
- Fine-tuning Strategies: Full fine-tuning vs. feature extraction
- Domain Adaptation: Adapting models to new domains
- Few-Shot Learning: Learning from limited examples
- Efficient Fine-tuning: LoRA, adapters, and prompt tuning
Prerequisites
- Completion of Courses 01-04 or equivalent knowledge
- Access to pre-trained models (Hugging Face)
- GPU resources recommended