DSPy topics
12 short topics in AI Evaluation LLMOps and Governance. Each one is a single focused exercise with a clear piece of evidence at the end.
12 topics, one exercise each.
Work through them in order, or jump to the one you need.
These are draft topic outlines (draft; instructor review pending). Full lessons follow after review; ask Ajay if you need a topic sooner.
- 01
Task and signature definition
Investigate task and signature definition using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 02
Input and output examples
Investigate input and output examples using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 03
Demonstration selection
Investigate demonstration selection using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 04
Candidate instruction construction
Investigate candidate instruction construction using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 05
Program and prompt composition
Investigate program and prompt composition using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 06
Dataset partitioning
Investigate dataset partitioning using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 07
Evaluation metric selection
Investigate evaluation metric selection using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 08
Optimization search constraints
Investigate optimization search constraints using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 09
Cost and runtime budgets
Investigate cost and runtime budgets using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 10
Overfitting and leakage checks
Investigate overfitting and leakage checks using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 11
Baseline comparison
Investigate baseline comparison using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course → - 12
Optimized-program versioning
Investigate optimized-program versioning using a repeatable DSPy experiment; compare results with a baseline and record limitations. Keep the exercise bounded and record the behavior supported by the chosen version.
Open micro-course →
More in LLM agent and retrieval frameworks.
You have the curiosity.
Let’s turn it into capability.
Start with a practical workshop or bring a specific question to Ajay.