How this works
| Step | What happens |
|---|---|
| 1 | Downloads embeddings + metadata from deirdosh/curriculum_embeddings |
| 2 | Runs UMAP-2D and UMAP-3D projections |
| 3 | Fits guided BERTopic (5-8 topics per concept) |
| 4 | Computes cross-concept measures (JSD, cosine sim, entropy, silhouette) |
| 5 | Builds semantic kNN graphs |
| 6 | Exports CSVs, interactive HTML charts, PNG images |
| 7 | Pushes everything to deirdosh/curriculum_analysis_results |
HF_TOKEN is read from Space secrets automatically. All heavy steps are individually cached — re-running skips completed work.