Loading listing…
knowledge-distillation
Security scan passedCompress large language models using knowledge distillation from teacher to student models.
Part of AI Research Skills (Orchestra)Pack · 98 skills
by Orchestra-Research· AI & agentsUnclaimed · Yours? Claim it
What knowledge-distillation does
About
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
Install knowledge-distillation
More from AI & agents
Skills people install alongside knowledge-distillation.
