moe-training
Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional
Installation
Pick a client and clone the repository into its skills directory.
Installation
About this skill
Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.
Related skills
backend-security-coder
by sickn33
openapi-spec-generation
by wshobson
architect-review
by sickn33
security-compliance
by davila7
reviewing-code
by CaptainCrouton89
better-auth-best-practices
by novuhq