Toolverse
All skills

pytorch-fsdp2

by Orchestra-Research

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.

Installation

Pick a client and clone the repository into its skills directory.

Installation

Quick info

Category
Data Science
GitHub stars
13,477
Views
25

About this skill

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.

Related skills