Toolverse
All skills

optimizing-attention-flash

by davila7

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (u003e512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA,

Installation

Pick a client and clone the repository into its skills directory.

Installation

Quick info

Author
davila7
Category
Security
GitHub stars
32,576
Views
13

About this skill

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (u003e512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

Related skills