FlashAttention
Memory-efficient exact attention implementation for transformers.
About FlashAttention
FlashAttention improves speed and memory use in large model training.
Memory-efficient exact attention implementation for transformers.