Winnow (June 2026)
A long-context handling pipeline combining soft activation-space compression, hard token pruning, and KV-cache quantization — relatively nascent research directions — into a unified system for faster long-context inference. Implemented token pruning by fusing two importance signals: LLMlingua-2 (token-level classification) and AttentionRAG (sentence-level attention scoring). Applied Latent Context Language Models (LCLM) with TurboQuant KV-cache quantization for A100 GPUs. Built with PyTorch, Transformers, Modal, and FastAPI.