Recent research papers on Natural Language Processing from arXiv

Summary of Recent research on Natural Language Processing

# Decoding Consumer Preferences Using Attention-Based Language Models

- Authors: Joshua Foster, Fredrik Odegaard
- Summary: This paper proposes a new method for demand estimation using attention-based language models, specifically an encoder-only language model trained in two stages. It analyzes natural language descriptions of used cars for market demand primitives. The model projects language encodings into the parameter space of a structural model and validates its counterfactual analysis capability on unique zero-shot auction instances.
- Category: econ.EM
- Published: 2025-07-23
- URL: http://arxiv.org/pdf/2507.17564v1

# DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs

- Authors: Haolin Jin, Mengbai Xiao, Yuan Yuan, Xiao Zhang, Dongxiao Yu, Guanghui Zhang, Haoliang Wang
- Summary: This work addresses the quadratic complexity of self-attention in Transformers by designing DistrAttention, a more efficient and flexible self-attention method using locality-sensitive hashing to group data by embedding dimensionality. It integrates with FlashAttention-2 for high GPU performance, showing 37% speedup and better accuracy in vision transformer inference and Llama3-1B with minimal accuracy loss.
- Categories: cs.LG, cs.AI
- Published: 2025-07-23
- URL: http://arxiv.org/pdf/2507.17245v1

## Summary

Recent research in natural language processing focuses on improving demand estimation with language models and enhancing Transformer efficiency through novel self-attention mechanisms. The attention-based approaches are applied to real-world applications like used car auctions and large-scale model inference acceleration, showing significant improvements in performance and scalability.

## Papers

1. Decoding Consumer Preferences Using Attention-Based Language Models [2507.17564v1]
2. DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs [2507.17245v1]