Linkup open-sources SPARSEUP, a 149M-parameter sparse embedding model
Linkup released SPARSEUP, a learned sparse embedding model on a ModernBERT backbone, under Apache 2.0 on Hugging Face.
Linkup's research team has released SPARSEUP, an open-source learned sparse embedding model. It runs on a 149M-parameter ModernBERT backbone and ships under Apache 2.0. The team reports an average of 56.4 nDCG@10 on BEIR-13 and describes it as the strongest public vocabulary-based sparse encoder it knows of below 150M parameters.
The weights are on Hugging Face. The model loads through Transformers or Sentence Transformers with trust_remote_code=True.
Why sparse
Most open retrieval models are dense, producing one vector per text. Sparse models instead output weights across a vocabulary. Each dimension maps to a real token, so the vectors fit inverted indexes and remain human-readable, and they tend to match rare words well.
The work follows LightOn's DenseOn and LateOn release, which included open data, a training recipe, a dense model and a late-interaction model. SPARSEUP fills the missing sparse slot, using the same backbone family and fine-tuning data so all three retrieval styles can be compared directly.
Training began from the LateOn-unsupervised checkpoint. That checkpoint had no MLM head, so the team grafted back ModernBERT's original one. Fine-tuning used contrastive learning only, with 7 hard negatives per query sampled from a pool of 50 plus in-batch negatives. There is no cross-encoder distillation, and training fits on a single H100.
Controlled comparison
With backbone and data fixed, LateOn scores 58.9, DenseOn 57.9 and SPARSEUP 56.4. SPARSEUP uses approximate Seismic search while LightOn reports exact search. SPARSEUP wins ArguAna and Touché and beats DenseOn on HotpotQA, but lags on more semantic sets, with FiQA showing the largest gap and DBPedia another weak spot. On decontaminated BEIR the gap to DenseOn narrows to 0.17 points, though Linkup warns that decontaminated NQ and MS MARCO hold only 21 and 46 queries, making those numbers noisy.
On MS MARCO, SPARSEUP averages 47 non-zero terms per query and 190 per document, against 25 and 170 for SPLADE-v3. With the Seismic inverted index it reaches over 97% recall versus exact search in roughly 380 microseconds per query, single-threaded. Linkup says inflating vector size could add 1 to 2 BEIR points, but it chose to stay sparse.
Weights and technical details are covered in MarkTechPost's report.
