qwen3-4b-ft-msmarco

This model is based on Qwen/Qwen3-4B and was fine-tuned for 1 epoch on the English MSMARCO training data using FlagEmbedding.

This model is released as part of the ReasonEmbed resources. ReasonEmbed studies enhanced text embeddings for reasoning-intensive document retrieval; for more details, please refer to our paper.

Training Data

The model was fine-tuned on the MSMARCO data from hanhainebula/bge-multilingual-gemma2-data.

Training Procedure

• Base model: Qwen/Qwen3-4B
• Training framework: FlagEmbedding
• Training data: MSMARCO
• Number of epochs: 1

Tokenizer Note

Following the tokenizer modification discussed in Qwen3-Embedding-0.6B discussion #2, we modified the tokenizer so that it can automatically add the EOS token during tokenization.

License

This model is released under the Apache 2.0 license.

Citation

If you find this repository useful, please consider giving a star ⭐ and citation:

@article{chen2025reasonembed,
  title={ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval},
  author={Chen, Jianlyu and Lan, Junwei and Li, Chaofan and Lian, Defu and Liu, Zheng},
  journal={arXiv preprint arXiv:2510.08252},
  year={2025}
}