F2LLM-v2-160M One Piece Retrieval
Fine-tuned text embedding model for page retrieval in the One Piece Indexer project.
This model starts from codefuse-ai/F2LLM-v2-160M and was fine-tuned on page descriptions, metadata, OCR bubble text, search logs, and explicit feedback from the project database. It is intended for French semantic search over manga page descriptions and related OCR text.
The repository includes:
model.safetensors: PyTorch/Sentence Transformers checkpoint.onnx/model.onnx: ONNX export for Transformers.js browser inference and local Node inference.- tokenizer/config files required by Transformers.js.
training_manifest.json: local training configuration snapshot.
Intended Use
Use the model as a dense retrieval encoder:
- query text uses the F2LLM query instruction prompt:
Instruct: Given a question, retrieve passages that can help answer the question.\nQuery: - document/page text is embedded without the query prompt.
- embeddings are 640 dimensions and should be L2-normalized before cosine/IP search.
Local Evaluation
Evaluation was run on page-level held-out splits, so generated queries for the same page do not leak across train/eval/test.
| Split | Queries | Hit@1 | Hit@5 | Hit@10 | MRR@10 | NDCG@10 | Known bad@10 |
|---|---|---|---|---|---|---|---|
| eval | 518 | 0.670 | 0.894 | 0.959 | 0.761 | 0.803 | 0.000 |
| test | 485 | 0.654 | 0.891 | 0.944 | 0.755 | 0.798 | 0.002 |
Baseline on the same splits with the original local F2LLM-v2-160M package:
| Split | Queries | Hit@1 | Hit@5 | Hit@10 | MRR@10 | NDCG@10 | Known bad@10 |
|---|---|---|---|---|---|---|---|
| eval | 518 | 0.552 | 0.728 | 0.820 | 0.632 | 0.673 | 0.000 |
| test | 485 | 0.520 | 0.713 | 0.800 | 0.610 | 0.653 | 0.002 |
Historical production-context benchmark rows from earlier real feedback labels are useful context only because they are not the same split:
| Variant | Queries | Hit@10 | MRR@10 | NDCG@10 |
|---|---|---|---|---|
| F2LLM 160M text document | 75 | 0.813 | 0.533 | 0.584 |
| current Voyage DB | 73 | 0.932 | 0.650 | 0.693 |
| current Gemini DB | 75 | 0.200 | 0.115 | 0.121 |
| current dual DB | 75 | 0.747 | 0.413 | 0.478 |
Training Snapshot
- pages: 1060
- training pairs: 4432
- epochs: 1
- batch size: 8
- learning rate: 2e-5
- max sequence length: 512
- loss: MultipleNegativesRankingLoss
Data source mix:
- synthetic page-scene queries
- character/arc/keyword queries
- search feedback positives
- search log top-result positives
- OCR bubble quotes where available
ONNX Export
The ONNX model was exported with Optimum as a feature-extraction graph at sequence length 512. Export verification completed with a maximum absolute PyTorch-vs-ONNX output difference of 0.000141.
Caveats
This model is specialized for the project dataset and is not a general-purpose embedding benchmark result. Real replacement quality should keep being monitored with fresh search_feedback and search_logs labels after deployment.