Dataset : Custom Dataset
VCTI-RoBERTa-Fiber
Model Summary
This model is a domain-adapted RoBERTa-base model fine-tuned using Masked Language Modeling (MLM) on optical communication and photonics data. It is optimized for generating domain-specific embeddings that capture the nuances and technical jargon of the optical domain.
⚠️ Note: This is the basic version of our ongoing development.
A significantly improved version trained on much larger and more diverse optical corpora will be released soon!
Training Data
The model was trained on:
- 1000+ Optical Wikipedia Articles
- 120+ Optical Communication & Photonics Textbooks
- 500+ ITUT and IEEE Papers
- 1000+ Web Articles
The training corpus includes content related to:
- Optical fibers
- Photonic devices
- Multiplexing (WDM, TDM, OTN )
- Optical amplifiers
- Modulation techniques
- Communication networks
- Laser systems ....etc
⚙️ Training Details
| Parameter | Value | Description |
|---|---|---|
| batch_size | 64 | Number of samples per training batch |
| epochs | 15 | Number of training epochs |
| patience | 6 | Early stopping patience |
| learning_rate | 5e-5 | Learning rate for the AdamW optimizer |
| weight_decay | 0.01 | Weight decay for regularization |
| objective | MLM | Masked Language Modeling |
The training was performed using the transformers library by Hugging Face.
Core Use Case: Domain-Specific Embeddings
The fine-tuned model is particularly effective at generating context-aware embeddings for the optical domain. This makes it highly suitable for tasks such as:
- Semantic Search across technical documents
- Retrieval-Augmented Generation (RAG) for Q&A systems
- Topic Modeling and document clustering
- Similarity Matching between questions, answers, or papers
How to Use
Load the model
from transformers import RobertaTokenizerFast, RobertaModel
tokenizer = RobertaTokenizerFast.from_pretrained("quantum-leap-vcti/VCTI-RoBERTa-Fiber")
model = RobertaModel.from_pretrained("quantum-leap-vcti/VCTI-RoBERTa-Fiber")
text = "Wavelength-division multiplexing increases the capacity of optical fibers."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
embedding = outputs.last_hidden_state.mean(dim=1)
MIT License
Copyright (c) 2025 Velankani Communications Technologies Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files , to deal in the Software without restriction