ReSB2-Base
ReSB2-Base is a domain-adapted version of jhu-clsp/mmBERT-base, obtained through continued pre-training with the Masked Language Modeling (MLM) objective on the SB2 Dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3.0
Framework versions
- Transformers 4.57.3
- Pytorch 2.13.0+cu130
- Tokenizers 0.22.2
Citation
If you use this model, please cite:
@inproceedings{lage2026resb2,
title={ReSB²: Machine-Assisted Linking of Legislative Bills using Domain-Adapted ModernBERT and Explainable AI},
author={Lage, Lucas Gabriel and others},
booktitle={Proceedings of the ACM Conference on Hypertext and Social Media},
year={2026}
}
License
The models developed and released in this repository, including the trained ReSB² models, are provided for research and academic purposes.
The use of the underlying pre-trained models is subject to the licenses and terms of use defined by their original providers. Users are responsible for complying with the respective licenses when using, modifying, or redistributing these models.
Acknowledgments
This work was supported by the Legislative Assembly of Minas Gerais (ALMG), CNPq, CAPES, and FAPEMIG.