NeoSumi
The text encoder can predict the logits behind the mask tokens.
Regardless of the captioning strategy, it can understand both booru captions and natural language.
The loss was calculated in the ForMaskedLM mode during training.
Anime images
The encoded text can be used with compatible, uncensored image generation models.
Masking syntax
<doc>word1 <mask>word2 ...
The tokens are often "word + whitespace". In the example, do not put a space between the mask token and the next word.
Setup
pip install "git+https://github.com/huggingface/transformers.git@4b24a197a1fa996496e3cf4f476c140e083218b5"
The model requires either the dev branch or future version v5.17 of the transformers library.
Base model
The NeoMME with 500k steps was chosen as the base model, not the predecay variant with 450k steps.
Tokenization
The tokenizer has remained unchanged.
The English stop words have been removed from the captions.
The comma-seperated keywords were scrambled and adjusted for the model.
Initial epochs
The model was initialised using a low learning rate and natural language captions from CC (Moondream) and Danbooru (Qwen3.5).
The masked language model then continued to learn from a combination of comma-separated and natural language texts.
The artists' given names, nicknames and fantasy names have been deliberately excluded from the dataset, as it would be difficult to predict these based on the text alone.
Fixed danbooru
Grio's patched tags were applied to the danbooru dataset. This makes it the most recent and accurate source.
Other booru sites were included as well.
Source data
- anime_captions
- artbench_captions
- cc12m_2mp_realistic
- danbooru_multitier_captions_202606
- dtg_character_tags
- furry_e621
- gelbooru_characters_enriched
- gelbooru_tags_full
- tag_cleaning
- wikiart_215k
- samples for a certain locked tag