LFM 350M-C
A masked model that is used as a text encoder. This is the fourth iteration of the model.
Regardless of the captioning strategy, it can understand both booru captions and natural language and provide a pooled embedding.
Anime images
The model has been verified for both the LFM (1024) and the CLIP (768) dimensions.
This encoded text can be used as input data for a compatible T2I model.
Contrastive Text Model
The pooled embedding can be computed from any of the previously described formats while preserving the underlying semantic representation.
Unlike image encoder models, the embeddings were learned from pairs of texts.
<|startoftext|><|pool|>text
Tokenization
The tokenizer includes the optional pool token.
The English stop words have been removed from the captions.
The comma-seperated keywords were scrambled and adjusted for the model.
Syntax
Just use the tokenizer for any of these.
<|startoftext|>comma, separated, words
<|startoftext|>A long, one-line sentence.
<|startoftext|>A simplified, general description in minimalist markdown format.
## character name
character description
## character name
character description
background, style
Changes
The initialization is better than the previous LFM-C.
Due to the lack of data, it doesn't understand images yet, so please refer to the LFM-R repo for VL tasks.
Source data
- anime_captions
- cc12m_2mp_realistic
- danbooru_multitier_captions_202606
- dtg_character_tags
- furry_e621
- gelbooru_characters_enriched
- gelbooru_tags_full
- markdown texts, mainly from Qwen 27-35B
- tag_cleaning
- wikiart_215k
- samples for a certain locked tag
References
- 2103.00020
- 2609.04720