BEE-spoke-data/tFINE-900m-e16-d32-flan

This is a basic text-to-text "instruct" model, similar to Google's original flan-t5 model series (but not trained for as long).

Fine-tuned from the base model on the pszemraj/flan-subsets-deduped dataset, subset flan-v2 for 1 epoch. It achieves the following results on the evaluation set:

• Loss: 1.4134
• Rouge1: 62.9142
• Rouge2: 22.5279
• Rougel: 61.4902
• Rougelsum: 61.7795
• Gen Len: 12.0586
• Num Input Tokens Seen: 1931815668

Model features

• pretrained & fine-tuned at 1024 context length (input)
• tokenizer with byte-pair fallback to support understanding and generating text beyond what the original T5 tokenizer does

Usage Example

from transformers import pipeline

pipe = pipeline(
    "text2text-generation",
    model="BEE-spoke-data/tFINE-900m-e16-d32-flan",
)
prompt = "What color is tuesday?"
res = pipe(prompt, max_new_tokens=96, top_k=4, penalty_alpha=0.6)
print(res[0]["generated_text"])

Quick eval

Quick eval for: BEE-spoke-data/tFINE-900m-e16-d32-flan

hf (pretrained=BEE-spoke-data/tFINE-900m-e16-d32-flan,trust_remote_code=True,dtype=bfloat16,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8