BrtGPT-0719

Summary

This model is trained on same dataset with BrtGPT-1-Pre trained on. But model is trained on 2,1 times more data than BrtGPT-1-Pre. "0719" is for: "This check-point only"

--CHANGE LOG--

• New evaluation: Model tested on GPQA Diamond scored: %15.6~!
• New evaluation: Model tested on MMLU scored: %16.5~!
• New evaluation: Model tested on HLE (Humanity's Last Exam) and scored %4+!
• We are sorry about wrong measurement! (6.6 is wrong!)

Use

Direct use (Hugging Face Space) is cooming soon! Code use (Google Colab) (Stream):

from transformers import AutoTokenizer, AutoModelForCausalLM, TextIteratorStreamer
import torch
from threading import Thread

# === MODEL and TOKENIZER ===
model_id = "Bertug1911/BrtGPT-1-0719"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
model.eval().to("cuda" if torch.cuda.is_available() else "cpu")

# === CHAT ===
messages = [
    {"role": "user", "content": "How to make a cup of coffee?"},
]

# === TEMPLATE PROMPT ===
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# === STREAMER ===
streamer = TextIteratorStreamer(
    tokenizer,
    skip_prompt=True,
    skip_special_tokens=True
)

# === GENERATE ===
def generate():
    model.generate(
        input_ids=inputs,
        streamer=streamer,
        max_new_tokens=128,
        do_sample=True,
        top_k=40,
        temperature=0.8,
    )

# === THREAD START ===
thread = Thread(target=generate)
thread.start()

# === POST-Processing ===
def clean(text):
    return text.replace(" ", "").replace("Ġ", " ").replace("Ċ", "\n")

# === STREAM and CLEAN ===
for token in streamer:
    cleaned = clean(token)
    print(cleaned, end="", flush=True)

Another code (No-stream):

from transformers import pipeline

# Pipeline
pipe = pipeline(
    "text-generation",
    model="Bertug1911/BrtGPT-1-0719",
    trust_remote_code=True,
    top_k=40,              # Good for creativity
    temperature=0.8,       # Good for creativity
    max_new_tokens=128     # Default maximum model output (Maximum 1024)
)

# Messages
messages = [
    {"role": "user", "content": "What is the capital of France?"},
]

# Take out
output = pipe(messages)

# Only write asistant's (Model output) answer
assistant_response = output[0]["generated_text"][-1]["content"].strip()
# Special token conversions
formatted_out = assistant_response.replace(" ", "").replace("Ġ", " ").replace("Ċ", "\n")

print(formatted_out)

Difference beetween previus model (BrtGPT-1-Pre)

This model is slightly more good at math.

Evaluation

Risks

May generates harmfull and Illegal output! USE WITH CAUTION!