EXAONE U-CURATE Random1K 1epoch v1
This repository contains a standalone BF16 merged full model, not a LoRA adapter-only release. It combines the pinned EXAONE base model with the selected U-CURATE Random1K QLoRA checkpoint and can be loaded without PEFT adapter files.
Base model
- Model:
LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct - Revision:
553ea250b9a5317231459279d5847d6cf955b9aa - License: EXAONE AI Model License Agreement 1.1 - NC (see
LICENSE)
Training
- Method: QLoRA (4-bit NF4 training; BF16 compute)
- Training data: U-CURATE Random1K, 1,000 authorized training records
- Epochs: 1
- Optimizer steps: 125
- Final train loss: 2.7517
- Sequence length: 2,048
- Effective batch size: 8 (micro-batch 1, gradient accumulation 8)
- LoRA rank / alpha / dropout: 8 / 16 / 0.05
- LoRA targets:
q_proj,k_proj,v_proj,out_proj - Selected checkpoint:
checkpoint-125 - Training hardware: NVIDIA A40 48 GB
- Training Git commit:
bfbd12f375b5c3ed3bfea61e701384cd6f1c2d5f
The checkpoint was selected because it is the completed final step and no independent validation evidence justified choosing an earlier checkpoint.
Release format
- Precision: BF16
- Format: safetensors, 8 weight shards
- Merge: PEFT
merge_and_unload(safe_merge=True)into the pinned base model - Adapter SHA-256:
c48554032752d18c7e120a40de9f6946c33792f3adf4a9129b8675bf22e158c9 - Shard and release-file hashes:
MERGED_MODEL_SHA256SUMS
The included EXAONE custom modeling code contains a Transformers 5.15 generation-API compatibility adjustment. The adjustment does not modify model weights.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "hoySky92/EXAONE-U-CURATE-Random1K-1epoch-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
torch_dtype="auto",
device_map="auto",
)
messages = [{"role": "user", "content": "한국의 사계절을 간단히 설명해 주세요."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Validation
- Fresh standalone Transformers load: PASS
- Korean greedy generation: 3/3 PASS
- Empty output / NaN or Inf / repetition explosion: none observed
- Adapter-versus-merged top-1 token agreement: PASS
- Adapter-versus-merged greedy 32-token sequence agreement: PASS
The stricter pre-registered numerical-logit fidelity threshold did not pass and has not been relaxed: cosine 0.9998158 versus required 0.9999, and relative L2 0.0428226 versus maximum 0.02. The original evidence is preserved under validation/; this release proceeds based on the exact functional token and greedy-generation agreement authorized by the release owner.
Limitations
This is an experimental one-epoch transfer run on 1,000 training samples. It is not a general guarantee of factuality, safety, or leaderboard performance. Users must comply with the base model license and independently evaluate the model for their intended use.