SmolLumi-8B-Instruct
____ _ _ _
/ ___| _ __ ___ ___ | | | _ _ _ __ ___ (_)
\___ \| '_ ` _ \ / _ \| | | | | | | '_ ` _ \| |
___) | | | | | | (_) | | |__| |_| | | | | | | |
|____/|_| |_| |_|\___/|_|_____\__,_|_| |_| |_|_|
- Developed by: safe049
- License: apache-2.0
- Finetuned from model : NeverSleep/Lumimaid-v0.2-8B
- GGUF[Q4_K_M] : safe049/SmolLumi-8B-Instruct-GGUF
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Description
Arguments:
- per_device_train_batch_size = 2,
- gradient_accumulation_steps = 4,
- warmup_steps = 5,
- max_steps = 60,
- learning_rate = 2e-4,
- fp16 = not is_bfloat16_supported(),
- bf16 = is_bfloat16_supported(),
- logging_steps = 1,
- optim = "adamw_8bit",
- weight_decay = 0.01,
- lr_scheduler_type = "linear",
- seed = 3407
Used Dataset
Used Library
- transformers
- unsloth
- trl
- sft
More
Yet another model created cuz of boring This Model is Uncensored, it might generate illegal,non-moral contents,and I am not reponsable for that.