About
Quartz-S1-Base-33.89M is a pre-trained AI model ready to be fine tuned (or use it as it is lol)
It uses GPT-2 architecture
Trained on: HuggingFaceFW/fineweb-edu
Training
Trained for 27,000 steps
Final Loss: 5.313 Final validation loss: 5.321
List of Features
- Text Generation (e.g., trying to complecte text)
- Fine Tuning Capabilities (e.g., customizing the model)
- Small Language Model Capabilities (e.g., while it is very dumb once you fine tune it it can answer basic questions)
Summary
It is a very small language model (SLM) and has very high chances to produce incorrect, repetitive, or nonsensical text. It should not be expected to have the capabilities of much larger language models.