VedaX-0.7B-Base
VedaX-0.7B-Base is a lightweight open-weight language model developed by VedaX Labs. The model is designed for efficient conversational AI, reasoning tasks, and lightweight deployment on consumer hardware. The model is entirely trained on CPU and on cosmopedia dataset
Training Specs
- CPU:- Intel(R) Core(TM) i3-14100
- Memory:- 16.0 GB DDR5
- GPU:- None (Intel UHD Graphics 730)
Model Overview
- Model Name: VedaX-0.7B-Base
- Organization: VedaX Labs
- Parameters: 0.7 Billion
- Architecture: Transformer-based causal language model
- Type: Base/Foundation Model
- License: Apache-2.0
Capabilities
- Text generation
- Lightweight inference
- Fine-tuning friendly
Intended Use
VedaX-0.7B-Base is intended for:
- Research and experimentation
- Educational purposes
- Building lightweight AI assistants
- Fine-tuning for downstream tasks
- Local inference applications
- Suitable for all devices
Context Length
- Context Window: 4096 tokens
Training
The model was trained on a mixture of publicly available text datasets and curated instruction-style data.
Example Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "VedaX-Labs/VedaX-0.7B-Base"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
prompt = "The magic forest was"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=128
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Benchmarking(Higher number means higher score)
Limitations
- May generate inaccurate information
- Limited reasoning compared to larger models
- Responses may contain hallucinations
- Performance depends on prompting quality
- May generate repettetive words(Chances is very low)
Future Plans
- Instruction-tuned versions
- Reasoning-enhanced variants
- Multimodal support
- Larger parameter models
Developed by VedaX Labs.