Indic-Sentence-Completion
license: other
Details
The model cannot be commercially used. It's a fine-tuned Bloom-3B in several Indian languages:
- Gujarati
- Marathi
- Bangali
- Punjabi
- Kannada
- Malayalam
- Telugu
- Tamil
- Hindi
Architecture
Same as Bloom-3B, the model is decoder only.
Motivation behind the model fine-tuning
- The model can be fine-tuned for any downstream task that requires the use of the aforementioned Indian languages
- PEFT LoRA is advised.
- Can be stacked with an Encoder if needed for any Sequence to Sequence task that requires aforementioned Indian languages
Example of getting inference from the model
from transformers import AutoModel, AutoConfig, AutoModelForCausalLM, AutoTokenizer
# Path to the directory containing the model files
model_directory = "autopilot-ai/Indic-sentence-completion"
tokenizer = AutoTokenizer.from_pretrained(model_directory)
model = AutoModelForCausalLM.from_pretrained(
model_directory,
load_in_8bit=True,
device_map="auto",
)
# Load the model configuration
config = AutoConfig.from_pretrained(model_directory)
# Load the model
model = AutoModel.from_pretrained(model_directory, config=config)
batch = tokenizer("હેલો કેમ છો?", return_tensors='pt')
with torch.cuda.amp.autocast():
output_tokens = model.generate(**batch, max_new_tokens=10)
print('\n\n', tokenizer.decode(output_tokens[0], skip_special_tokens=True))
To run the above code snippet (in 8 bits), make sure to install the following
pip install accelerate bitsandbytes