Satish_News_Generator

A fine-tuned version of Qwen/Qwen2.5-3B-Instruct that writes short, informative, abstract-style passages in one of two requested styles: human-written or AI-generated. It was trained with QLoRA (4-bit quantization + LoRA adapters) on a free Google Colab T4 GPU, and the LoRA weights were merged into the base model, so it loads as a standard transformers model without peft.

This model was built as a portfolio / learning project to demonstrate an end-to-end supervised fine-tuning (SFT) pipeline: environment setup, data formatting, QLoRA training, adapter merging, and publishing to the Hub.

Model Details

Model Description

• Developed by: Satish Marineni (MSatish04)
• Model type: Causal language model (decoder-only transformer, Qwen2 architecture), instruction/chat tuned
• Language(s) (NLP): English
• License: Inherits the license of the base model (Qwen Research License). Check the base model's license before any use.
• Finetuned from model: Qwen/Qwen2.5-3B-Instruct
• Parameters: ~3.1B (merged, float16, single model.safetensors file of ~6.2 GB)

Model Sources

• Repository: https://huggingface.co/MSatish04/Satish_News_Generator
• Training data: https://huggingface.co/datasets/dataspoof/Fine_tuned_project

Uses

Direct Use

Generating short, informative passages, typically in the style of research-paper abstracts, conditioned on a requested writing style:

• "...in the style of a human writer."
• "...in the style of an AI system."

The model responds best to the exact prompt format it was trained on (see How to Get Started).

Downstream Use

• Generating synthetic human-style and AI-style text samples, for example to build or stress-test AI-text detection datasets.
• A starting point or reference for further QLoRA fine-tuning experiments on small models.

Out-of-Scope Use

• Not a source of facts. The model produces plausible-sounding but invented content, including fabricated studies, locations, dates, and statistics. Do not treat its output as real research.
• Not an AI-text detector. It was trained to generate text in a style, not to classify text.
• Passing off generated text as genuine human-written or academic work (e.g., academic dishonesty, fake abstracts, or misleading publications).
• High-stakes use in medical, legal, scientific, or news-reporting contexts.

Bias, Risks, and Limitations

• Name vs. behaviour: despite the "News_Generator" name, the training data consists mostly of short, academic-abstract-style texts, so outputs resemble research abstracts more than news articles.
• Hallucination: outputs often describe realistic-sounding but non-existent studies.
• No topic control in training: training prompts only specified a style, never a topic. The model may not reliably follow a topic you add to the prompt.
• Limited training: one epoch on ~6k short examples, with no formal evaluation. Style separation between "human" and "AI" outputs has only been checked qualitatively.
• Inherited biases: it carries over any biases and limitations of the base Qwen2.5-3B-Instruct model and of the source dataset.

Recommendations

Label generated text as AI-generated, verify any factual claims independently, and avoid using the model where fabricated content could cause harm.

How to Get Started with the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

repo = "MSatish04/Satish_News_Generator"
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(repo)

messages = [
    {"role": "system", "content": "You are a helpful writing assistant."},
    {"role": "user",   "content": "Write a short informative passage in the style of a human writer."},
    # or: "Write a short informative passage in the style of an AI system."
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
out = pipe(prompt, max_new_tokens=200, do_sample=True, temperature=0.7, return_full_text=False)
print(out[0]["generated_text"])

Training Details

Training Data

dataspoof/Fine_tuned_project (data_for_preprocessing.csv): 6,069 rows, each containing:

The dataset appears to be derived from the Kaggle AI and Human Text Dataset.

Training Procedure

Preprocessing

Each row was converted into a three-turn chat conversation, and TRL applied Qwen's chat template automatically:

• system: You are a helpful writing assistant.
• user: Write a short informative passage in the style of {a human writer | an AI system}. (chosen from the Author label)
• assistant: the row's Text

Training Hyperparameters

• Method: QLoRA supervised fine-tuning with TRL SFTTrainer
• Quantization (training): 4-bit NF4, float16 compute dtype, no double quantization
• LoRA: r = 64, alpha = 16, dropout = 0.1, bias = none, task = CAUSAL_LM (PEFT default target modules for Qwen2)
• Epochs: 1 (~1,518 optimizer steps)
• Batch size: 4 per device, gradient accumulation 1
• Learning rate: 2e-4, cosine schedule, 3% linear warmup
• Optimizer: paged_adamw_32bit, weight decay 0.001, max grad norm 0.3
• Max sequence length: 256 tokens, no packing
• Gradient checkpointing: enabled
• Training regime: fp16/bf16 mixed precision disabled; 4-bit base weights with float16 compute
• Post-training: LoRA adapter merged into the float16 base model with merge_and_unload()

Evaluation

No formal quantitative evaluation was performed. The model was checked qualitatively by prompting it with both style instructions and confirming that it produces coherent, abstract-style passages consistent with the training data.

Environmental Impact

• Hardware Type: NVIDIA Tesla T4 (16 GB)
• Cloud Provider: Google Colab (Google Cloud)
• Hours used: Approximately one to two hours of GPU time, including training, merging, and upload

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Technical Specifications

Model Architecture and Objective

Qwen2 decoder-only transformer (Qwen2.5-3B-Instruct), fine-tuned with the standard next-token prediction (causal language modeling) objective on chat-formatted examples.

Compute Infrastructure

Hardware

Single NVIDIA Tesla T4 GPU (Google Colab, free tier).

Software

• transformers 5.17.0
• trl 1.13.0
• peft 0.21.0
• bitsandbytes 0.50.2
• accelerate 1.15.0
• datasets 5.0.1
• huggingface_hub 1.33.0
• torch 2.11.0 (CUDA 12.8)

Model Card Authors

Satish Marineni (MSatish04)

Model Card Contact

Via the MSatish04 Hugging Face profile.