🇮🇳 ZY-S1 Preview

Born in India. Built for Intelligence.

ZY-S1 is an experimental language-model project developed as part of the ZYR AI project.

The ZYR project is being developed in India, with a focus on building, training, evaluating, and improving AI models.

ZY-S1 Preview is an early experimental checkpoint built on Sapient AI's HRM-Text-1B. It explores fine-tuning, corrective training, and evaluation across reasoning, reading comprehension, mathematics, and general knowledge tasks.

🇮🇳 Born in India. Built for Intelligence.

Developed in India. Shared with the world.

⚠️ Preview Status

ZY-S1 is a preview release, not a final or production-ready model.

This checkpoint is intended for experimentation, evaluation, and research. Its benchmark results are preliminary and should not be interpreted as proof of broad reasoning or mathematical capabilities.

The model has demonstrated encouraging results on selected tasks, but it also has significant limitations.

📌 Model Overview

Important: The ZYR project is being developed in India. ZY-S1 builds upon an existing upstream base model; this statement does not claim that the original base architecture or all model components originated in India.

🎯 Project Goals

ZY-S1 is part of the broader effort to develop and evaluate models under the ZYR project.

The current goals are to:

• Experiment with language-model fine-tuning and corrective training.
• Evaluate performance on reading comprehension, mathematics, science reasoning, and general knowledge.
• Identify weaknesses through structured benchmark testing.
• Improve answer accuracy and consistency through further training.
• Establish a reproducible evaluation process for future ZYR checkpoints.

ZY-S1 Preview represents an experimental step in this ongoing development process.

🧠 Architecture and Training

Base Model

ZY-S1 uses Sapient AI HRM-Text-1B as its starting point.

The checkpoint retains its relationship to the upstream model and should not be described as a model trained entirely from scratch.

Training Process

The reported training process consists of:

The training process used a filtered subset of:

MnemicAI/Ling-Coder-SFT-English-Clean

The results depend on the selected training data, training configuration, base-model behavior, and evaluation methodology.

These training-step counts describe the reported runs, not a guarantee of convergence or general capability.

📊 Benchmark Results — ZY-S1 Preview

The following results were measured in a custom PrefixLM-aware generation evaluation using the ZY-S1 Preview checkpoint. ARC-Challenge and MMLU used 200-question samples. DROP used a 200-question sample. These are sample-based results, not official full-benchmark leaderboard scores.

Evaluation details

• ARC-Challenge: 155 correct answers out of 200 questions.
• MMLU: 113 correct answers out of 200 questions, sampled across multiple subjects.
• DROP Exact Match: 79.50% on the evaluated 200-question sample.
• DROP Token F1: 82.16% on the same DROP sample.
• MATH-500: No verified final result is included here. Add the score only after the 500-question evaluation has completed and the final output has been checked.

Important evaluation caveats

These results were produced with a custom PrefixLM-aware generation script. They should not be presented as official full-dataset scores or directly compared with results from a different evaluation harness without matching prompts, answer extraction, normalization, few-shot settings, and dataset split.

• ARC-Challenge and MMLU were evaluated on 200 questions each, not their complete official evaluation sets.
• DROP was evaluated on a 200-question sample using custom DROP-style normalization and scoring. The reported EM and Token F1 have not been independently verified with the official DROP evaluator.
• DROP contains structured and multi-span answers; differences in reference handling and normalization can change scores.
• The MMLU sample spans subjects unevenly, so per-subject results from this sample should not be treated as reliable subject rankings.
• Scores can vary with question selection, prompt formatting, answer extraction, decoding settings, and evaluation implementation.
• These measurements describe this checkpoint and this evaluation run; they do not guarantee performance on other tasks or deployments.

For reproducible comparisons, publish the evaluation scripts, exact dataset revision and split, sampling seed, prompt format, scoring implementation, and prediction files alongside future results.

🧪 Intended Use

ZY-S1 Preview is intended for:

• AI research and experimentation.
• Small-scale benchmark evaluation.
• Fine-tuning and training workflow experiments.
• Studying the behavior of the HRM-Text architecture after additional training.
• Identifying weaknesses that can guide future ZYR development.

The checkpoint is not yet recommended for production deployments or tasks requiring dependable reasoning.

⚠️ Known Limitations

ZY-S1 Preview may:

• Generate incorrect or inconsistent answers.
• Fail on multi-step mathematical problems.
• Produce incorrect answers even when its output appears confident.
• Struggle with complex instructions or unfamiliar tasks.
• Show substantial variation across different evaluation samples.
• Inherit limitations from its upstream base model and training data.

Do not rely on this preview as the sole basis for consequential decisions.

💻 Loading the Model

The model uses the HRM-Text architecture. The example below illustrates the PrefixLM inference approach used during evaluation. It assumes the repository includes compatible architecture code, tokenizer files, configuration, and model weights.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_ID = "zyr-AGENT/ZY-S1"

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dtype = torch.float16 if device.type == "cuda" else torch.float32

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    torch_dtype=dtype,
)

model.to(device)
model.eval()

if tokenizer.eos_token_id is None:
    raise RuntimeError("The tokenizer must define an EOS token.")

prompt = (
    "### Instruction:\n"
    "Programming language: Python\n"
    "Task:\n"
    "Write a function add(a, b) that returns the sum of two numbers.\n\n"
    "### Response:\n"
)

encoded = tokenizer(
    prompt,
    return_tensors="pt",
    truncation=True,
    max_length=384,
)

input_ids = encoded["input_ids"].to(device)
attention_mask = torch.ones_like(input_ids)

# HRM-Text PrefixLM convention used in the evaluation:
# prompt/prefix positions have token_type_id 1;
# newly generated response positions have token_type_id 0.
token_type_ids = torch.ones_like(input_ids)
generated_ids = []

with torch.inference_mode():
    for _ in range(128):
        outputs = model(
            input_ids=input_ids,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids,
            use_cache=False,
        )

        next_token = outputs.logits[0, -1, :].argmax().view(1, 1)
        next_token = next_token.to(device)
        token_id = int(next_token.item())

        if token_id == tokenizer.eos_token_id:
            break

        generated_ids.append(token_id)

        input_ids = torch.cat([input_ids, next_token], dim=1)
        attention_mask = torch.cat(
            [attention_mask, torch.ones_like(next_token)], dim=1
        )
        token_type_ids = torch.cat(
            [token_type_ids, torch.zeros_like(next_token)], dim=1
        )

print(tokenizer.decode(generated_ids, skip_special_tokens=True))

Compatibility note: This is an example of the PrefixLM inference procedure used during evaluation, not a guarantee that every Transformers version will support this custom architecture automatically. Verify that the repository contains all required custom model code and test inference after uploading.

🗺️ The ZYR Project

ZYR is an AI development project being developed in India.

Its work includes experimentation with model architectures, training methods, evaluation workflows, and future model releases.

ZY-S1 Preview is one experimental checkpoint within this broader project. It should be understood as an early development release rather than a representation of the final capabilities envisioned for ZYR.

Our tagline

Born in India. Built for Intelligence. 🇮🇳

Developed in India. Shared with the world.

🙏 Acknowledgements

We acknowledge the creators and maintainers of the upstream HRM-Text model, the datasets used in training, and the open-source tools that support model development and evaluation.

Please consult the upstream repository and dataset documentation for their respective authorship, terms, and attribution requirements.

• Base model: Sapient AI HRM-Text-1B
• Training-data source: MnemicAI/Ling-Coder-SFT-English-Clean

📜 License and Third-Party Notices

This repository declares the Apache-2.0 license in its metadata.

Before redistributing the model weights or publishing a final release, verify that:

• The upstream base model's license permits the intended use and distribution.
• The applicable license terms for the training dataset are respected.
• All required attribution and third-party notices are included.
• The repository's license metadata accurately reflects the rights applicable to the actual released artifacts.

The license declared in this repository does not override any obligations applicable to upstream components.

🚀 What's Next?

Potential future work includes:

• Larger and more representative benchmark evaluations.
• Additional training and corrective fine-tuning.
• Better mathematical reasoning and answer accuracy.
• Improved reproducibility and evaluation reporting.
• Further experiments under the ZYR project.

These are development goals, not guaranteed features or results.

ZY-S1 Preview

Born in India. Built for Intelligence.

🇮🇳 The ZYR project is being developed in India.