Run BART with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/bart_base

Paper: BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (arXiv:1910.13461) · HF Papers

BART is a denoising seq2seq transformer: a bidirectional encoder (like BERT) and an autoregressive decoder (like GPT) trained to reconstruct corrupted text. It excels at summarization, translation, and other text-to-text tasks. Byte-level BPE tokenizer (shared with RoBERTa); the decoder starts from </s>.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of facebook/bart-base for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a conditional generation (base seq2seq) checkpoint (BartConditionalGenerate). Other task heads load the shared backbone from this repo (start randomly initialized, ready for fine-tuning); fine-tuned task checkpoints load via the hf: prefix.

Base checkpoint (not task fine-tuned): use it as a backbone (BartModel) for features, or fine-tune a task head.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.bart import BartConditionalGenerate, BartTokenizer

model = BartConditionalGenerate.from_weights("zeromodels/bart_base")
tokenizer = BartTokenizer.from_weights("zeromodels/bart_base")

inputs = tokenizer('The quick brown fox jumps over the lazy dog.')
ids = model.generate(
    inputs,
    [[model.decoder_start_token_id]],
    max_new_tokens=64,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(ids[0], skip_special_tokens=True))

Load any BART variant the same way with from_weights("zeromodels/<variant>"):

Available classes

Load any of these from this repo with from_weights("zeromodels/bart_base") (or on the fly via the hf: prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a hf: fine-tune).

from zeromodels.models.bart import BartSequenceClassify
# zero-shot / NLI fine-tune loads on the fly via the hf: prefix
model = BartSequenceClassify.from_weights("hf:facebook/bart-large-mnli")

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Prefer BartTokenizer.from_weights(...) so the byte-level BPE matches.
• BART's decoder starts from </s> (decoder_start_token_id = 2); pass eos_token_id=tokenizer.eos_token_id to stop generation.
• See the BART docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. BartConditionalGenerate.from_weights("hf:facebook/bart-base").

Special Thanks

A huge thank you to the Meta AI (FAIR) authors for creating and releasing BART.

License: apache-2.0.