Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation
Note: This is a huggingface-compatible version of the model converted by leloy. Currently only works on this branch of the Transformers library.
Anole is the first open-source, autoregressive, and natively trained large multimodal model capable of interleaved image-text generation (without using stable diffusion). While it builds upon the strengths of Chameleon, Anole excels at the complex task of generating coherent sequences of alternating text and images. Through an innovative fine-tuning process using a carefully curated dataset of approximately 6,000 images, Anole achieves remarkable image generation and understanding capabilities with minimal additional training. This efficient approach, combined with its open-source nature, positions Anole as a catalyst for accelerated research and development in multimodal AI. Preliminary tests demonstrate Anole's exceptional ability to follow nuanced instructions, producing high-quality images and interleaved text-image content that closely aligns with user prompts.
The major functionalities of Anole are listed below:
- Interleaved Text-Image Structured Generation (support added by leloy)
- Text-to-Image Generation
- Interleaved Text-Image Generation
- Text Generation
- MultiModal Understanding
where Bold represents newly added capabilities on the basis of Chameleon.
Please refer to our github repo for examples generated by Anole!