gguf quantized and fp8/16 scaled version of bagel
- base model from bytedance-seed
- multimodal trial model (i.e., t2i, image editing/recogition)
run it with gguf-connector
ggc b2
review/reference
- simply execute the command (
ggc b2) above in console/terminal - opt a
vaethen opt amodelfile in the current directory to interact (see example below)
Detecting GGUF/Safetensors...
GGUF file(s) available. Select which one for VAE:
- pig_ae_fp32-f16.gguf
- pig_ae_fp32-f32.gguf
Enter your choice (1 to 2): 1
VAE file: pig_ae_fp32-f16.gguf is selected!
Safetensors file(s) available. Select which one for MODEL:
- ema_bf16.safetensors
- ema_fp16.safetensors (for non-cuda user)
- ema_fp8_e4m3fn.safetensors (recommended)
- ema_fp8_e5m2.safetensors
Enter your choice (1 to 4): _
- note: for the latest update, only tokenizer will be pulled to gguf-connector folder (cache) automatically during the first launch; you need to prepare the bulky model and vae files still, working like vision connector right away; mix and match, more flexible
- run it entirely offline; i.e., from local URL: http://127.0.0.1:7860 with lazy webui
- require dependency: bagel2;
pip install bagel2; for flash-attn and triton, could opt to install it with pre-built wheels, i.e.,here, unless you can build the wheel yourself successfully - might need some optional dependencies, please refer to the checklist; as connector won't force your machine to install any of those by default
- gguf-connector (pypi)