SqueezeNet-SwiGLU (Modernized SqueezeNet Architecture)
SqueezeNet-SwiGLU is a modernized, ultra-lightweight Convolutional Neural Network (CNN) architecture based on the original SqueezeNet v1.1 design. It incorporates state-of-the-art deep learning architectural enhancements, including SwiGLU gated activations, FP32 Layer Normalization, and residual block scaling, delivering superior feature representation while maintaining a minimal parameter footprint.
Key Architectural Improvements (vs. Original SqueezeNet)
Compared to the legacy SqueezeNet v1.1 (Iandola et al., 2016), this model introduces several architectural modernizations:
| Feature | Legacy SqueezeNet v1.1 | SqueezeNet-SwiGLU (This Model) |
|---|---|---|
| Activation Function | Standard ReLU | Residual-Scaled SwiGLU Gated Activation |
| Normalization | Batch Normalization | FP32 Layer Normalization (GroupNorm(1, C)) |
| Batch Size Dependency | High (sensitive to batch stats & EMA lag) | Zero (Inference identical across any batch size) |
| Gradient Flow | Standard Fire Connections | Residual Fire Block Skip Connections & Scaling |
| Activation Variance | Prone to un-bounded drift | Strictly bounded via LayerNorm & FP32 Precision |
Benchmark & Evaluation
- Evaluation Dataset: ImageNet 200-Class Test Split (Tiny-ImageNet Categories)
- Input Resolution: 64 × 64 pixels (native) / 224 × 224 (interpolated)
- Top-1 Accuracy: 50.51%
- Top-5 Accuracy: 75.06%
Target Usecases & Applications
Due to its ultra-compact size and high throughput, SqueezeNet-SwiGLU is optimized for resource-constrained deployment environments:
- Edge & IoT Intelligence: Microcontrollers, Raspberry Pi, NVIDIA Jetson, and embedded vision hardware.
- Mobile AI Applications: On-device real-time visual classification (iOS CoreML / Android ONNX).
- High-FPS Video Analytics: Lightweight feature backbone for real-time surveillance, robotics, and drone navigation.
- Microservice Backends: Serving high-throughput image classification with minimal memory overhead per GPU/CPU node.
How to Use
Fast Inference with Hugging Face pipeline
from transformers import pipeline
# Initialize the classification pipeline (requires trust_remote_code=True)
classifier = pipeline(
"image-classification",
model="kd13/Modern-SqueezeNet",
trust_remote_code=True
)
# Run prediction on an image URL or local PIL Image
results = classifier("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")
for pred in results:
print(f"Label: {pred['label']} | Score: {pred['score']:.4f}")