Dynamic GPU Allocation for Spaces

ZeroGPU is a shared infrastructure that optimizes GPU usage for AI models and demos on Hugging Face Spaces. It dynamically allocates and releases NVIDIA A100 GPUs as needed, offering:

1. Free GPU Access: Enables cost-effective GPU usage for Spaces.
2. Multi-GPU Support: Allows Spaces to leverage multiple GPUs concurrently on a single application.

Unlike traditional single-GPU allocations, ZeroGPU’s efficient system lowers barriers for developers, researchers, and organizations to deploy AI models by maximizing resource utilization and power efficiency.

Using and hosting ZeroGPU Spaces

• Using existing ZeroGPU Spaces
  • ZeroGPU Spaces are available to use for free to all users. (Visit the curated list).
  • PRO users get x5 more daily usage quota and highest priority in GPU queues when using any ZeroGPU Spaces.
• Hosting your own ZeroGPU Spaces

Technical Specifications

• GPU Type: Nvidia A100
• Available VRAM: 40GB per workload

Compatibility

ZeroGPU Spaces are designed to be compatible with most PyTorch-based GPU Spaces. While compatibility is enhanced for high-level Hugging Face libraries like transformers and diffusers, users should be aware that:

• Currently, ZeroGPU Spaces are exclusively compatible with the Gradio SDK.
• ZeroGPU Spaces may have limited compatibility compared to standard GPU Spaces.
• Unexpected issues may arise in some scenarios.

Supported Versions

• Gradio: 4+
• PyTorch: 2.0.1, 2.1.2, 2.2.2, 2.4.0 (Note: 2.3.x is not supported due to a PyTorch bug)
• Python: 3.10.13

Getting started with ZeroGPU

To utilize ZeroGPU in your Space, follow these steps:

1. Make sure the ZeroGPU hardware is selected in your Space settings.
2. Import the spaces module.
3. Decorate GPU-dependent functions with @spaces.GPU.

This decoration process allows the Space to request a GPU when the function is called and release it upon completion.

Example Usage

import spaces
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(...)
pipe.to('cuda')

@spaces.GPU
def generate(prompt):
    return pipe(prompt).images

gr.Interface(
    fn=generate,
    inputs=gr.Text(),
    outputs=gr.Gallery(),
).launch()

Note: The @spaces.GPU decorator is designed to be effect-free in non-ZeroGPU environments, ensuring compatibility across different setups.

Duration Management

For functions expected to exceed the default 60-second of GPU runtime, you can specify a custom duration:

@spaces.GPU(duration=120)
def generate(prompt):
   return pipe(prompt).images

This sets the maximum function runtime to 120 seconds. Specifying shorter durations for quicker functions will improve queue priority for Space visitors.

Hosting Limitations

By leveraging ZeroGPU, developers can create more efficient and scalable Spaces, maximizing GPU utilization while minimizing costs.

Feedback

You can share your feedback on Spaces ZeroGPU directly on the HF Hub: https://huggingface.co/spaces/zero-gpu-explorers/README/discussions

> Update on GitHub

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!