Skip to main content

llm-d Community Update - June 2025

· 3 min read
Pete Cheslock
Pete Cheslock
AI Community Architect, Red Hat

Hey everyone! We've been making great progress with the llm-d project, and I wanted to share some important updates and opportunities to get involved.

Help Shape the Future of the llm-d Project​

To guide the future development of the llm-d project, we need to understand the real-world challenges, configurations, and performance needs of our community. We've created a short survey to gather insight into how you serve Large Language Models, from the hardware you use to the features you need most.

This anonymous, vendor-agnostic survey will take approximately 5 minutes to complete. Your input will directly influence the project's roadmap and priorities. The aggregated results will be shared with the llm-d-contributors mailing list to benefit the entire community.

Your Input Will Define Our Roadmap

We've created an llm-d Community Roadmap Survey to gather information about your LLM workloads. We are looking to learn more about:

  • Your Serving Environment: This includes the hardware you use now and anticipate using in a year (like NVIDIA GPUs, AMD GPUs, or CPUs), and whether you run on-premise, in the cloud, or on edge devices.
  • Your Model Strategy: Do you serve a few large models or many smaller ones, which model families (like Llama or Mistral) are most common, and how you utilize techniques like LoRA adapters.
  • Your Performance Requirements: Your real-world SLOs for latency and throughput and the biggest LLM serving challenges you face—from cost optimization to operational ease of use.
  • Your Future Needs: What single new feature you would prioritize for an LLM Model-as-a-Service to help guide our innovation.

Take the 5-Minute Survey​

Your participation is invaluable. Please take a few minutes to complete the survey. We encourage you to share it with other users or proxy their needs in your response to ensure our direction reflects the community's diverse requirements.

New YouTube Channel​

We're excited to announce our new YouTube channel! We've been recording our SIG meetings and creating tutorial content to help you get started with llm-d.

Subscribe to our YouTube

Subscribe to get notified when new videos are posted, presentations, demos, and SIG meeting recordings: @llm-d-project

Reminder: Join Our Google Group to Access Project Docs​

Join our Google Group

We use Google Groups to share architecture diagrams, SIG meeting notes, and other important project content. To get full access to all project resources, please join: llm-d-contributors Google Group

The Google Group is where all the magic happens! Here you'll find:

  • Detailed architecture diagrams and design documents
  • SIG meeting notes and recordings
  • Early access to new features and proposals
  • Direct communication with the core team and other contributors

Get Involved​

There are many ways to contribute to llm-d:

  1. Join the Google Group for full access to project resources
  2. Review the project public calendar and join an upcoming community meeting
  3. Subscribe to our YouTube channel for tutorials and meeting recordings
  4. Fill out the Use Case Survey to help shape our benchmarks
  5. Join our Slack workspace for real-time discussions
  6. Check out our Contributor Guidelines to start contributing code

We're looking forward to hearing from you and working together to make llm-d even better!