Dr. Ronen Dar (Co-Founder/ CTO of @runailabs) talks about the challenges of running compute infrastructure for AI, the GPU ecosystem, sizing LLMs and more. 

SHOW: 739

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"

SHOW SPONSORS:

• GCore - Global Hosting, CDN, Edge and Cloud Services
• Use promocode “CLOUDCAST” to receive a €100 credit on Gcore services
• Equinix Global Data Centers and Networking 
• Learn more and signup at https://deploy.equinix.com/. Use the coupon code CLOUDCAST to get $500 in credits to get started.
• Datadog Kubernetes Solution: Maximum Visibility into Container Environments
• Start monitoring the health and performance of your container environment with a free 14 day Datadog trial. Listeners of The Cloudcast will also receive a free Datadog T-shirt.

SHOW NOTES:

• Run:ai - Build Your Next Large Model (homepage)
• Ronen Dar, Run:AI's CTO, on managing computation resources in ML pipelines

Topic 1 - Welcome to the show. Tell us a little bit about your background, and what you focus on at Run:ai.

Topic 2 - Let’s begin by talking about the challenges of running AI applications. What unique characteristics and requirements do AI applications have?

Topic 3 - Most AI applications run on GPUs. How do things change when using GPUs vs. CPUs to power AI applications? What is needed to get the most out of GPUs?

Topic 4 - As environments grow larger, what is needed to scale-up environments, both in terms of scheduling applications and managing the underlying GPU infrastructure?

Topic 5 - GPUs are not only expensive resources, but also in high-demand. How are companies doing capacity planning with GPUs? What struggles are you seeing companies have as they manage planning for AI projects?

Topic 6 - Are the new Large Language Models (LLMs) much different in size than AI models of the past? 

Topic 7 - How well is the industry prepared to deal with the new interest in AI from across the industry? 

FEEDBACK?

• Email: show at the cloudcast dot net
• Twitter: @thecloudcastnet