- Publisher |
- Lukas Biewald
- Media Type |
- audio
- Podknife tags |
- Artificial Intelligence,
- Interview,
- Machine Learning,
- Technology
- Categories Via RSS |
- Technology
This podcast currently has no reviews.
Submit ReviewIn this episode of Gradient Dissent, Isomorphic Labs Chief AI Officer Max Jaderberg, and Chief Technology Officer Sergei Yakneen join our host Lukas Biewald to discuss the advancements in biotech and drug discovery being unlocked with machine learning.
With backgrounds in advanced AI research at DeepMind, Max and Sergei offer their unique insights into the challenges and successes of applying AI in a complex field like biotechnology. They share their journey at Isomorphic Labs, a company dedicated to revolutionizing drug discovery with AI. In this episode, they discuss the transformative impact of deep learning on the drug development process and Isomorphic Labs' strategy to innovate from molecular design to clinical trials.
You’ll come away with valuable insights into the challenges of applying AI in biotech, the role of AI in streamlining the drug discovery pipeline, and peer into the future of AI-driven solutions in healthcare.
Connect with Sergei Yakneen & Max Jaderberg:
https://www.linkedin.com/in/maxjaderberg/
https://www.linkedin.com/in/yakneensergei/
https://twitter.com/SergeiIakhnin
https://twitter.com/maxjaderberg
Follow Weights & Biases:
🚀 Discover the cutting-edge AI hardware development for enterprises in this episode of Gradient Dissent, featuring Rodrigo Liang, CEO of SambaNova Systems.
Rodrigo Liang’s journey from Oracle to founding SambaNova is a tale of innovation and determination. In this episode, Rodrigo discusses the importance of specialized hardware in unlocking AI's potential for Enterprise businesses and SambaNova's mission to deliver comprehensive AI solutions from chips to models.
Explore the critical insights on navigating the challenges of introducing AI to executives and the evolution of AI applications within large enterprises, and get a glimpse into the future of AI in the business world.
🎙 Get our podcasts on these platforms:
Apple Podcasts: http://wandb.me/apple-podcasts
Spotify: http://wandb.me/spotify
Google: http://wandb.me/gd_google
YouTube: http://wandb.me/youtube
Connect with Rodrigo Liang:
https://www.linkedin.com/in/rodrigo-liang/
https://twitter.com/RodrigoLiang
Follow Weights & Biases:
https://twitter.com/weights_biases
https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
🚀 This episode of Gradient Dissent welcomes Edo Liberty, the mind behind Pinecone's revolutionary vector database technology.
As a former leader at Amazon AI Labs and Yahoo's New York lab, Edo Liberty's extensive background in AI research and development showcases the complexities behind vector databases and their essential role in enhancing AI's capabilities.
Discover the pivotal moments and key decisions that have defined Pinecone's journey, learn about the different embedding strategies that are reshaping AI applications, and understand how Pinecone's success has had a profound impact on the technology landscape.
Connect with Edo Liberty:
https://www.linkedin.com/in/edo-liberty-4380164/
https://twitter.com/EdoLiberty
Follow Weights & Biases:
https://twitter.com/weights_biases
https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
🚀 In this episode of Gradient Dissent, we explore the revolutionary impact of AI across industries with Clara Shih, CEO of Salesforce AI and Founder of Hearsay Systems.
Dive into Salesforce AI's cutting-edge approach to customer service through AI, the importance of a trust-first strategy, and the future of AI policies and education. Learn how Salesforce empowers businesses and shapes the future with AI innovations like Prompt Builder and Copilot Studio. Whether you're an AI enthusiast, a business leader, or someone curious about the future of technology, this discussion offers valuable insights into navigating the rapidly evolving world of AI.
✅ Subscribe to Weights & Biases YouTube → https://bit.ly/45BCkYz
🎙 Get our podcasts on these platforms:
Apple Podcasts: http://wandb.me/apple-podcasts
Spotify: http://wandb.me/spotify
Google: http://wandb.me/gd_google
Connect with Clara:
https://www.linkedin.com/in/clarashih/
Follow Weights & Biases:
In the newest episode of Gradient Dissent, we explore the intersecting worlds of AI and Healthcare with John Halamka, President of the Mayo Clinic Platform.
Journey with us down John Halamka's remarkable path from his early tech startup days to leading innovations as the President of the Mayo Clinic Platform, one of the world's most esteemed healthcare institutions. This deep dive into AI's role in modern medicine covers the technology's evolution, its potential to redefine patient care, and the visionary work of Mayo Clinic Platform in harnessing AI responsibly.
Explore the misconceptions surrounding AI in healthcare and discover the ethical and regulatory frameworks guiding its application. Glimpse into the future with Halamka's visionary perspective on AI's potential to democratize and revolutionize healthcare across the globe. Join us for an enlightening discussion on the challenges, triumphs, and the horizon of AI in healthcare through the lens of John Halamka's pioneering experiences.
🎙 Get our podcasts on these platforms:
Apple Podcasts: http://wandb.me/apple-podcasts
Spotify: http://wandb.me/spotify
Google: http://wandb.me/gd_google
YouTube: http://wandb.me/youtube
✅ Follow Weights & Biases:
In the newest episode of Gradient Dissent, Chelsea Finn, Assistant Professor at Stanford's Computer Science Department, discusses the forefront of robotics and machine learning.
Discover her groundbreaking work, where two-armed robots learn to cook shrimp (messes included!), and discuss how robotic learning could transform student feedback in education.
We'll dive into the challenges of developing humanoid and quadruped robots, explore the limitations of simulated environments and discuss why real-world experience is key for adaptable machines. Plus, Chelsea will offer a glimpse into the future of household robotics and why it may be a few years before a robot is making your bed.
Whether you're an AI enthusiast, a robotics professional, or simply curious about the potential and future of the technology, this episode offers unique insights into the evolving world of robotics and where it's headed next.
*Subscribe to Weights & Biases* → https://bit.ly/45BCkYz
🎙 Get our podcasts on these platforms:
Apple Podcasts: http://wandb.me/apple-podcasts
Spotify: http://wandb.me/spotify
Google: http://wandb.me/gd_google
YouTube: http://wandb.me/youtube
Connect with Chelsea Finn:
https://www.linkedin.com/in/cbfinn/
https://twitter.com/chelseabfinn
Follow Weights & Biases:
https://twitter.com/weights_biases
https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
In the latest episode of Gradient Dissent, Richard Socher, CEO of You.com, shares his insights on the power of AI in search. The episode focuses on how advanced language models like GPT-4 are transforming search engines and changing the way we interact with digital platforms. The discussion covers the practical applications and challenges of integrating AI into search functionality, as well as the ethical considerations and future implications of AI in our digital lives. Join us for an enlightening conversation on how AI and you.com are reshaping how we access and interact with information online.
*Subscribe to Weights & Biases* → https://bit.ly/45BCkYz
Timestamps:
00:00 - Introduction to Gradient Dissent Podcast
00:48 - Richard Socher’s Journey: From Linguistic Computer Science to AI
06:42 - The Genesis and Evolution of MetaMind
13:30 - Exploring You.com's Approach to Enhanced Search
18:15 - Demonstrating You.com's AI in Mortgage Calculations
24:10 - The Power of AI in Search: A Deep Dive with You.com
30:25 - Security Measures in Running AI-Generated Code
35:50 - Building a Robust and Secure AI Tech Stack
42:33 - The Role of AI in Automating and Transforming Digital Work
48:50 - Discussing Ethical Considerations and the Societal Impact of AI
55:15 - Envisioning the Future of AI in Daily Life and Work
01:02:00 - Reflecting on the Evolution of AI and Its Future Prospects
01:05:00 - Closing Remarks and Podcast Wrap-Up
🎙 Get our podcasts on these platforms:
Apple Podcasts: http://wandb.me/apple-podcasts
Spotify: http://wandb.me/spotify
Google: http://wandb.me/gd_google
YouTube: http://wandb.me/youtube
Connect with Richard Socher:
https://www.linkedin.com/in/richardsocher/
https://twitter.com/RichardSocher
Follow Weights & Biases:
https://twitter.com/weights_biases
https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
Explore the Future of Investment & Impact in AI with Host Lukas Biewald and Guests Elad Gill and Sarah Guo of the No Priors podcast.
Sarah is the founder of Conviction VC, an AI-centric $100 million venture fund. Elad, a seasoned entrepreneur and startup investor, boasts an impressive portfolio in over 40 companies, each valued at $1 billion or more, and wrote the influential "High Growth Handbook."
Join us for a deep dive into the nuanced world of AI, where we'll explore its broader industry impact, focusing on how startups can seamlessly blend product-centric approaches with a balance of innovation and practical development.
*Subscribe to Weights & Biases* → https://bit.ly/45BCkYz
Timestamps:
0:00 - Introduction
5:15 - Exploring Fine-Tuning vs RAG in AI
10:30 - Evaluating AI Research for Investment
15:45 - Impact of AI Models on Product Development
20:00 - AI's Role in Evolving Job Markets
25:15 - The Balance Between AI Research and Product Development
30:00 - Code Generation Technologies in Software Engineering
35:00 - AI's Broader Industry Implications
40:00 - Importance of Product-Driven Approaches in AI Startups
45:00 - AI in Various Sectors: Beyond Software Engineering
50:00 - Open Source vs Proprietary AI Models
55:00 - AI's Impact on Traditional Roles and Industries
1:00:00 - Closing Thoughts
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
Follow Weights & Biases:
YouTube: http://wandb.me/youtube
Twitter: https://twitter.com/weights_biases
LinkedIn: https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
#OCR #DeepLearning #AI #Modeling #ML
In the latest episode of Gradient Dissent, we explore the innovative features and impact of LlamaIndex in AI data management with Jerry Liu, CEO of LlamaIndex. Jerry shares insights on how LlamaIndex integrates diverse data formats with advanced AI technologies, addressing challenges in data retrieval, analysis, and conversational memory. We also delve into the future of AI-driven systems and LlamaIndex's role in this rapidly evolving field. This episode is a must-watch for anyone interested in AI, data science, and the future of technology.
Timestamps:
0:00 - Introduction
4:46 - Differentiating LlamaIndex in the AI framework ecosystem.
9:00 - Discussing data analysis, search, and retrieval applications.
14:17 - Exploring Retrieval Augmented Generation (RAG) and vector databases.
19:33 - Implementing and optimizing One Bot in Discord.
24:19 - Developing and evaluating datasets for AI systems.
28:00 - Community contributions and the growth of LlamaIndex.
34:34 - Discussing embedding models and the use of vector databases.
39:33 - Addressing AI model hallucinations and fine-tuning.
44:51 - Text extraction applications and agent-based systems in AI.
49:25 - Community contributions to LlamaIndex and managing refactors.
52:00 - Interactions with big tech's corpus and AI context length.
54:59 - Final thoughts on underrated aspects of ML and challenges in AI.
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
Connect with Jerry:
https://twitter.com/jerryjliu0
https://www.linkedin.com/in/jerry-liu-64390071/
Follow Weights & Biases:
YouTube: http://wandb.me/youtube
Twitter: https://twitter.com/weights_biases
LinkedIn: https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
#OCR #DeepLearning #AI #Modeling #ML
In the latest episode of Gradient Dissent, we hear from Joseph Spisak, Product Director, Generative AI @Meta, to explore the boundless impacts of AI and its expansive role in reshaping various sectors.
We delve into the intricacies of models like GPT and Llama2, their influence on user experiences, and AI's groundbreaking contributions to fields like biology, material science, and green hydrogen production through the Open Catalyst Project. The episode also examines AI's practical business applications, from document summarization to intelligent note-taking, addressing the ethical complexities of AI deployment.
We wrap up with a discussion on the significance of open-source AI development, community collaboration, and AI democratization.
Tune in for valuable insights into the expansive world of AI, relevant to developers, business leaders, and tech enthusiasts.
We discuss:
- 0:00 Intro
- 0:32 Joe is Back at Meta
- 3:28 What Does Meta Get Out Of Putting Out LLMs?
- 8:24 Measuring The Quality Of LLMs
- 10:55 How Do You Pick The Sizes Of Models
- 16:45 Advice On Choosing Which Model To Start With
- 24:57 The Secret Sauce In The Training
- 26:17 What Is Being Worked On Now
- 33:00 The Safety Mechanisms In Llama 2
- 37:00 The Datasets Llama 2 Is Trained On
- 38:00 On Multilingual Capabilities & Tone
- 43:30 On The Biggest Applications Of Llama 2
- 47:25 On Why The Best Teams Are Built By Users
- 54:01 The Culture Differences Of Meta vs Open Source
- 57:39 The AI Learning Alliance
- 1:01:34 Where To Learn About Machine Learning
- 1:05:10 Why AI For Science Is Under-rated
- 1:11:36 What Are The Biggest Issues With Real-World Applications
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
In the premiere episode of Gradient Dissent Business, we're joined by Weights & Biases co-founder Chris Van Pelt for a deep dive into the world of large language models like GPT-3.5 and GPT-4. Chris bridges his expertise as both a tech founder and AI expert, offering key strategies for startups seeking to connect with early users, and for enterprises experimenting with AI. He highlights the melding of AI and traditional web development, sharing his insights on product evolution, leadership, and the power of customer conversations—even for the most introverted founders. He shares how personal development and authentic co-founder relationships enrich business dynamics. Join us for a compelling episode brimming with actionable advice for those looking to innovate with language models, all while managing the inherent complexities. Don't miss Chris Van Pelt's invaluable take on the future of AI in this thought-provoking installment of Gradient Dissent Business.
We discuss:
- 0:00 - Intro
- 5:59 - Impactful relationships in Chris's life
- 13:15 - Advice for finding co-founders
- 16:25 - Chris's fascination with challenging problems
- 22:30 - Tech stack for AI labs
- 30:50 - Impactful capabilities of AI models
- 36:24 - How this AI era is different
- 47:36 - Advising large enterprises on language model integration
- 51:18 - Using language models for business intelligence and automation
- 52:13 - Closing thoughts and appreciation
Thanks for listening to the Gradient Dissent Business podcast, with hosts Lavanya Shukla and Caryn Marooney, brought to you by Weights & Biases. Be sure to click the subscribe button below, to keep your finger on the pulse of this fast-moving space and hear from other amazing guests
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Brandon Duderstadt, Co-Founder and CEO of Nomic AI. Both of Nomic AI’s products, Atlas and GPT4All, aim to improve the explainability and accessibility of AI.
We discuss:
- (0:55) What GPT4All is and its value proposition.
- (6:56) The advantages of using smaller LLMs for specific tasks.
- (9:42) Brandon’s thoughts on the cost of training LLMs.
- (10:50) Details about the current state of fine-tuning LLMs.
- (12:20) What quantization is and what it does.
- (21:16) What Atlas is and what it allows you to do.
- (27:30) Training code models versus language models.
- (32:19) Details around evaluating different models.
- (38:34) The opportunity for smaller companies to build open-source models.
- (42:00) Prompt chaining versus fine-tuning models.
Resources mentioned:
Brandon Duderstadt - https://www.linkedin.com/in/brandon-duderstadt-a3269112a/
Nomic AI - https://www.linkedin.com/company/nomic-ai/
Nomic AI Website - https://home.nomic.ai/
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Soumith Chintala, VP/Fellow of Meta and Co-Creator of PyTorch. Soumith and his colleagues’ open-source framework impacted both the development process and the end-user experience of what would become PyTorch.
We discuss:
- The history of PyTorch’s development and TensorFlow’s impact on development decisions.
- How a symbolic execution model affects the implementation speed of an ML compiler.
- The strengths of different programming languages in various development stages.
- The importance of customer engagement as a measure of success instead of hard metrics.
- Why community-guided innovation offers an effective development roadmap.
- How PyTorch’s open-source nature cultivates an efficient development ecosystem.
- The role of community building in consolidating assets for more creative innovation.
- How to protect community values in an open-source development environment.
- The value of an intrinsic organizational motivation structure.
- The ongoing debate between open-source and closed-source products, especially as it relates to AI and machine learning.
Resources:
- Soumith Chintala
https://www.linkedin.com/in/soumith/
- Meta | LinkedIn
https://www.linkedin.com/company/meta/
- Meta | Website
- Pytorch
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Andrew Feldman, Founder and CEO of Cerebras Systems. Andrew and the Cerebras team are responsible for building the largest-ever computer chip and the fastest AI-specific processor in the industry.
We discuss:
- The advantages of using large chips for AI work.
- Cerebras Systems’ process for building chips optimized for AI.
- Why traditional GPUs aren’t the optimal machines for AI work.
- Why efficiently distributing computing resources is a significant challenge for AI work.
- How much faster Cerebras Systems’ machines are than other processors on the market.
- Reasons why some ML-specific chip companies fail and what Cerebras does differently.
- Unique challenges for chip makers and hardware companies.
- Cooling and heat-transfer techniques for Cerebras machines.
- How Cerebras approaches building chips that will fit the needs of customers for years to come.
- Why the strategic vision for what data to collect for ML needs more discussion.
Resources:
Andrew Feldman - https://www.linkedin.com/in/andrewdfeldman/
Cerebras Systems - https://www.linkedin.com/company/cerebras-systems/
Cerebras Systems | Website - https://www.cerebras.net/
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Harrison Chase, Co-Founder and CEO of LangChain. Harrison and his team at LangChain are on a mission to make the process of creating applications powered by LLMs as easy as possible.
We discuss:
- What LangChain is and examples of how it works.
- Why LangChain has gained so much attention.
- When LangChain started and what sparked its growth.
- Harrison’s approach to community-building around LangChain.
- Real-world use cases for LangChain.
- What parts of LangChain Harrison is proud of and which parts can be improved.
- Details around evaluating effectiveness in the ML space.
- Harrison's opinion on fine-tuning LLMs.
- The importance of detailed prompt engineering.
- Predictions for the future of LLM providers.
Resources:
Harrison Chase - https://www.linkedin.com/in/harrison-chase-961287118/
LangChain | LinkedIn - https://www.linkedin.com/company/langchain/
LangChain | Website - https://docs.langchain.com/docs/
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Jean Marc Alkazzi, Applied AI at idealworks. Jean focuses his attention on applied AI, leveraging the use of autonomous mobile robots (AMRs) to improve efficiency within factories and more.
We discuss:
- Use cases for autonomous mobile robots (AMRs) and how to manage a fleet of them.
- How AMRs interact with humans working in warehouses.
- The challenges of building and deploying autonomous robots.
- Computer vision vs. other types of localization technology for robots.
- The purpose and types of simulation environments for robotic testing.
- The importance of aligning a robotic fleet’s workflow with concrete business objectives.
- What the update process looks like for robots.
- The importance of avoiding your own biases when developing and testing AMRs.
- The challenges associated with troubleshooting ML systems.
Resources:
Jean Marc Alkazzi - https://www.linkedin.com/in/jeanmarcjeanazzi/
idealworks |LinkedIn - https://www.linkedin.com/company/idealworks-gmbh/
idealworks | Website - https://idealworks.com/
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Stella Biderman, Executive Director at EleutherAI and Lead Scientist - Mathematician at Booz Allen Hamilton.
EleutherAI is a grassroots collective that enables open-source AI research and focuses on the development and interpretability of large language models (LLMs).
We discuss:
- How EleutherAI got its start and where it's headed.
- The similarities and differences between various LLMs.
- How to decide which model to use for your desired outcome.
- The benefits and challenges of reinforcement learning from human feedback.
- Details around pre-training and fine-tuning LLMs.
- Which types of GPUs are best when training LLMs.
- What separates EleutherAI from other companies training LLMs.
- Details around mechanistic interpretability.
- Why understanding what and how LLMs memorize is important.
- The importance of giving researchers and the public access to LLMs.
Stella Biderman - https://www.linkedin.com/in/stellabiderman/
EleutherAI - https://www.linkedin.com/company/eleutherai/
Resources:
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
On this episode, we’re joined by Aidan Gomez, Co-Founder and CEO at Cohere. Cohere develops and releases a range of innovative AI-powered tools and solutions for a variety of NLP use cases.
We discuss:
- What “attention” means in the context of ML.
- Aidan’s role in the “Attention Is All You Need” paper.
- What state-space models (SSMs) are, and how they could be an alternative to transformers.
- What it means for an ML architecture to saturate compute.
- Details around data constraints for when LLMs scale.
- Challenges of measuring LLM performance.
- How Cohere is positioned within the LLM development space.
- Insights around scaling down an LLM into a more domain-specific one.
- Concerns around synthetic content and AI changing public discourse.
- The importance of raising money at healthy milestones for AI development.
Aidan Gomez - https://www.linkedin.com/in/aidangomez/
Cohere - https://www.linkedin.com/company/cohere-ai/
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
Resources:
#OCR #DeepLearning #AI #Modeling #ML
Jonathan Frankle, Chief Scientist at MosaicML and Assistant Professor of Computer Science at Harvard University, joins us on this episode. With comprehensive infrastructure and software tools, MosaicML aims to help businesses train complex machine-learning models using their own proprietary data.
We discuss:
- Details of Jonathan’s Ph.D. dissertation which explores his “Lottery Ticket Hypothesis.”
- The role of neural network pruning and how it impacts the performance of ML models.
- Why transformers will be the go-to way to train NLP models for the foreseeable future.
- Why the process of speeding up neural net learning is both scientific and artisanal.
- What MosaicML does, and how it approaches working with clients.
- The challenges for developing AGI.
- Details around ML training policy and ethics.
- Why data brings the magic to customized ML models.
- The many use cases for companies looking to build customized AI models.
Jonathan Frankle - https://www.linkedin.com/in/jfrankle/
Resources:
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.
#OCR #DeepLearning #AI #Modeling #ML
About This Episode
Shreya Shankar is a computer scientist, PhD student in databases at UC Berkeley, and co-author of "Operationalizing Machine Learning: An Interview Study", an ethnographic interview study with 18 machine learning engineers across a variety of industries on their experience deploying and maintaining ML pipelines in production.
Shreya explains the high-level findings of "Operationalizing Machine Learning"; variables that indicate a successful deployment (velocity, validation, and versioning), common pain points, and a grouping of the MLOps tool stack into four layers. Shreya and Lukas also discuss examples of data challenges in production, Jupyter Notebooks, and reproducibility.
Show notes (transcript and links): http://wandb.me/gd-shreya
---
💬 *Host:* Lukas Biewald
---
*Subscribe and listen to Gradient Dissent today!*
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
About this episode
In this episode of Gradient Dissent, Lukas interviews Dave Rogenmoser (CEO & Co-Founder) and Saad Ansari (Director of AI) of Jasper AI, a generative AI company with a focus on text generation for content like blog posts, articles, and more. The company has seen impressive growth since it's launch at the start of 2021.
Lukas talks with Dave and Saad about how Jasper AI was able to sell the capabilities of large language models as a product so successfully, and how they are able to continually improve their product and take advantage of steps forward in the AI industry at large.
They also speak on how they keep their business ahead of the competition, where they put their focus on in terms of R&D, and how they are able to keep the insights they've learned over the years relevant at all times as their company grows in employee count and company value.
Other topics include the potential use of generative AI in domains it hasn't necessarily seen yet, as well as the impact that community and user feedback plays on the constant tweaking and tuning processes that machine learning models go through.
Connect with Dave & Saad:
Find Dave on Twitter and LinkedIn.
Find Saad on LinkedIn.
---
💬 Host: Lukas Biewald
---
Subscribe and listen to Gradient Dissent today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Sarah Catanzaro is a General Partner at Amplify Partners, and one of the leading investors in AI and ML. Her investments include RunwayML, OctoML, and Gantry.
Sarah and Lukas discuss lessons learned from the "AI renaissance" of the mid 2010s and compare the general perception of ML back then to now. Sarah also provides insights from her perspective as an investor, from selling into tech-forward companies vs. traditional enterprises, to the current state of MLOps/developer tools, to large language models and hype bubbles.
Show notes (transcript and links): http://wandb.me/gd-sarah-catanzaro
---
⏳ Timestamps:
0:00 Intro
1:10 Lessons learned from previous AI hype cycles
11:46 Maintaining technical knowledge as an investor
19:05 Selling into tech-forward companies vs. traditional enterprises
25:09 Building point solutions vs. end-to-end platforms
36:27 LLMS, new tooling, and commoditization
44:39 Failing fast and how startups can compete with large cloud vendors
52:31 The gap between research and industry, and vice versa
1:00:01 Advice for ML practitioners during hype bubbles
1:03:17 Sarah's thoughts on Rust and bottlenecks in deployment
1:11:23 The importance of aligning technology with people
1:15:58 Outro
---
📝 Links
📍 "Operationalizing Machine Learning: An Interview Study" (Shankar et al., 2022), an interview study on deploying and maintaining ML production pipelines: https://arxiv.org/abs/2209.09125
---
Connect with Sarah:
📍 Sarah on Twitter: https://twitter.com/sarahcat21
📍 Sarah's Amplify Partners profile: https://www.amplifypartners.com/investment-team/sarah-catanzaro
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan
---
Subscribe and listen to Gradient Dissent today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Cristóbal Valenzuela is co-founder and CEO of Runway ML, a startup that's building the future of AI-powered content creation tools. Runway's research areas include diffusion systems for image generation.
Cris gives a demo of Runway's video editing platform. Then, he shares how his interest in combining technology with creativity led to Runway, and where he thinks the world of computation and content might be headed to next. Cris and Lukas also discuss Runway's tech stack and research.
Show notes (transcript and links): http://wandb.me/gd-cristobal-valenzuela
---
⏳ Timestamps:
0:00 Intro
1:06 How Runway uses ML to improve video editing
6:04 A demo of Runway’s video editing capabilities
13:36 How Cris entered the machine learning space
18:55 Cris’ thoughts on the future of ML for creative use cases
28:46 Runway’s tech stack
32:38 Creativity, and keeping humans in the loop
36:15 The potential of audio generation and new mental models
40:01 Outro
---
🎥 Runway's AI Film Festival is accepting submissions through January 23! 🎥
They are looking for art and artists that are at the forefront of AI filmmaking. Submissions should be between 1-10 minutes long, and a core component of the film should include generative content
--
📝 Links
📍 "High-Resolution Image Synthesis with Latent Diffusion Models" (Rombach et al., 2022)", the research paper behind Stable Diffusion: https://research.runwayml.com/publications/high-resolution-image-synthesis-with-latent-diffusion-models
📍 Lexman Artificial, a 100% AI-generated podcast: https://twitter.com/lexman_ai
---
Connect with Cris and Runway:
📍 Cris on Twitter: https://twitter.com/c_valenzuelab
📍 Runway on Twitter: https://twitter.com/runwayml
📍 Careers at Runway: https://runwayml.com/careers/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan
---
Subscribe and listen to Gradient Dissent today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Jeremy Howard is a co-founder of fast.ai, the non-profit research group behind the popular massive open online course "Practical Deep Learning for Coders", and the open source deep learning library "fastai".
Jeremy is also a co-founder of #Masks4All, a global volunteer organization founded in March 2020 that advocated for the public adoption of homemade face masks in order to help slow the spread of COVID-19. His Washington Post article "Simple DIY masks could help flatten the curve." went viral in late March/early April 2020, and is associated with the U.S CDC's change in guidance a few days later to recommend wearing masks in public.
In this episode, Jeremy explains how diffusion works and how individuals with limited compute budgets can engage meaningfully with large, state-of-the-art models. Then, as our first-ever repeat guest on Gradient Dissent, Jeremy revisits a previous conversation with Lukas on Python vs. Julia for machine learning.
Finally, Jeremy shares his perspective on the early days of COVID-19, and what his experience as one of the earliest and most high-profile advocates for widespread mask-wearing was like.
Show notes (transcript and links): http://wandb.me/gd-jeremy-howard-2
---
⏳ Timestamps:
0:00 Intro
1:06 Diffusion and generative models
14:40 Engaging with large models meaningfully
20:30 Jeremy's thoughts on Stable Diffusion and OpenAI
26:38 Prompt engineering and large language models
32:00 Revisiting Julia vs. Python
40:22 Jeremy's science advocacy during early COVID days
1:01:03 Researching how to improve children's education
1:07:43 The importance of executive buy-in
1:11:34 Outro
1:12:02 Bonus: Weights & Biases
---
📝 Links
📍 Jeremy's previous Gradient Dissent episode (8/25/2022): http://wandb.me/gd-jeremy-howard
📍 "Simple DIY masks could help flatten the curve. We should all wear them in public.", Jeremy's viral Washington Post article: https://www.washingtonpost.com/outlook/2020/03/28/masks-all-coronavirus/
📍 "An evidence review of face masks against COVID-19" (Howard et al., 2021), one of the first peer-reviewed papers on the effectiveness of wearing masks: https://www.pnas.org/doi/10.1073/pnas.2014564118
📍 Jeremy's Twitter thread summary of "An evidence review of face masks against COVID-19": https://twitter.com/jeremyphoward/status/1348771993949151232
📍 Read more about Jeremy's mask-wearing advocacy: expat-s-push-for-universal-mask-wearing-catches-fire-in-the-us-20200401-p54fu2.html">https://www.smh.com.au/world/north-america/australian-expat-s-push-for-universal-mask-wearing-catches-fire-in-the-us-20200401-p54fu2.html
---
Connect with Jeremy and fast.ai:
📍 Jeremy on Twitter: https://twitter.com/jeremyphoward
📍 fast.ai on Twitter: https://twitter.com/FastDotAI
📍 Jeremy on LinkedIn: https://www.linkedin.com/in/howardjeremy/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan
Jerome Pesenti is the former VP of AI at Meta, a tech conglomerate that includes Facebook, WhatsApp, and Instagram, and one of the most exciting places where AI research is happening today.
Jerome shares his thoughts on Transformers-based large language models, and why he's excited by the progress but skeptical of the term "AGI". Then, he discusses some of the practical applications of ML at Meta (recommender systems and moderation!) and dives into the story behind Meta's development of PyTorch. Jerome and Lukas also chat about Jerome's time at IBM Watson and in drug discovery.
Show notes (transcript and links): http://wandb.me/gd-jerome-pesenti
---
⏳ Timestamps:
0:00 Intro
0:28 Jerome's thought on large language models
12:53 AI applications and challenges at Meta
18:41 The story behind developing PyTorch
26:40 Jerome's experience at IBM Watson
28:53 Drug discovery, AI, and changing the game
36:10 The potential of education and AI
40:10 Meta and AR/VR interfaces
43:43 Why NVIDIA is such a powerhouse
47:08 Jerome's advice to people starting their careers
48:50 Going back to coding, the challenges of scaling
52:11 Outro
---
Connect with Jerome:
📍 Jerome on Twitter: https://twitter.com/an_open_mind
📍 Jerome on LinkedIn: https://www.linkedin.com/in/jpesenti/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
D. Sculley is CEO of Kaggle, the beloved and well-known data science and machine learning community.
D. discusses his influential 2015 paper "Machine Learning: The High Interest Credit Card of Technical Debt" and what the current challenges of deploying models in the real world are now, in 2022. Then, D. and Lukas chat about why Kaggle is like a rain forest, and about Kaggle's historic, current, and potential future roles in the broader machine learning community.
Show notes (transcript and links): http://wandb.me/gd-d-sculley
---
⏳ Timestamps:
0:00 Intro
1:02 Machine learning and technical debt
11:18 MLOps, increased stakes, and realistic expectations
19:12 Evaluating models methodically
25:32 Kaggle's role in the ML world
33:34 Kaggle competitions, datasets, and notebooks
38:49 Why Kaggle is like a rain forest
44:25 Possible future directions for Kaggle
46:50 Healthy competitions and self-growth
48:44 Kaggle's relevance in a compute-heavy future
53:49 AutoML vs. human judgment
56:06 After a model goes into production
1:00:00 Outro
---
Connect with D. and Kaggle:
📍 D. on LinkedIn: https://www.linkedin.com/in/d-sculley-90467310/
📍 Kaggle on Twitter: https://twitter.com/kaggle
---
Links:
📍 "Machine Learning: The High Interest Credit Card of Technical Debt" (Sculley et al. 2014): https://research.google/pubs/pub43146/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan, Anish Shah, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Emad Mostaque is CEO and co-founder of Stability AI, a startup and network of decentralized developer communities building open AI tools. Stability AI is the company behind Stable Diffusion, the well-known, open source, text-to-image generation model.
Emad shares the story and mission behind Stability AI (unlocking humanity's potential with open AI technology), and explains how Stability's role as a community catalyst and compute provider might evolve as the company grows. Then, Emad and Lukas discuss what the future might hold in store: big models vs "optimal" models, better datasets, and more decentralization.
-
🎶 Special note: This week’s theme music was composed by Weights & Biases’ own Justin Tenuto with help from Harmonai’s Dance Diffusion.
-
Show notes (transcript and links): http://wandb.me/gd-emad-mostaque
-
⏳ Timestamps:
00:00 Intro
00:42 How AI fits into the safety/security industry
09:33 Event matching and object detection
14:47 Running models on the right hardware
17:46 Scaling model evaluation
23:58 Monitoring and evaluation challenges
26:30 Identifying and sorting issues
30:27 Bridging vision and language domains
39:25 Challenges and promises of natural language technology
41:35 Production environment
43:15 Using synthetic data
49:59 Working with startups
53:55 Multi-task learning, meta-learning, and user experience
56:44 Optimization and testing across multiple platforms
59:36 Outro
-
Connect with Jehan and Motorola Solutions:
📍 Jehan on LinkedIn: https://www.linkedin.com/in/jehanw/
📍 Jehan on Twitter: https://twitter.com/jehan/
📍 Motorola Solutions on Twitter: https://twitter.com/MotoSolutions/
📍 Careers at Motorola Solutions: https://www.motorolasolutions.com/en_us/about/careers.html
-
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan, Lavanya Shukla, Anish Shah
-
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Jehan Wickramasuriya is the Vice President of AI, Platform & Data Services at Motorola Solutions, a global leader in public safety and enterprise security.
In this episode, Jehan discusses how Motorola Solutions uses AI to simplify data streams to help maximize human potential in high-stress situations. He also shares his thoughts on augmenting synthetic data with real data and the challenges posed in partnering with startups.
Show notes (transcript and links): http://wandb.me/gd-jehan-wickramasuriya
-
⏳ Timestamps:
00:00 Intro
00:42 How AI fits into the safety/security industry
09:33 Event matching and object detection
14:47 Running models on the right hardware
17:46 Scaling model evaluation
23:58 Monitoring and evaluation challenges
26:30 Identifying and sorting issues
30:27 Bridging vision and language domains
39:25 Challenges and promises of natural language technology
41:35 Production environment
43:15 Using synthetic data
49:59 Working with startups
53:55 Multi-task learning, meta-learning, and user experience
56:44 Optimization and testing across multiple platforms
59:36 Outro
-
Connect with Jehan and Motorola Solutions:
📍 Jehan on LinkedIn: https://www.linkedin.com/in/jehanw/
📍 Jehan on Twitter: https://twitter.com/jehan/
📍 Motorola Solutions on Twitter: https://twitter.com/MotoSolutions/
📍 Careers at Motorola Solutions: https://www.motorolasolutions.com/en_us/about/careers.html
-
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Cayla Sharp, Angelica Pan, Lavanya Shukla
-
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Will Falcon is the CEO and co-founder of Lightning AI, a platform that enables users to quickly build and publish ML models.
In this episode, Will explains how Lightning addresses the challenges of a fragmented AI ecosystem and reveals which framework PyTorch Lightning was originally built upon (hint: not PyTorch!) He also shares lessons he took from his experience serving in the military and offers a recommendation to veterans who want to work in tech.
Show notes (transcript and links): http://wandb.me/gd-will-falcon
---
⏳ Timestamps:
00:00 Intro
01:00 From SEAL training to FAIR
04:17 Stress-testing Lightning
07:55 Choosing PyTorch over TensorFlow and other frameworks
13:16 Components of the Lightning platform
17:01 Launching Lightning from Facebook
19:09 Similarities between leadership and research
22:08 Lessons from the military
26:56 Scaling PyTorch Lightning to Lightning AI
33:21 Hiring the right people
35:21 The future of Lightning
39:53 Reducing algorithm complexity in self-supervised learning
42:19 A fragmented ML landscape
44:35 Outro
---
Connect with Lightning
📍 Website: https://lightning.ai
📍 Twitter: https://twitter.com/LightningAI
📍 LinkedIn: https://www.linkedin.com/company/pytorch-lightning/
📍 Careers: https://boards.greenhouse.io/lightningai
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Anish Shah, Cayla Sharp, Angelica Pan, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Aaron Colak is the Leader of Core Machine Learning at Qualtrics, an experiment management company that takes large language models and applies them to real-world, B2B use cases.
In this episode, Aaron describes mixing classical linguistic analysis with deep learning models and how Qualtrics organized their machine learning organizations and model to leverage the best of these techniques. He also explains how advances in NLP have invited new opportunities in low-resource languages.
Show notes (transcript and links): http://wandb.me/gd-aaron-colak
---
⏳ Timestamps:
00:00 Intro
00:57 Evolving from surveys to experience management
04:56 Detecting sentiment with ML
10:57 Working with large language models and rule-based systems
14:50 Zero-shot learning, NLP, and low-resource languages
20:11 Letting customers control data
25:13 Deep learning and tabular data
28:40 Hyperscalers and performance monitoring
34:54 Combining deep learning with linguistics
40:03 A sense of accomplishment
42:52 Causality and observational data in healthcare
45:09 Challenges of interdisciplinary collaboration
49:27 Outro
---
Connect with Aaron and Qualtrics
📍 Aaron on LinkedIn: https://www.linkedin.com/in/aaron-r-colak-3522308/
📍 Qualtrics on Twitter: https://twitter.com/qualtrics/
📍 Careers at Qualtrics: https://www.qualtrics.com/careers/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Cayla Sharp, Angelica Pan, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Jordan Fisher is the CEO and co-founder of Standard AI, an autonomous checkout company that’s pushing the boundaries of computer vision.
In this episode, Jordan discusses “the Wild West” of the MLOps stack and tells Lukas why Rust beats Python. He also explains why AutoML shouldn't be overlooked and uses a bag of chips to help explain the Manifold Hypothesis.
Show notes (transcript and links): http://wandb.me/gd-jordan-fisher
---
⏳ Timestamps:
00:00 Intro
00:40 The origins of Standard AI
08:30 Getting Standard into stores
18:00 Supervised learning, the advent of synthetic data, and the manifold hypothesis
24:23 What's important in a MLOps stack
27:32 The merits of AutoML
30:00 Deep learning frameworks
33:02 Python versus Rust
39:32 Raw camera data versus video
42:47 The future of autonomous checkout
48:02 Sharing the StandardSim data set
52:30 Picking the right tools
54:30 Overcoming dynamic data set challenges
57:35 Outro
---
Connect with Jordan and Standard AI
📍 Jordan on LinkedIn: https://www.linkedin.com/in/jordan-fisher-81145025/
📍 Standard AI on Twitter: https://twitter.com/StandardAi
📍 Careers at Standard AI: https://careers.standard.ai/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Cayla Sharp, Angelica Pan, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Drago Anguelov is a Distinguished Scientist and Head of Research at Waymo, an autonomous driving technology company and subsidiary of Alphabet Inc.
We begin by discussing Drago's work on the original Inception architecture, winner of the 2014 ImageNet challenge and introduction of the inception module. Then, we explore milestones and current trends in autonomous driving, from Waymo's release of the Open Dataset to the trade-offs between modular and end-to-end systems.
Drago also shares his thoughts on finding rare examples, and the challenges of creating scalable and robust systems.
Show notes (transcript and links): http://wandb.me/gd-drago-anguelov
---
⏳ Timestamps:
0:00 Intro
0:45 The story behind the Inception architecture
13:51 Trends and milestones in autonomous vehicles
23:52 The challenges of scalability and simulation
30:19 Why LiDar and mapping are useful
35:31 Waymo Via and autonomous trucking
37:31 Robustness and unsupervised domain adaptation
40:44 Why Waymo released the Waymo Open Dataset
49:02 The domain gap between simulation and the real world
56:40 Finding rare examples
1:04:34 The challenges of production requirements
1:08:36 Outro
---
Connect with Drago & Waymo
📍 Drago on LinkedIn: https://www.linkedin.com/in/dragomiranguelov/
📍 Waymo on Twitter: https://twitter.com/waymo/
📍 Careers at Waymo: https://waymo.com/careers/
---
Links:
📍 Inception v1: https://arxiv.org/abs/1409.4842
📍 "SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation", Qiangeng Xu et al. (2021), https://arxiv.org/abs/2108.06709
📍 "GradTail: Learning Long-Tailed Data Using Gradient-based Sample Weighting", Zhao Chen et al. (2022), https://arxiv.org/abs/2201.05938
---
💬 Host: Lukas Biewald
📹 Producers: Cayla Sharp, Angelica Pan, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
James Cham is a co-founder and partner at Bloomberg Beta, an early-stage venture firm that invests in machine learning and the future of work, the intersection between business and technology.
James explains how his approach to investing in AI has developed over the last decade, which signals of success he looks for in the ever-adapting world of venture startups (tip: look for the "gradient of admiration"), and why it's so important to demystify ML for executives and decision-makers.
Lukas and James also discuss how new technologies create new business models, and what the ethical considerations of a world where machine learning is accepted to be possibly fallible would be like.
Show notes (transcript and links): http://wandb.me/gd-james-cham
---
⏳ Timestamps:
0:00 Intro
0:46 How investment in AI has changed and developed
7:08 Creating the first MI landscape infographics
10:30 The impact of ML on organizations and management
17:40 Demystifying ML for executives
21:40 Why signals of successful startups change over time
27:07 ML and the emergence of new business models
37:58 New technology vs new consumer goods
39:50 What James considers when investing
44:19 Ethical considerations of accepting that ML models are fallible
50:30 Reflecting on past investment decisions
52:56 Thoughts on consciousness and Theseus' paradox
59:08 Why it's important to increase general ML literacy
1:03:09 Outro
1:03:30 Bonus: How James' faith informs his thoughts on ML
---
Connect with James:
📍 Twitter: https://twitter.com/jamescham
📍 Bloomberg Beta: https://github.com/Bloomberg-Beta/Manual
---
Links:
📍 "Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions" by Ali Alkhatib and Michael Bernstein (2019): https://doi.org/10.1145/3290605.3300760
---
💬 Host: Lukas Biewald
📹 Producers: Cayla Sharp, Angelica Pan, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Check out this report by Boris about DALL-E mini:
https://twitter.com/weirddalle
Connect with Boris:
📍 Twitter: https://twitter.com/borisdayma
---
💬 Host: Lukas Biewald
📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Tristan Handy is CEO and founder of dbt Labs. dbt (data build tool) simplifies the data transformation workflow and helps organizations make better decisions.
Lukas and Tristan dive into the history of the modern data stack and the subsequent challenges that dbt was created to address; communities of identity and product-led growth; and thoughts on why SQL has survived and thrived for so long. Tristan also shares his hopes for the future of BI tools and the data stack.
Show notes (transcript and links): http://wandb.me/gd-tristan-handy
---
⏳ Timestamps:
0:00 Intro
0:40 How dbt makes data transformation easier
4:52 dbt and avoiding bad data habits
14:23 Agreeing on organizational ground truths
19:04 Staying current while running a company
22:15 The origin story of dbt
26:08 Why dbt is conceptually simple but hard to execute
34:47 The dbt community and the bottom-up mindset
41:50 The future of data and operations
47:41 dbt and machine learning
49:17 Why SQL is so ubiquitous
55:20 Bridging the gap between the ML and data worlds
1:00:22 Outro
---
Connect with Tristan:
📍 Twitter: https://twitter.com/jthandy
📍 The Analytics Engineering Roundup: https://roundup.getdbt.com/
---
💬 Host: Lukas Biewald
📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Johannes Otterbach is VP of Machine Learning Research at Merantix Momentum, an ML consulting studio that helps their clients build AI solutions.
Johannes and Lukas talk about Johannes' background in physics and applications of ML to quantum computing, why Merantix is investing in creating a cloud-agnostic tech stack, and the unique challenges of developing and deploying models for different customers. They also discuss some of Johannes' articles on the impact of NLP models and the future of AI regulations.
Show notes (transcript and links): http://wandb.me/gd-johannes-otterbach
---
⏳ Timestamps:
0:00 Intro
1:04 Quantum computing and ML applications
9:21 Merantix, Ventures, and ML consulting
19:09 Building a cloud-agnostic tech stack
24:40 The open source tooling ecosystem
30:28 Handing off models to customers
31:42 The impact of NLP models on the real world
35:40 Thoughts on AI and regulation
40:10 Statistical physics and optimization problems
42:50 The challenges of getting high-quality data
44:30 Outro
---
Connect with Johannes:
📍 LinkedIn: https://twitter.com/jsotterbach
📍 Personal website: http://jotterbach.github.io/
📍 Careers at Merantix Momentum: momentum.com/about#jobs">https://merantix-momentum.com/about#jobs
---
💬 Host: Lukas Biewald
📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Mircea Neagovici is VP, AI and Research at UiPath, where his team works on task mining and other ways of combining robotic process automation (RPA) with machine learning for their B2B products.
Mircea and Lukas talk about the challenges of allowing customers to fine-tune their models, the trade-offs between traditional ML and more complex deep learning models, and how Mircea transitioned from a more traditional software engineering role to running a machine learning organization.
Show notes (transcript and links): http://wandb.me/gd-mircea-neagovici
---
⏳ Timestamps:
0:00 Intro
1:05 Robotic Process Automation (RPA)
4:20 RPA and machine learning at UiPath
8:20 Fine-tuning & PyTorch vs TensorFlow
14:50 Monitoring models in production
16:33 Task mining
22:37 Trade-offs in ML models
29:45 Transitioning from software engineering to ML
34:02 ML teams vs engineering teams
40:41 Spending more time on data
43:55 The organizational machinery behind ML models
45:57 Outro
---
Connect with Mircea:
📍 LinkedIn: https://www.linkedin.com/in/mirceaneagovici/
📍 Careers at UiPath: https://www.uipath.com/company/careers
---
💬 Host: Lukas Biewald
📹 Producers: Cayla Sharp, Angelica Pan, Sanyam Bhutani, Lavanya Shukla
Jensen Huang is founder and CEO of NVIDIA, whose GPUs sit at the heart of the majority of machine learning models today.
Jensen shares the story behind NVIDIA's expansion from gaming to deep learning acceleration, leadership lessons that he's learned over the last few decades, and why we need a virtual world that obeys the laws of physics (aka the Omniverse) in order to take AI to the next era. Jensen and Lukas also talk about the singularity, the slow-but-steady approach to building a new market, and the importance of MLOps.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-jensen-huang
---
⏳ Timestamps:
0:00 Intro
0:50 Why NVIDIA moved into the deep learning space
7:33 Balancing the compute needs of different audiences
10:40 Quantum computing, Huang's Law, and the singularity
15:53 Democratizing scientific computing
20:59 How Jensen stays current with technology trends
25:10 The global chip shortage
27:00 Leadership lessons that Jensen has learned
32:32 Keeping a steady vision for NVIDIA
35:48 Omniverse and the next era of AI
42:00 ML topics that Jensen's excited about
45:05 Why MLOps is vital
48:38 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Peter Welinder is VP of Product & Partnerships at OpenAI, where he runs product and commercialization efforts of GPT-3, Codex, GitHub Copilot, and more. Boris Dayma is Machine Learning Engineer at Weights & Biases, and works on integrations and large model training.
Peter, Boris, and Lukas dive into the world of GPT-3:
- How people are applying GPT-3 to translation, copywriting, and other commercial tasks
- The performance benefits of fine-tuning GPT-3-
- Developing an API on top of GPT-3 that works out of the box, but is also flexible and customizable
They also discuss the new OpenAI and Weights & Biases collaboration, which enables a user to log their GPT-3 fine-tuning projects to W&B with a single line of code.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-peter-and-boris
---
Connect with Peter & Boris:
📍 Peter's Twitter: https://twitter.com/npew
📍 Boris' Twitter: https://twitter.com/borisdayma
---
⏳ Timestamps:
0:00 Intro
1:01 Solving real-world problems with GPT-3
6:57 Applying GPT-3 to translation tasks
14:58 Copywriting and other commercial GPT-3 applications
20:22 The OpenAI API and fine-tuning GPT-3
28:22 Logging GPT-3 fine-tuning projects to W&B
38:25 Engineering challenges behind OpenAI's API
43:15 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Ion Stoica is co-creator of the distributed computing frameworks Spark and Ray, and co-founder and Executive Chairman of Databricks and Anyscale. He is also a Professor of computer science at UC Berkeley and Principal Investigator of RISELab, a five-year research lab that develops technology for low-latency, intelligent decisions.
Ion and Lukas chat about the challenges of making a simple (but good!) distributed framework, the similarities and differences between developing Spark and Ray, and how Spark and Ray led to the formation of Databricks and Anyscale. Ion also reflects on the early startup days, from deciding to commercialize to picking co-founders, and shares advice on building a successful company.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-ion-stoica
---
Timestamps:
0:00 Intro
0:56 Ray, Anyscale, and making a distributed framework
11:39 How Spark informed the development of Ray
18:53 The story behind Spark and Databricks
33:00 Why TensorFlow and PyTorch haven't monetized
35:35 Picking co-founders and other startup advice
46:04 The early signs of sky computing
49:24 Breaking problems down and prioritizing
53:17 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Stephan Fabel is Senior Director of Infrastructure Systems & Software at NVIDIA, where he works on Base Command, a software platform to coordinate access to NVIDIA's DGX SuperPOD infrastructure.
Lukas and Stephan talk about why having a supercomputer is one thing but using it effectively is another, why a deeper understanding of hardware on the practitioner level is becoming more advantageous, and which areas of the ML tech stack NVIDIA is looking to expand into.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-stephan-fabel
---
Timestamps:
0:00 Intro
1:09 NVIDIA Base Command and DGX SuperPOD
10:33 The challenges of multi-node processing at scale
18:35 Why it's hard to use a supercomputer effectively
25:14 The advantages of de-abstracting hardware
29:09 Understanding Base Command's product-market fit
36:59 Data center infrastructure as a value center
42:13 Base Command's role in tech stacks
47:16 Why crowdsourcing is underrated
49:24 The challenges of scaling beyond a POC
51:39 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Chris Padwick is Director of Computer Vision Machine Learning at Blue River Technology, a subsidiary of John Deere. Their core product, See & Spray, is a weeding robot that identifies crops and weeds in order to spray only the weeds with herbicide.
Chris and Lukas dive into the challenges of bringing See & Spray to life, from the hard computer vision problem of classifying weeds from crops, to the engineering feat of building and updating embedded systems that can survive on a farming machine in the field. Chris also explains why user feedback is crucial, and shares some of the surprising product insights he's gained from working with farmers.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-chris-padwick
---
Connect with Chris:
📍 LinkedIn: https://www.linkedin.com/in/chris-padwick-75b5761/
📍 Blue River on Twitter: https://twitter.com/BlueRiverTech
---
Timestamps:
0:00 Intro
1:09 How does See & Spray reduce herbicide usage?
9:15 Classifying weeds and crops in real time
17:45 Insights from deployment and user feedback
29:08 Why weed and crop classification is surprisingly hard
37:33 Improving and updating models in the field
40:55 Blue River's ML stack
44:55 Autonomous tractors and upcoming directions
48:05 Why data pipelines are underrated
52:10 The challenges of scaling software & hardware
54:44 Outro
55:55 Bonus: Transporters and the singularity
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Kathryn Hume is Vice President Digital Investments Technology at the Royal Bank of Canada (RBC). At the time of recording, she was Interim Head of Borealis AI, RBC's research institute for machine learning.
Kathryn and Lukas talk about ML applications in finance, from building a personal finance forecasting model to applying reinforcement learning to trade execution, and take a philosophical detour into the 17th century as they speculate on what Newton and Descartes would have thought about machine learning.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-kathryn-hume
---
Connect with Kathryn:
📍 Twitter: https://twitter.com/humekathryn
📍 Website: https://quamproxime.com/
---
Timestamps:
0:00 Intro
0:54 Building a personal finance forecasting model
10:54 Applying RL to trade execution
18:55 Transparent financial models and fairness
26:20 Semantic parsing and building a text-to-SQL interface
29:20 From comparative literature and math to product
37:33 What would Newton and Descartes think about ML?
44:15 On sentient AI and transporters
47:33 Why casual inference is under-appreciated
49:25 The challenges of integrating models into the business
51:45 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Sean McClain is the founder and CEO, and Gregory Hannum is the VP of AI Research at Absci, a biotech company that's using deep learning to expedite drug discovery and development.
Lukas, Sean, and Greg talk about why Absci started investing so heavily in ML research (it all comes back to the data), what it'll take to build the GPT-3 of DNA, and where the future of pharma is headed. Sean and Greg also share some of the challenges of building cross-functional teams and combining two highly specialized fields like biology and ML.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-sean-and-greg
---
Connect with Sean and Greg:
📍 Sean's Twitter: https://twitter.com/seanrmcclain
📍 Greg's Twitter: https://twitter.com/gregory_hannum
📍 Absci's Twitter: https://twitter.com/abscibio
---
Timestamps:
0:00 Intro
0:53 How Absci merges biology and AI
11:24 Why Absci started investing in ML
19:00 Creating the GPT-3 of DNA
25:34 Investing in data collection and in ML teams
33:14 Clinical trials and Absci's revenue structure
38:17 Combining knowledge from different domains
45:22 The potential of multitask learning
50:43 Why biological data is tricky to work with
55:00 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
You might know him as the host of Gradient Dissent, but Lukas is also the CEO of Weights & Biases, a developer-first ML tools platform!
In this special episode, the three W&B co-founders — Chris (CVP), Shawn (CTO), and Lukas (CEO) — sit down to tell the company's origin stories, reflect on the highs and lows, and give advice to engineers looking to start their own business.
Chris reveals the W&B server architecture (tl;dr - React + GraphQL), Shawn shares his favorite product feature (it's a hidden frontend layer), and Lukas explains why it's so important to work with customers that inspire you.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-wandb-cofounders
---
Connect with us:
📍 Chris' Twitter: https://twitter.com/vanpelt
📍 Shawn's Twitter: https://twitter.com/shawnup
📍 Lukas' Twitter: https://twitter.com/l2k
📍 W&B's Twitter: https://twitter.com/weights_biases
---
Timestamps:
0:00 Intro
1:29 The stories behind Weights & Biases
7:45 The W&B tech stack
9:28 Looking back at the beginning
11:42 Hallmark moments
14:49 Favorite product features
16:49 Rewriting the W&B backend
18:21 The importance of customer feedback
21:18 How Chris and Shawn have changed
22:35 How the ML space has changed
28:24 Staying positive when things look bleak
32:19 Lukas' advice to new entrepreneurs
35:29 Hopes for the next five years
38:09 Making a paintbot & model understanding
41:30 Biggest bottlenecks in deployment
44:08 Outro
44:38 Bonus: Under- vs overrated technologies
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Pete is the Technical Lead of the TensorFlow Micro team, which works on deep learning for mobile and embedded devices.
Lukas and Pete talk about hacking a Raspberry Pi to run AlexNet, the power and size constraints of embedded devices, and techniques to reduce model size. Pete also explains real world applications of TensorFlow Lite Micro and shares what it's been like to work on TensorFlow from the beginning.
The complete show notes (transcript and links) can be found here: http://wandb.me/gd-pete-warden
---
Connect with Pete:
📍 Twitter: https://twitter.com/petewarden
📍 Website: https://petewarden.com/
---
Timestamps:
0:00 Intro
1:23 Hacking a Raspberry Pi to run neural nets
13:50 Model and hardware architectures
18:56 Training a magic wand
21:47 Raspberry Pi vs Arduino
27:51 Reducing model size
33:29 Training on the edge
39:47 What it's like to work on TensorFlow
47:45 Improving datasets and model deployment
53:05 Outro
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Pieter is the Chief Scientist and Co-founder at Covariant, where his team is building universal AI for robotic manipulation. Pieter also hosts The Robot Brains Podcast, in which he explores how far humanity has come in its mission to create conscious computers, mindful machines, and rational robots.
Lukas and Pieter explore the state of affairs of robotics in 2021, the challenges of achieving consistency and reliability, and what it'll take to make robotics more ubiquitous. Pieter also shares some perspective on entrepreneurship, from how he knew it was time to commercialize Gradescope to what he looks for in co-founders to why he started Covariant.
Show notes: http://wandb.me/gd-pieter-abbeel
---
Connect with Pieter:
📍 Twitter: https://twitter.com/pabbeel
📍 Website: https://people.eecs.berkeley.edu/~pabbeel/
📍 The Robot Brains Podcast: https://www.therobotbrains.ai/
---
Timestamps:
0:00 Intro
1:15 The challenges of robotics
8:10 Progress in robotics
13:34 Imitation learning and reinforcement learning
21:37 Simulated data, real data, and reliability
27:53 The increasing capabilities of robotics
36:23 Entrepreneurship and co-founding Gradescope
44:35 The story behind Covariant
47:50 Pieter's communication tips
52:13 What Pieter's currently excited about
55:08 Focusing on good UI and high reliability
57:01 Outro
In this episode we're joined by Chris Albon, Director of Machine Learning at the Wikimedia Foundation.
Lukas and Chris talk about Wikimedia's approach to content moderation, what it's like to work in a place so transparent that even internal chats are public, how Wikimedia uses machine learning (spoiler: they do a lot of models to help editors), and why they're switching to Kubeflow and Docker. Chris also shares how his focus on outcomes has shaped his career and his approach to technical interviews.
Show notes: http://wandb.me/gd-chris-albon
---
Connect with Chris:
- Twitter: https://twitter.com/chrisalbon
- Website: https://chrisalbon.com/
---
Timestamps:
0:00 Intro
1:08 How Wikimedia approaches moderation
9:55 Working in the open and embracing humility
16:08 Going down Wikipedia rabbit holes
20:03 How Wikimedia uses machine learning
27:38 Wikimedia's ML infrastructure
42:56 How Chris got into machine learning
46:43 Machine Learning Flashcards and technical interviews
52:10 Low-power models and MLOps
55:58 Outro
In this episode, Emily and Lukas dive into the problems with bigger and bigger language models, the difference between form and meaning, the limits of benchmarks, and why it's important to name the languages we study.
Show notes (links to papers and transcript): http://wandb.me/gd-emily-m-bender
---
Emily M. Bender is a Professor of Linguistics at and Faculty Director of the Master's Program in Computational Linguistics at University of Washington. Her research areas include multilingual grammar engineering, variation (within and across languages), the relationship between linguistics and computational linguistics, and societal issues in NLP.
---
Timestamps:
0:00 Sneak peek, intro
1:03 Stochastic Parrots
9:57 The societal impact of big language models
16:49 How language models can be harmful
26:00 The important difference between linguistic form and meaning
34:40 The octopus thought experiment
42:11 Language acquisition and the future of language models
49:47 Why benchmarks are limited
54:38 Ways of complementing benchmarks
1:01:20 The #BenderRule
1:03:50 Language diversity and linguistics
1:12:49 Outro
Jeff talks about building Facebook's early data team, founding Cloudera, and transitioning into biomedicine with Hammer Lab and Related Sciences.
(Read more: http://wandb.me/gd-jeff-hammerbacher)
---
Jeff Hammerbacher is a scientist, software developer, entrepreneur, and investor. Jeff's current work focuses on drug discovery at Related Sciences, a biotech venture creation firm that he co-founded in 2020.
Prior to his work at Related Sciences, Jeff was the Principal Investigator of Hammer Lab, a founder and the Chief Scientist of Cloudera, an Entrepreneur-in-Residence at Accel, and the manager of the Data team at Facebook.
---
Follow Gradient Dissent on Twitter: https://twitter.com/weights_biases
---
0:00 Sneak peek, intro
1:13 The start of Facebook's data science team
6:53 Facebook's early tech stack
14:20 Early growth strategies at Facebook
17:37 The origin story of Cloudera
24:51 Cloudera's success, in retrospect
31:05 Jeff's transition into biomedicine
38:38 Immune checkpoint blockade in cancer therapy
48:55 Data and techniques for biomedicine
53:00 Why Jeff created Related Sciences
56:32 Outro
Josh explains how astronomy and machine learning have informed each other, their current limitations, and where their intersection goes from here.
(Read more: http://wandb.me/gd-josh-bloom)
---
Josh is a Professor of Astronomy and Chair of the Astronomy Department at UC Berkeley. His research interests include the intersection of machine learning and physics, time-domain transients events, artificial intelligence, and optical/infared instrumentation.
---
Follow Gradient Dissent on Twitter: https://twitter.com/weights_biases
---
0:00 Intro, sneak peek
1:15 How astronomy has informed ML
4:20 The big questions in astronomy today
10:15 On dark matter and dark energy
16:37 Finding life on other planets
19:55 Driving advancements in astronomy
27:05 Putting telescopes in space
31:05 Why Josh started using ML in his research
33:54 Crowdsourcing in astronomy
36:20 How ML has (and hasn't) informed astronomy
47:22 The next generation of cross-functional grad students
50:50 How Josh started coding
56:11 Incentives and maintaining research codebases
1:00:01 ML4Science's tech stack
1:02:11 Uncertainty quantification in a sensor-based world
1:04:28 Why it's not good to always get an answer
1:07:47 Outro
This podcast could use a review! Have anything to say about it? Share your thoughts using the button below.
Submit Review
