"Ultimately, Tim and I know Modular is a moonshot, but that makes it very exciting. And personally, it gives me enormous satisfaction knowing that if my high school self went down a rabbit hole today, I'd do it with Modular tools."
The computing industry has followed a simple but powerful long-term growth pattern for decades: Hardware turns into software, and software turns into open-source-based services. It's an inexorable march down this path for technology innovation, market power, and profits.
As it has been for general-purpose computers, platform software, networking, and many application software domains – culminating in cloud computing – so will the AI world follow a similar path. But today we're very much in the AI hardware era, as companies like NVIDIA have racked up nearly $100 billion in profit over the last four quarters. Morgan Stanley predicts that $2.9 trillion will be spent on global data center construction and AI infrastructure buildout between 2025 and 2028. The expansion is so aggressive that traditionally non-issues—like access to power, electric grid capacity, and cooling technologies—are becoming significant bottlenecks.
These are real and material challenges that are constraining and shaping value creation, and ultimately, the winners in the AI world. The institutions adopting AI and applying AI infrastructure face an even bigger challenge: How to increase the fungibility of their huge AI investments across platforms and clouds as different hardware technologies and suppliers win and lose?
Chris Lattner co-founded Modular with Tim Davis to accelerate the adoption of simpler, more open AI software programming technologies, with an eye toward expanding AI value creation across industries, company sizes, and users.
"The benefit flows of AI shouldn't hinge on who can handle the complexity and risk of AI hardware and infrastructure choices," said Lattner. "If we can break down happenstance AI infrastructure walls and crack open new forms of AI compute – making AI more 'normal' – then we'll get a higher-quality, better-engineered AI out there. That's what I want to be a part of."
Expanding access to AI by increasing openness, commonality, and usability is the key to turning AI experiments into amazing AI-based products and experiences. Moreover, the more open the approaches, the greater the introspection will be into core AI technologies, which will lead to leaps in AI safety and predictability on multiple dimensions, including the growth of the AI footprint on the world's scarce resources.
Q: You grew up in rural Oregon on a small farm. How did you get started on a journey that took you through senior positions at Apple, Google, and Tesla to a deep technology startup like Modular?
Yeah, I did grow up in rural Oregon, in Banks, on a small farm. Basically, you drive west from Portland until just before you hit the mountains and stop. Both my parents worked. I had a loving family, and it was wonderful, but my days revolved around long, "can't miss or you walk" bus rides to and from school and entertaining myself. My parents decided to get me a Commodore 64, which they acquired as a takeaway gift for attending a condo timeshare presentation. I admit, I fell down the rabbit hole. I started by learning how to do basic programming, and I kept going, acquiring more programming knowledge and skills, pretty much until I ran out of memory, so to speak. Eventually, my parents got a PC, which I guess you could say was the first time I busted through a computing barrier. My mom's career grew, and she ended up in HR, which, at the time in her place of employment, oversaw IT. So, I got to help her out with some bigger computing things. My dad was at NCR, but not on the computing side, the programming side. He spent years servicing those mechanically complex NCR cash registers. He definitely had a strong technical streak but stayed in the mechanical world.
Q: Would you say that your parents catalyzed your desire to become an engineer?
Well, I like to understand and build things. That's been the consistent thing. And I find that I understand things best when I build them. That's my virtuous cycle. But certainly, my parents and their interests and work were a big influence. As I said, my dad has always been very mechanical. He's raced motorcycles and flown airplanes and done all these very exotic-sounding things that always emerged out of his desire to understand how things work. I think curiosity is in my DNA, but I had computers. I didn't follow a path from "what causes lift" to flying airplanes. I was more like, "How are these machines communicating over this phone-based modem?" to programming complex systems that solved real problems.
I entered computing communities of the day, like bulletin boards and user groups, which gave me access to tools and other downloadable software. Eventually, that all opened up, and the internet happened. So, I went from "how does the internet work" to getting a Linux computer in the mid-'90s and jumping headfirst into the early internet era. The openness of those early years was just great, and so essential to the innovation that followed. I could log into other sites, understand what's going on, and learn from other people. I came to realize that the great thing about software is that the primary thing separating your imagination from your reality is your programming knowledge and skill. I attacked that barrier aggressively and with great ambition.
Q: How did you go from high school hobbyist to broadly respected software engineer?
Countless people have shaped the journey across my life and career. A ton of people have influenced me. I can't say I followed some grand plan. I'd meet folks who interested me, and they subsequently shaped my interests. In college, for example, I had a professor who really loved compilers. Now, compilers aren't everyone's cup of tea, but his enthusiasm rubbed off on me, which is why I ended up going to graduate school and obsessing over low-level virtual machine (LLVM) technologies.
Happily, LLVM became a really important technology as non-Intel-based minicomputer and PC companies tried to plot more open futures. My "niche" interests directly led to a job at Apple. And at Apple, I guzzled from the firehouse, learning a tremendous amount about leadership, about building and shipping products, about building commercial software, about inspiring people, and actually building things that could touch the world.
Q: Tell me about your experience with LLVM. When did you realize your PhD thesis and subsequent professional life were focused on something that would become foundational to modern computing?
There was never an "Aha!" moment. When you're embedded in a creative process that takes years, you're not asking, "What does the outcome look like?" You're asking, "What does the next incremental win look like?" But those incremental wins generate compound interest, pun intended. The more interesting and useful problems you solve, the more interesting and important questions you get to ask. That's how your scope gets bigger and bigger. Open source has been really important to my journey. When I encountered a challenge, open source allowed me to find a contributor who had started solving the problem. When I had a head start on solving the problem, I could implement a solution without worrying about navigating complex networks of copyrights. When I was part of a team that was figuring out how to present that solution to the market as a problem, we could focus on the challenges our customers would face exploiting the product, not so much on how we established limits intended to protect our intellectual property.
This notion of compounding software value from open source really is important. The folks that own or manage established systems, the stuff that represents today's "best practices" and "mature use cases," always look at an incremental step and ask, "Why put 95% of this stuff that works at risk to bet on your 5% incremental advance?" And the answer is that those 5% gains happen so much more quickly and frequently and cheaply and broadly that they compound and overwhelm existing streams of value. Suddenly, your sum of 5% increments can do something that nothing else can do. Honestly, Modular is going through this right now! It's a super fun company growth phase to be in, but I have to admit that when you're in one of those 5% moments, everything seems like it's in peril.
Q: Can you give an example of all this in action?
The Apple iPhone 5S. An absolutely transformative product. Smartphones had been around for, like, six years. The 5S had the first 64-bit chip in a smartphone, user-friendly Touch ID, and deeply integrated motion processing. Competitors up and down the stack dismissed the 5S as a collection of marketing gimmicks. Everybody thought it was impossible. But we, that is, Apple, when I was there, realized that the situation in software had changed. Because of LLVM and open source, we could adapt our software at market speed. Everybody thought it was stupid? Boom! That was our cue to do it.
We—as in the market generally and Modular specifically—are still riding that wave of software advances. I'm getting excited thinking about it. LLVMs have passed through several epochs. By the time I graduated from Illinois, LLVM was in a very advanced research project stage. About five years later, Google adopted it and started contributing heavily. That was a huge turning point. The project went from many small contributors to even more small and some really big contributors, like Google and then Apple. It was phenomenal, especially given that Google and Apple don't always get along. So, I guess it was phenomenal and interesting.
Then later, LLVM ended up replacing a lot of the Intel compiler and ARM compiler technologies. I mean, hardware vendors typically followed 20-year plans for advancing these technologies in-house. Huge institutional memory and investment momentum all undone when they realized that LLVM just worked better. You asked earlier about Aha! moments. I often wonder what it was like at established computer companies when they realized that they were driving a Gremlin in a drag race. Very publicly, I might add. What was the experience of realizing that an "open" group had advanced the technology you'd been working on for 20 years by 10x without any help from you? Suddenly, people that mattered, like customers, are like, "Wow, this is amazing and awesome," and the whole market consolidates around the "obvious" thing that months before was being discounted by the Gremlin drivers.
But I digress. Today, LLVM continues growing in strength. Communities are getting more influential, and the number of people working on them is rapidly expanding. Today, LLVM developer meetings gather hundreds of people together from all kinds of different companies, organizations, and universities multiple times a year. It's just phenomenal to see that community in action. It's very exciting to me, personally.
Obviously, the project's grown much bigger than me and my PhD thesis. I'm still one of the biggest contributors to it, but I'm most proud that LLVM is attracting very passionate, very strong engineers at very different institutions to come together and agree on building something together that's technically excellent despite their corporate politics. Today, LLVM powers CUDA; it powers all the Apple devices, it powers all of Google's data centers and Meta's data centers, which constitute a sizable portion of the cloud. I think that's just magical.
Q: Today, you are the co-founder and CEO at Modular. Do you see parallels between the evolution of open-source LLVM and company-building at Modular?
Yeah, so I'm a very ambitious person; I always want to solve the world's problems. But I've learned you must do things incrementally, one step at a time. I love every step of building something, including Modular. I love the drawing board, where there's nothing but ambiguity. There's nothing you can't do, but everything seems impossible. It's a weird state to be in: You simultaneously can and can't do anything.
The art is figuring out how to attack a huge problem. Some people just want the outcome, they just value the end result, but I think the journey is a huge reward. I love it when I'm saying, "I face this really complicated set of things, how do I factor the right way?" I love embarking on the consensus approach and adjusting every step along the way. I love the revealing of the outcome, when it all clicks, becomes easy and simple, and knowing we can scale our new and improved state.
Each of those moments is completely different. You have to do all the work. I think that great outcomes always take time. It takes years to do things that are big in a way that is correct and lasting.
You have to build the structure beneath you. You need the funding, you need the talent, you need the culture, you need the team, you need to be able to inspire people, you need to break through the politics. You need to solve each of these problems to make something like that successful. But when we get that right, it can be really transformative, and it can be very fun.
Q: We have your LLVM background. What about your experience with computing languages, which is another big part of Modular?
As I said, I joined Apple straight out of my PhD program. I joined as an engineer, and I was fortunate that during my time there, I rose to the executive management ranks and became a member of the elite circle who got to go off with Tim Cook every year to talk about the company's three-year roadmap in secret.
I was very privileged and very fortunate to be a part of that group. The journey was just remarkable. It was during the meteoric rise of the iPhone and the associated rise of Apple. Developer communities absolutely helped fuel that rise. Apple went from engaging with a passionate, but rather oddball, community of Mac and Objective-C developers to an iPhone developer community that was hundreds of times larger and very commercially focused. The App Store was ground zero for a lot of application development. It just completely changed the game in terms of the requirements.
Apple knew that this new community of iPhone developers valued speed and profit; we sensed that we needed a new platform if we wanted to lift and propel our iPhone – and Apple overall – developer community. But a lot of execs at Apple believed that any effort to modernize the stack wasn't worth the risk of dislocating the ecosystem's software portfolio.
Enter the Swift programming language. It certainly didn't emerge from the executive gatherings. Rather, it was kind of my personal side hustle. I worked on Swift as a weekend project for a year and a half before I showed it to my manager. Once I did, though, the technical team pretty quickly saw the benefits for app developers and the overall ecosystem. Marketing execs? Not so much. To them, it was an unnecessary risk. But Apple's culture truly did care about anticipating and solving our ecosystem's challenges. That's a big part of its culture, and it helped me convince folks that the downside risk was low while the upside potential was high. I learned a lot about influencing an executive team, for sure. Lots of egos. Lots of agendas. But with a lot of cajoling, I, with my growing mass of Swift adoptees inside Apple, convinced enough folks that Swift could be the next big thing. We got it launched in 2014. Big day!
Q: Are you applying those lessons to diffusing Modular's Mojo programming language in the AI market?
Well, the hardest part of shepherding a programming language actually is that programmers are smart and typically time constrained. Consequently, they all have strongly held opinions. You must operate with a set of easily communicated and agreed-upon principles to guide what you're doing. A collection of lights that constitute a clear North Star. And, crucially, you have to be able to say no to people when a yes takes you outside the bounds of those principles. Programmers get very passionate about programming languages. They spend their days and nights with them. Their livelihoods and comforts can be based on them. This is big stuff for programmers. One of the benefits of an open-source approach to changing programming languages is that programmers don't feel coerced; instead, they can feel more personally connected to the changes.
So, yeah, I'm applying a lot of these lessons to diffusing Mojo. We've open-sourced the Mojo standard library, language specs, and documentation, which encourages adoption and subsequent reinvention. But we're staying a bit more principled about how we evolve development of the core compiler technology, for both technical and business reasons. That's our North Star, for now.
Q: How did you meet your co-founder, Tim Davis?
In 2016, a funny thing happened. I was playing around in Apple's Photos app. I thought it was really cool that it could tell me that there's a dog or a cat in the picture. By this time, I was head of developer tools at Apple, which was a pretty big and interesting job, but I also had that DNA that demanded to understand how something worked – and I had no idea how to write code to build that function. Turns out the answer is AI and neural nets, but it became a bee in my bonnet.
And so, in 2016, I fell down another rabbit hole, this one being "What is this AI thing?" I got AI pilled, which was shocking to me. At the time, Apple was less interested in pursuing that arena, so I started wondering where I might be able to immerse myself in this burgeoning AI thing. That led to Tesla, where I ran the autopilot software team for self-driving cars. Talk about factoring a big problem! I learned a lot at Tesla. We spent a lot of time trying to get the TPU software, which was central to the whole autopilot effort, to work. We had a lot of grumpy developers at Tesla; folks just agitated that the TPU software stack wasn't well done or reliable. We made some pretty significant advances in that domain, and I learned a tremendous amount about AI, applied AI, and AI research. But the other thing I learned at Tesla was that I didn't want to work at Tesla.
So, I decided to join another company that was way ahead in AI: Google. At Google, two great things happened. First, I joined TensorFlow early and helped scale the TPU software platform, which is central to its big data center accelerator program. As part of that effort, we built MLIR, a new compiler platform that's used by all the AI infrastructure companies, including NVIDIA. We open-sourced that to the LLVM community, so conceptually it became LLVM 2.0.
Second, I met Tim. We became great co-workers, but even more, great friends. I was the headstrong engineer building stuff. Tim offered a deep product management perspective that forced a constant mapping from technology to customer. Really, we were "forced introduced," as in management telling Tim, "Chris needs help. Go help him." You can imagine my headstrong engineer's response to that!
But I got over it pretty quickly. Tim used to say to me, "Well, Chris, it doesn't matter if you're right. What matters is if other people understand that you're right." Sobering – and ultimately soothing – words for someone who can get too deep into the "right way or the highway" weeds. I intrinsically knew he was right. If you want lasting change, you must have people want what you're building and be willing to invest their time to pull you to understand their requirements. Building the thing isn't enough if your goal is market adoption, and that has always been my goal. He and I got off to a great relationship because our perspective and skills complement each other so well.
Q: How did you come to start Modular?
I'd constructed the core of LLVM, Swift, and I’d been a big part of simplifying TPU programming with TensorFlow. Also, where I worked, I'd been a technology contributor who built things and an executive who shepherded things. I realized that I wanted to combine all that experience, but bias my time a bit more to building things. And that got me into founder mode, where I could build technologies and a company simultaneously.
When I landed in founder mode, I knew I couldn't do it inside YouTube or Waymo or something like that. I knew it needed to be a VC-backed startup. We needed to be able to attract amazing people. We needed to be able to hire them from Apple, Google, Meta, Tesla, and all these companies, and so we needed to raise significant capital.
But I also learned that a startup must constantly map its technology to customers, as Tim Davis always preached. When I decided to leave Google, Tim stayed and continued his path. But in late 2021, we connected and started brainstorming about how best to provide a more open, hardware-independent target for more easily developed AI software. We knew that the AI world needed to solve the programmer side of AI before it truly could serve the complete customer side of AI. We knew it was going to require a long-term investment in effort and capital. It confirmed my belief that the only approach that could work was a VC-backed startup. And our conversations confirmed our partnership. So, we founded Modular.
Q: Modular comprises many huge ideas. How is it going?
It's going great. Two things are blowing a huge wind into our sails. First, the AI technical community has come to realize that technology like ours is essential. The AI world knows we need an LLVM-like platform that can facilitate squeezing the last drops of performance out of any AI-capable hardware without extensive and unique software migrations and rewrites. It also knows that without an AI-specific language, developers will grow even grumpier as they try to make these technologies work.
It's an especially impossible problem set. Very complicated. Requires very rare and unique skills. Takes a long time. The risk of failure is huge. Nobody has built an AI programming platform and language, except us. Not Nvidia. Not Microsoft. Not Tesla. Not Google. Only Modular, because we understand the problem and have amassed the capital, talent, and industry connections required to get it done.
But, like I said before, any lasting solution must map to customer needs, and the second wind in our sails is the emergence of ChatGPT and related technologies and companies as not just an industry phenomenon, but a cultural phenomenon. Every business executive on the planet, and a lot of consumers, are spending their days imagining what they can do with AI. But, like my dad and his airplanes and me with my Commodore 64, they can't turn what they imagine into real stuff without the tools that help them understand and build things. Ultimately, Tim and I know Modular is a moonshot, but that makes it very exciting. And personally, it gives me enormous satisfaction knowing that if my high school self went down a rabbit hole today, I'd do it with Modular tools.
