TheChek News Flow

The Takeaway

Vinod Khosla thinks companies which say they’re struggling to get a return on their spending on artificial-intelligence services may be doing it wrong. In an interview with The Information’s TITV this past week, he said “most enterprises who are executing AI are doing it with their people who are not qualified to execute.” In contrast, when businesses hire an AI specialist firm to help them, “it goes swimmingly well.”

Khosla is one of the most experienced venture capitalists in Silicon Valley, whose firm was a very early investor in OpenAI, among many others. That makes his take on the state of play in AI worth listening to. In the TITV interview, he talked about current developments in AI, the risks inherent in the AI ecosystem right now, power generation and other topics.

Below is a lightly edited transcript of the interview. The full video is here.

Vinod Khosla: Well, to start with, I think if you’re going to do something significant, you’re going to have to be an optimist. But not only just an optimist, a knowledgeable optimist. There are too many skeptics who live in the past and will extrapolate the past to invent the future. That’s a bad thing to do. Inventing the future you want is the right way, and optimism is key to that.

Having said that, I think the one thing I don’t know and there’s things I can reasonably project and things I can be uncertain about. The thing I’m uncertain about is timing of various breakthroughs that we expect to see in the next two years, five years, ten years. Almost certainly it’s easier to predict AI in 2035 and probably 2030, than it is to predict it today and what will happen next year. So I’m not skeptical, but I’m less certain about the pace of development of certain capabilities that are essential to expanding use of AI.

Vinod Khosla: I think the way to look at margins is when AI is a substantial business. Yes, it’s [a] large business, tens of billions of dollars, but it’s not hundreds of billions of dollars. And so the question to ask is where will margins stabilize? I think some of it depends on R&D breakthroughs by the various companies and knowing who gets what breakthrough, which is, as I said, uncertain with respect to who and timing. I’m pretty optimistic about what OpenAI is doing. I do think if you provide value and you have differentiated models, you will have good margins. Look, at the pace at which pricing is declining, both in supplying inferencing to the AI companies and the price at which things are sold, it’s hard to predict the dynamics. But it will settle down [with] good returns on capital invested. So I’d be surprised if we don’t have healthy increasing margins well into the early 2030s.

Vinod Khosla: So there’s a number of ways in which cost of supplying inference will go down. One, chips will become more economical in per inference. I do think the algorithms will get much better, which means the software will get better in the amount of compute needed per inference. So there’s two vectors. You can make the algorithms 10X more efficient over time. I think that will happen. The cost of chips will go down for a given number of inferences. So both those are vectors for cost reduction.

The question of pricing on the input side, what customers will pay, will be a function of value. If Amjad—and I love that company and what Amjad is doing—if he keeps adding more and more value, he will be able to charge more. So that’s a question of value addition. And that’s where there’s lots of headroom. You know when you’re kicking something that costs, say, a professional, an accounting professional, for example, or a design product designer, and you’re paying them $100 to $300 an hour and your cost is $1 to $3 an hour, you have lots of pricing room if you can provide more complete solutions. So I do think, price per hour of worker-time equivalent will increase pretty dramatically. Also as well, as well the decline in costs per inference. This business will be healthy starting, let’s say, 2030 and beyond, when scale data centers will be in operation.

Vinod Khosla: So you have to look at a couple of factors. One, where is the value being provided great. In software development it’s absolutely great value. So companies like Replit and Cognition, which we are both investors in, Cursor included, all growing very, very rapidly, because they’re adding real value. So there are functions in which the product isn’t complete or mature. And so you need almost so much hand-holding, the economics break down. I would say another factor that most people haven’t considered, most enterprises who are executing AI are doing it with their people who are not qualified to execute. It’s like saying, hey, we have a race car and Joe Blow can go drive it, and he’s not going to get most of that race car.

I think they need to take a very different approach. They take an online IT software person, say build an agent for me and hope it works great. It’s not the way it’s working. So generally if you take a company like Distyl, which is in our portfolio, if they execute a project for a large Fortune 500 company, it goes swimmingly well. If in-house people in the same company executed, it goes very poorly. So, and each of these, even the in-house people will get better over time, so the third or fourth generation of execution will do better because their people will get trained, but they’re really not qualified to operate in this area. While the AI native companies are able to get real value, and they’re even able to pick the projects that will be valuable and the projects that are more experimental.

Vinod Khosla: Well, they don’t, they do and they don’t. It’s hard to tell what the details are behind each of these deals. If Nvidia is financing customers to buy their chips, that could be perfectly reasonable. Look, General Motors finances its cars too, when a consumer buys it, it’s just a regularized business. The question is hidden in the contracts that are mostly not publicly available: Who’s taking on what part of the risk? Is it an enterprise? Are you saying a customer you’re financing is viable or not? Are you saying the risk is the customers’ or their customers if they’ve done a contract with somebody else to buy X million dollars of inferencing? So where’s the risk hidden? Well, in fact is the key question before one can opine. And frankly, most of those risks and who takes what risk is in the fine print and not visible to almost anybody outside?

Vinod Khosla: Well, I would say I don’t care. I don’t care for the following reason. If Nvidia is taking a bad credit risk, that’s their problem, right? But if Nvidia loses $50 or $100 billion, does it kill the company? Probably not. And I suspect [Nvidia CEO] Jensen [Huang] is pretty smart about what credit risks he’s taking, with which customer. Is Oracle taking a larger risk? Possibly. It depends on the details of the contract between Nvidia and Oracle and Oracle and OpenAI or other people buying their cloud service.

So you have to think of it as traditional business and say where does the risk lie? If Oracle goes under, for example, because they took the risk of $100 billion spend and then get that back, then it’s their problem if Oracle disappears from the scene. Do I care? No. I hope they don’t. I think the ecosystem will be healthier, but people are selectively taking risk. You know, Coreweave’s taking a lot of risks with the money that belongs to certain lenders, to Coreweave. Do the lenders have risk? I don’t know. Does Coreweave have risk? And Oracle’s doing the same thing, it’s taking risk. But I don’t know the details of these contracts.

Vinod Khosla: I would say the fundamental notion of will there be more demand for API inference calls doesn’t concern me at all. The fundamental is how many inferencing calls we’ll see in 2030 and 2035. That generally doesn’t concern me because I believe AI will add a lot of value. Look, US economy’s $15 trillion of labor alone—just labor costs in the US economy $15 trillion. If you could replace 5 trillion of that, there’s plenty of room for inferencing to be paid for, if you can do that. So again I say the fundamental is, is there a demand for AI inferencing the next five years and next ten years, I’m not worried about that. has learned how clever a contract and who takes on risk if demand is slower or faster to emerge? … I’m not responsible for a lender financing a data center. If they fail, their problem, not mine. I care about the innovation ecosystem that drives more API calls in AI.

Vinod Khosla: Well, obviously AMD is trying things, Arm is trying things, Broadcom is trying things. Nvidia is in an unenviable position because they have so many different things they can do in parallel because the cash flow they have right now. Do I know all of Jensen’s plans inside and how many different things he’s trying? No. In fact, nobody outside really knows when he’s going to announce what my bet is he has a pretty precise roadmap to 2030 and beyond. So hard to say.

Vinod Khosla: Well, AMD is doing pretty well by signing, looking at their deals. Broadcom is doing pretty well. But are they going to grab majority share and be larger than Nvidia. I wouldn’t expect that today. Can they take reasonable shares, especially at slightly lower margins? Yes.

Vinod Khosla: Well I’ve seen a lot of specialized chip startups that do one thing. You can run your whole model locally in your shop. Well that’s that’s a market. It’s not as large as the data center inferencing market. So there’s many submarkets that the chip startups can do ok in.But I haven’t seen the chip startup that could completely blow everybody away. Now, if you have a sudden breakthrough in photonic chips that can do multiply, accumulate inferencing functions, and cut the power consumption by 70% for inferencing, that’s entirely possible, even likely sometime in the next five years.

And I hope some of those show up because it changed the power equation, how much power we need for AI. Frankly, most of the data center investment can be repurposed with a new kind of chip that gets slotted in. Photonics is pretty promising. I suspect digital semiconductor chips are going to be hard to beat Nvidia at, in a massive way. In specialized segments of the market, you can beat them but generally you’d have to have a radical breakthrough in technology. Photonics is one of them. There’s a few others, but I see the most promising candidate for an alternative to Nvidia becoming from photonics if one can scale it, and photonics typically is hard to scale.

Vinod Khosla: So the simplest way, very, very short term, to address electricity demand in the country is pricing. This is why the economics of marketplaces work. Prices will go up some, as we consume more pricing for data centers. Data centers themselves can be aggressive sources of power management. You can consume at certain times of the day for training runs and other times of the day for inferencing. You can dial up the performance or dial down the performance. So data center input of electricity itself is a variable that’ll probably be part of electricity trading.

There’s some good startups in that area.

I think there’s a short term solution which I think is super hot geothermal. I think we can get to gigawatt scale. If you imagine a couple of extra gigawatts of demand emerging every year. Some of it will be met through pricing energy appropriately. Some of it will be met through shorter term projects. Geothermal is one that’s much shorter term than, say, fusion, fusion’s a possibility but fusion to me will take the longest…but we’ll have an array of factors. We’ll have more natural gas turbines coming on. We do have companies where you can start with natural gas and switch to hydrogen whenever the economics warrant it. So there’s a number of ways to adjust, but it is a non-trivial problem.

Vinod Khosla: Well, first thing to keep in mind, it takes a couple of years to build a data center. There’s a few hacks, but fundamentally, if you’re trying to add a gigawatt of data center, which is about $30 billion of spend, it’ll take a couple of years. I don’t think that’s a six month or 12 month or even an 18 month project.

Then there’s demand-based electricity consumption as a tool. And then I think things like geothermal and other technologies will come along. Some of them will be natural gas fired. I hope there’s no more coal facilities. Mainspring installs capacity for data centers that can switch seamlessly from natural gas to hydrogen when you want to go clean, so you can decide how much you want to pay for power and what carbon reduction you want, and increase the carbon reduction over time. So that’s one solution. All I’m saying is there’s an array of tools …not non-trivial. This will be a serious issue. And policy is trying to address it. But I do think there’s many solutions.

Vinod Khosla : I think that op ed was misinterpreted a little bit. What I said was AI valuations in general are bonkers. For the best companies they’re not bonkers and if as a venture capitalist, you have access to those two, three, 4% of the startups, there will be huge wins. You will do well. You will have great returns. The people who are plowing money in without having special access to these opportunities for whatever reason, will suffer in venture capital as a class, I think broadly, for funds raised in ‘24 and ‘25 will have decreasing returns, returns lower because they’re not getting access to good deals early. They’re paying higher prices much later. Some of the robotics valuations are getting bonkers. I would venture to guess 95% of those startups will lose money.

Vinod Khosla: It’s more than that. I think I like to say most AI startups will lose money, but more money will be made than lost. That means it’ll be highly asymmetric: 2 or 3% of the startups will account for 85-90% of the valuation by 2035 of market cap of companies. So that asymmetry, which has generally been true in venture capital, but will be significantly more asymmetric in AI because of this valuation wave, I think is the reason we will see average returns decline and the best returns for the top firms will do okay, will do well, in fact, because AI is such a large opportunity.