I head up Product at Lambda. We are an NVIDIA preferred partner (and keep winning the preferred partner award). I don't know what the allocation numbers are for other companies, but we get a lot.
It's in NVIDIAs best interest to spread the love around and make sure all the GPUs don't go to the hyperscalers.
We have new product coming in just a couple weeks to get clusters as small as 16 GPUs per cluster and only a two week minimum reservation, with very short lead times (the first people to reserve will have almost no lead time!).
Also we expect to have a bunch of capacity come online of the on-demand cloud this year. We're getting the GPUs as quickly as we can and racking them as quickly as possible, but we have to wait just like everyone else for the GPUs to roll off the fab. :)
I was at a Christmas party a couple jobs ago and they did a presentation that was supposed to be all good news and instead basically ruined the party for me.
The bit that freaked me out was 2 customers were 60% of revenue.
Those two customers can change your policies in ways you don't want to, because you can't afford to piss them off. And if something happens to one of them you're fucked.
Nvidia is a shovel maker. If you only have 1-2 shovel buyers, buyers who've started on making their own shovels in-house, then you run the risk of being trapped in a monopsony. You lose 1 shovel buying client and then you might be insolvent before too long.
I think one of Jensen Huang's big interests is to enable a diverse number of markets that can utilize the technology in a short feedback loop. I don't believe that can be achieved if all of the hardware goes to Microsoft/Google.
It’s better to have a diversified customer-base rather than being completely beholden to a small number of big customers that might decide they don’t need you down the line.
Indeed, Google and Amazon have already shown an interest in designing their own silicon (TPU and Graviton). Probably Microsoft too, but I'm not aware of an example off the top of my head.
Upstarts are not easily going to be able to pursue that, so NVidia has a strong interest in supporting them.
Why couldnt NVIDIA auction some to get best price?
I mean locking customers long term is probably better, but a certain % of product still vould be sold for a spot price? This made me wonder what is the optimum split on that - obviously depends on your market placement, but isnt nvidia basically a monopoly now?
MI250 and MI300 are competitive for inference and training. The problem is that there's no cloud with simple registration that offers these for reasonable on demand price. This means that basically no one in the open source AI community can use them, so most development and tooling is being tested and built for NVIDIA.
I wonder what Lisa tells the shareholders & board when they ask why billions fly past their noses in valuation. Curious why ATI/AMD never could be on par to Nvidia regarding the software stack (drivers mainly), I would assume bad leadership and goals.
I'd call it a focus on very profitable ventures, like gaming.
Nvidia effectively created this AI world by building tooling and software for something we didn't know existed up until not too long ago. AI was created on the shoulders of the work that was done.
AMD wasn't taking the risk on the unknown, for whatever reason.
Water under the bridge. The ship is certainly busy course correcting now. Give it some time.
Well they didn’t have any money. They bet big on cpu r&d and they did great but there wasn’t enough money left over for other moon shoots. AMD was at risk of becoming a failed company not too long ago so they recovered quite well.
