General News
-
What's interesting. This is compute which is sold to customers for their business needs(inferencing). What about the LLM owners training their own models-just as big? And usage in the grand scheme of things, speculating is actually tiny cf where it will be in the years ahead.
-
I think the squeeze at the top among frontier models is going to produce a wave of 'Cursor-like' orchestration tools , whispering IDEs and workflow layers that quietly manage which model runs underneath and when. Also Google, Amazon and others are all building their own cheaper TPUs that need less memory, purpose-built for inference rather than training.
I don't think this is a bad thing, but suspect we're heading for a model in which the more you pay the more concentrated intelligence you get. -
The nuance in 'TPU adoption' is compute used internally whereas their customers(cloud) are using GPUs mainly because of CUDA. Today the TPU/GPU mix in total(TAM) is 15-20% and that won't change for years. Any news of GOOG/AMZN building huge TPU clusters is already planned and not something to suggest derailing the Nvidia growth story.
TPUs came about due to: scarcity of alternatives and don't forget TPUs are built for a very narrow process as opposed to GPUs which are programmable. So it's only natural that a device built to perform one thing is less complex.
As to 'TPUs use less memory' another pit fall if you then leap to assumptions . Yes a TPU uses less HBM than a GPU, maybe half but the sheer scale of TPU cluster deployment (in the multi millions) means the demand for HBM is enormous. And all we have to know is that whatever that demand is, it significantly exceeds the ability to meet it. And that imbalance will be worse next quarter and the one after that and so on (years).
Regardless, compute will devour all memory supply indefinitely. Every quarter into the 2030s will see the gap-demand vs supply, widen, imo.
In a constrained market, who theoretically is gaining or losing some market share is irrelevant, and don't forget TPUs are produced by TSM and they too are constrained. Nvidia created the market, are TSMs biggest customer and culturally there are very strong bonds between the two and their founders. Business is business so even if you ignore that, Nvidia has the money to book capacity many years in advance so as I have suggested before, taking AMD as an example, they will be constrained by how much they can commit to and or be allocated simply because their balance sheet can't fund it even if the market wanted it. It's not a case of asking for product at scale years out without handing over billions in up front payments.
And if you ignore all of that for a moment. Fact..
TPUs can be cheaper than GPUs in optimised, large-scale, Google-style workloads such as search optimisation.
GPUs often win in real-world enterprise environments due to flexibility and ecosystem eg, multi modal, inference across models. -
Samsungs record profit having been spotted by the employee union. In Korea avg employee earnings are USD 110k per annum-the union demands that the company pays employees on avg $425K as a bonus. I suppose you Can ask
-
Big old bump today, after only being about 3 weeks ago when it was down in the doldrums, feels like we're almost back to the highs of the start of the year again.
-
The nuance in 'TPU adoption' is compute used internally whereas their customers(cloud) are using GPUs mainly because of CUDA. Today the TPU/GPU mix in total(TAM) is 15-20% and that won't change for years. Any news of GOOG/AMZN building huge TPU clusters is already planned and not something to suggest derailing the Nvidia growth story.
TPUs came about due to: scarcity of alternatives and don't forget TPUs are built for a very narrow process as opposed to GPUs which are programmable. So it's only natural that a device built to perform one thing is less complex.
As to 'TPUs use less memory' another pit fall if you then leap to assumptions . Yes a TPU uses less HBM than a GPU, maybe half but the sheer scale of TPU cluster deployment (in the multi millions) means the demand for HBM is enormous. And all we have to know is that whatever that demand is, it significantly exceeds the ability to meet it. And that imbalance will be worse next quarter and the one after that and so on (years).
Regardless, compute will devour all memory supply indefinitely. Every quarter into the 2030s will see the gap-demand vs supply, widen, imo.
In a constrained market, who theoretically is gaining or losing some market share is irrelevant, and don't forget TPUs are produced by TSM and they too are constrained. Nvidia created the market, are TSMs biggest customer and culturally there are very strong bonds between the two and their founders. Business is business so even if you ignore that, Nvidia has the money to book capacity many years in advance so as I have suggested before, taking AMD as an example, they will be constrained by how much they can commit to and or be allocated simply because their balance sheet can't fund it even if the market wanted it. It's not a case of asking for product at scale years out without handing over billions in up front payments.
And if you ignore all of that for a moment. Fact..
TPUs can be cheaper than GPUs in optimised, large-scale, Google-style workloads such as search optimisation.
GPUs often win in real-world enterprise environments due to flexibility and ecosystem eg, multi modal, inference across models. -
Pleased you’re spending time investing in the knowledge. Hear the comment, ‘I hear a lot of opinions but I can’t always adjudicate. ‘You’re talking to the expert’ yes he is.
Cuda/cuda. Cuda is so pervasive because it’s been built over 20 years. It has more developers than any alternative. AMD have no answer for it. It makes gpu’s sing. The end
-
A great couple of weeks capped off with positive vibes out of the ME. SK Hynix(memory giant) reports on Thursday so that will likely move Micron in sympathy. Have a good wend all.
-
A very strong day today (if it holds) across the board. Nice to see. Micron hit a new high of $485
-
A very strong day today (if it holds) across the board. Nice to see. Micron hit a new high of $485
-
In particular. Cobens Tech
YTD +16.67%
1 Year +97.2%A lot of tech is barely positive
Biff Lusitania Fund -3%Lifestyle and Tech are ATH as is Index 100
-
Funny coincidence that, but the fund isnt named after the ship
-
Iceberg Fund for the win.
-
Iceberg?
Don't mention Icesave......Landsbanki......a minor shudder down my spine remembering how we had a large sum (between houses!) at risk back in the day...
https://www.theguardian.com/money/2008/nov/10/credit-crunch-savings-icesave for those unfamiliar with the episode
