This is interesting, but I suspect the real story is that Bing Chat is lower priority than OpenAI. And it should be! If Bing Chat is based on GPT-N, training GPT-N+1 and building the network effects of selling that technology absolutely should be the first priority. If Bing Chat goes down, someone surely gets paged but nobody is going to be ruined. And frankly, most consumers probably won't notice. At the same time, it's Microsoft's flag that they've planted in search. How much is that worth? Apparently (at least) the cost of GPU-hours from Oracle but less than the cost of holding back those GPUs from OpenAI.
My guess is that Microsoft’s own resources have to go into supporting enterprise commitments to M365 Copilot (and other Copilots) and their Azure OpenAI service. Bing Chat is easier to carve out from the enterprise / Azure stack.
I suspect you are right. However, we should not underestimate the power Open Ai tech has brought to Bing. Combining the GPT4 capabilities with live search data gives you an incredible strong and useful search service. Personally I have entirely shifted my desktop use from Google search to Edge and Bing Chat Enterprise since I got access to it. If my shift is representative for just a few percentage of the population, it represents a major shift in the search market.
I use ChatGPT paid for at least 75% of my queries. Only reason to venture out into search engine land is to find updated information which often is done by adding reddit to the end of the search.
I'd say Google is in trouble, I mean there is a limit to how many ads you can pack on the first page of results and I'd say they've passed it a long time ago.
In that context Bing + GPT would be more of a winner I think.
Probably a problem fixed by time. We laugh a little when we hear the ENIAC took up an entire room and required an entire staff all for 500 floating-point operations per second. Sure, maybe it's difficult to scale current lithography much further, but someone will probably come up with something.
Apparently the demand is large, as Microsoft is not a small customer by any measure.
The actual cause here is something else: NVIDIA have been allocating GPUs according to long term corporate priorities rather than just auctioning them off and letting price signals do the work. The reason is twofold:
1. They want a diversity of customers for GPUs, not just one or two LLM mega-players that everyone else invokes via high level cloud APIs. That would put NVIDIA in a bad negotiating position.
2. They don't want GPU supply to be dominated by companies that are simultaneously designing their own silicon. Microsoft is. [1]
Oracle has been getting large allocations because they aren't designing their own AI hardware (or at least, if they are, this isn't public), and because they are in the business of selling CUDA and not hardware abstracted model access.
Oracle's market capitalization is $298 billion, while Microsoft's market capitalization is $2.6 trillion. If there's anyone that can stave off Oracle's lawyers Microsoft is probably one of a handful that can.
Also, I'm guessing that Oracle is sufficiently desperate to wring any revenue out of their investment in cloud hardware so this might be a rare occasion where Oracle will just take the money and call it day.
"In Japan, we believe our competitors are stealing the rice out of the mouths of our children. In Japan, we think anything less than 100 percent market share is not enough. In Japan, we believe it is--and this is a quote that supposedly came from Genghis Khan, but I have heard it attributed La Rochefoucauld and other people--it is not sufficient that I succeed; everyone else must fail. We believe that this is not sufficient. We must destroy our competition.
More precisely, it is Larry Ellison quoting a Japanese business executive. Otherwise he would not have begun the sentence with "In Japan, we..."
Ellison's opinion is given a moment later:
"And we certainly believe that that is absolutely correct, that we have to pursuit market share. Today we have more share than all of our competitors combined. And our primary goal is to improve our share, our market share."
You said Ellison's opinion is given a moment later but that part you quote isn't about the Japanese executive's story. It's about something said by Jack Welch:
Jack Welch over at General Electric: "If you're not one or two in the market, you don't make money. The market leaders make the money." You know, you have to have share. You have to have substantial market share in order to be profitable in most businesses. The Japanese recognize this very, very clearly. And not just the Japanese. The great business leaders in the United States are the same, whether it is the old guard--and Jack Welch forgive me for calling you the old guard--or the young guard like Bill Gates. And we certainly believe that that is absolutely correct, that we have to pursuit market share. Today we have more share than all of our competitors combined. And our primary goal is to improve our share, our market share.
Still at least you provided a source. The actual full quote is Ellison saying that he was told this story by a Japanese business executive, and when he repeated this story to a New York Times journalist they took it totally out of context and made it sound like it was him saying it. He even says he's got in trouble for repeating this story before, but it's OK here because it's on video and therefore hard to take out of context! But on HN that's no obstacle apparently.
DM: Going back to Japanese culture, one of the quintessential elements of the Japanese approach to competition has been the aggressive pursuit of market share. Was your aggressive pursuit of market share early on something you learned, or was it just the natural outcome of heads-up business?
LE: No. I think it was absolutely our primary goal. I was always influenced by Japanese business people. In fact, it was a very famous story that I've gotten in a lot of trouble for, and I'm very hesitant to mention here, but I will, because at least it's on video and it's harder to take things on video out of context. But I was in Japan and I was talking to a Japanese business executive. And he told me that the problem with America is that we just have no stomach for competition. And I was defending America. I was defending my country and my culture. I said, "What do you mean? America, at the turn of the century was the leading edge I mean, we were the laboratory, you know, for free market economics. The industrial revolution reached its zenith in the Eastern States of North America. What do you mean?" And he says to me, "Don't tell me about things that happened almost 100 years ago. Let's talk about today. You listen to your business executives and they say things like, Well, we have great respect for our competition; the market is very large; there is room for all of us to compete."
"In Japan, you know, that would be sacrilege," he said. "In Japan, we believe our competitors are stealing the rice out of the mouths of our children. In Japan, we think anything less than 100 percent market share is not enough. In Japan, we believe it is--and this is a quote that supposedly came from Genghis Khan, but I have heard it attributed La Rochefoucauld and other people--it is not sufficient that I succeed; everyone else must fail. We believe that this is not sufficient. We must destroy our competition.
And I remember I came back with that story and told it to people at Oracle. And I remember telling a New York Times reporter this story. And the New York Times reporter had a huge picture of me, took the whole thing out of context. It said, Larry Ellison, quote, "It's not sufficient that I succeed; everyone else must fail." With a huge picture. Then I started getting hate mail. (Laughter) I mean, this story stuck with me for a long time.
I think this has to be something to do with Oracles Mellanox holdings, they must be getting a better price or have a license that means they can deploy GPU clusters cheaper. Otherwise why would nvidia give an allocation to oracle over Microsoft.
"According to StatCounter, for October 2023, Bing had a 3.1 percent global web search market share for all platforms – that's compared to Google's 91.6 percent, but up from 3 percent the month before."
Bing's market share increased by 0.1 percent? What is the point if mentioning this?
Just in case anyone gets tempted, the P106-100's were mining cards and locked to PCIe 1.1, so the bus bandwidth is terrible. Add to that the limited amount of memory (6GB), low memory bandwidth, Pascal (1/64 speed FP16), and that these were all likely being run in extremely shoddy data centers, it's not even worth the power costs to run IMO.
For those looking for the cheapest higher memory solutions, 24GB P40s are available (decent amount of VRAM, 3X the memory bandwidth, but requires server or DIY cooling, same bad FP16) or IMO the best bang/buck for hobbyists/home devs atm, used RTX 3090s are going for about $600-700 each.
(Note: if you're doing training, unless you have very high utilization/already calculated your costs, you will probably be much better off renting cloud GPUs from Vast.ai, RunPod, etc)
While mining may not be what Nvidia made them for, it's what most of them were sold for, so I think the warning on their likely condition still applies.
Beyond that, I'm pretty sure that the Techpowerup DB is wrong on the PCIe bandwidth. The first-hand reports I've seen online says the P106's are limited to PCIe 1.1:
$20 is a fair price for those that actually need exactly what the P106 provides, but the cheeky flipside argument is that having a near monopoly might make even the e-waste overpriced. :)
Using a full-blown GPU just for neural network inference is crazy inefficient. They should hire some blockchain dudes to build them custom hardware for one tenth of the price.
Google hired a bunch of chip designers to make TPUs for their ML people, so are highly optimized for this kind of work, which are currently on their 5th gen, and are available on their cloud.
(Disclaimer: I used to work there but not on them.)
All I'm saying is that there are viable alternatives. Oracle and SQL Server being successful doesn't really force anyone into using either of those things. Similarly, sure, .NET and Java are popular enough, but they have alternative runtimes available (remember when Oracle sued Google over that). To me, I don't think the industry landscape changes meaningfully if these tools were to become unavailable.
I think I get it. There's no horizontal scroll bar until you reach the very distant bottom, so if you don't know to middle click to scroll sideways, it looks like pricing is the last column, and it's cut off until you scroll that way.
I don't think that's possible. Very few fabs can manufacture these chips, and they are either several years away from completion, or fully booked for about that long.
Oracle's GPU setup is pretty good - good availability, interconnects, and support. It's the primary selling point of OCI these days and they do it well - unlike the other clouds, they are focused on making NVIDIA chips work well instead of creating their own competing chip.
The rest of OCI is a step down, but it's mostly good enough
That line of business was proven unprofitable with the purchase of Sun by Oracle in 2009, finally closing decades of non-x86 server architectures like SPARC, PowerPC, MIPS, Alpha, and Itanium.
1. It will take over many creative pursuits, taking away the feeling of usefulness from many people.
2. It will cause humans to be too independent from each other, and remove the sense of community (technology does this to some extent, only AI enhances the effect much more so)
3. It will concentrate wealth into the hands of tech companies, who relentlessly make life worse by continually making consumerism more efficient and thereby destroying our connection with the natural world
4. It will create disinformation and be a tool of manipulation and crime that will be hard to counteract (such as spoofing, etc.)
5. It will foster a world where there is far too much information, which itself plays on our basal instinct to collect information. This instinct was adaptive in the time of information scarcity, but is now maladaptive
This is like saying developing modern rocket and satellite tech is bad because it can be used in missiles. Is GPS bad because it was made for questionable purposes before it was made available to the public? Without rockets we wouldn't even have WD-40 that was made to solve a certain rocket rusting. Pushing boundaries in tech has always had a positive effect to the general public, because even with an insane goal such as AGI we get a lot of tech that can be applied in different ways along the way that we can't even imagine.
Positive effect: so climate change, species decimation, destruction of communities, etc? A positive effect? Can you be so sure that your comforts are worth that?
Climate change: rockets pollute less than planes with less harmful impurities (rocket propellant is heavily refined) while also allowing us to launch satellites that can help us study climate change, also helping farmers be more aware of their land through satellite monitoring. Also, some rockets use Hydrogen and Oxygen - waste is H2O. Modern rockets are starting to use Methane and Oxygen - while still polluting, it is still much less than the impure kerosene used by planes.
Species decimation: Satellite imagery has been used to expose illegal farming, mining, etc. operations that would have been otherwise undetected for a long time. Yay biodiversity!
Destruction of communities: Idk man I feel like Starlink is helping communities a lot by providing cheap and fast shared internet to communities that wouldn't otherwise have the technology.
It's not just my comforts that I'm thinking about. Rockets while individually may be seen as useless, just like AI, are a platform where a lot of overwhelmingly positive projects could stem from.
The critiques you mentioned could be applied to mobile phones. I doubt you would actually agree phones have no place in society - it's not just comforts that are a positive.
Net harm for what? nobody takes away your creative pursuit, you will always be able to do it as a hobby.
Health and other related research getting massive boosts by future more powerful AI is not a "net negative", if you don't wanna live longer or have your family not die of an illness then good for you, not everyone thinks the same
You're confusing technology and the economic system that technology exists under. Your first three points are symptoms of Capitalism, not AI.
Your fifth point is largely moot, since there's already more information available to people than anyone could possibly obtain in their lifetime, and the amount is growing faster than anybody could consume it. The problem can't really get any worse.
Your fourth point is more interesting, but I'd argue that the tech is already widely available for any type of crime you'd want to commit using AI. Banning it at this point just makes it so that only the criminals have access to it, and ceasing further progress on AI doesn't really make sense because from here, improvements will benefit legitimate users who want consistent, correct responses from their chatbots far moreso than the criminals who can afford to cast wide nets.
> Banning it at this point just makes it so that only the criminals have access to it
I would argue that this isn't really the case; the number of people who have the ability to create something at the level of ChatGPT or to make use of what's out there right now properly, is really quite small.
In that sense, if we were to utterly ban the tech, right now, it would probably be sufficient to prevent criminals from using it for most of the things it will be used for criminally, 5 years from now. It's too niche of a field, in its infancy, that if it were utterly and completely shut down, the cartels would not pick up the tab to single-handedly do it themselves.
Obviously that won't happen though, nor am I advocating for it. Personally I think the good done will outtweigh the bad many, many times over.
I'm just pointing out that societal problems arising from the adoption of AI are completely self-inflicted by our steadfast belief in our economic system. How fucked is it that we can look at a technology that reduces the need for human toil, and claim that's a bad thing? The complete elimination of the necessity of human labor should be one of, if not the highest, good.
Thats ridiculous, how would that be calculated? (Your profile says you have a PhD in math, so I'm expecting rigor)
Is all power equal? Here in New Zealand a large amount of our energy is green and comes from hydroelectric - how does this compare to the many many dirty coal power stations in China?
What about something like an Aluminium smeltering plant? ... it doesn't need many people relativly speaking, but it consumes an enormous amount of energy and produces a good chunk of the worlds aluminium.
How would this be enforced geo-politically? Or do you just want to enforce it in your own country so that all of your businesses become uncompetitive and your economy tanks?
I mean yeah 1.3b is great but your revenue is coming from everyone else chasing the hype train. It will be a big viable business but it’s probably not that earth shattering
It's always hard to tell. Is this "No Wifi. Less space than a Nomad. Lame.", or is it NFTs? You can never tell until the hype train either crashes or takes over the world.
It is earth shattering. This thing is writing code and can do very very simple decision making in a wide variety of domains out of the box. Possibilities are endless.
