I think one should attribute a good amount of credit of this to Yann LeCun (head of meta's research). From early stage he have been vocal about keeping things open-source, and voiced that to Mark before even joining. (and credit to Mark for keeping that).
It probably also stems from his experience working at Bell Labs, and how his pioneering work very much required a lot of help from things available openly, as is still the case in academia.
The man have been repeating the advantage of open-source being the better option, and been very vocal about his opposition to "regulate AI (read: let a handful of US government aligned closed-source companies have complete monopoly on AI under the guise of ethics)". On this point he (and meta) stands in a pretty stark contrast to a lot of big-tech AI mainstream voices.
Me myself recently moved some research tech stack to meta stuff, and unlike platform that basically stops being supported as soon as the main contributor finishes his PhD, it have been great to work on it (and they're usually open to feedback and fixes/PR).
>I think one should attribute a good amount of credit of this to Yann LeCun (head of meta's research). From early stage he have been vocal about keeping things open-source, and voiced that to Mark before even joining. (and credit to Mark for keeping that).
That's all very nice and good, but Meta keeps things open (for now) because it's perfectly aligned with their business goals. It couldn't have happened any other way.
You make it sound like it's a bad thing that it aligns with their business goals. I'd turn this around: if it didn't align with their business goals I would be worried that they would course correct very soon.
No, just that they shouldn't be showered with praise for doing what is in their best interests - commoditizing their complement.
For the same reason Microsoft doesn't deserve credit for going all in on linux/open source when they did. They were doing it to stave off irrelevance that came from being out-competed by open source.
They were not doing it because they had a sudden "come to jesus" moment about how great open source was.
In both cases, it was a business decision being marketed as an ideological decision.
I really don’t understand this weird, almost zealous resistance to admitting that a company can do a good thing once in a while. You’d think Meta had kidnapped people’s families and held them at gunpoint or something.
It's not. It's resistance to the idea that their demands to be lavished with praise should be acceded to because their single minded focus on profit aligned with a good thing once or twice.
They're supposed to do good things all the time without praise. That's why society grants them the right to profit.
Where is Meta demanding to be lavished with praise? As far as I can tell, nobody from Meta is demanding or asking that. The only thing that’s happening is people in the comments here on HN saying “wow this is neat, and it’s cool that it’s open source”, and then getting UM ACKCHYUALLY’d by reply guys telling them that Facebook is actually full of corporate genociders who occasionally write open source to fool us all.
Morality aside, I do like the open source work coming out of Meta. It's possible for a company to be "bad guys" in one area, and "good guys" in another.
Preposterous, it’s not like Zuck got on the horn with his algorithm devs and was like “let’s get rid of some people in Myanmar in a really roundabout way.” Do you hold the guy behind Curl to the same standard every time his software gets used in a way he didn’t intend?
Basically it's accusing Meta of should have knowing that their algorithm and their user generated stickers was spreading this content.
Yes in an ideal world they should catch any campaign of this sort, but global moderation is difficult and they offer no proof that Meta knew about this.
It's disingenuous to say that Meta agitated this event. Those specific users of Meta agitated it and Meta did not catch it.
> Yes in an ideal world they should catch any campaign of this sort, but global moderation is difficult
It really isn't, it just is expensive to do it. They could just hire people to do that. Thats the accusation. Of course they don't catch it if they don't try.
Meta (or TikTok or Twitter or any other social media company/product) can't both algorithmically create specific types of discourse (because higher engagement means more ad views) and deny responsiblity for the side effects of said discourse.
> Meta keeps things open (for now) because it's perfectly aligned with their business goals.
How?
I would have said a flood of LLM-generated spam is a pretty big threat to Facebook's business. Facebook don't seem to have any shortage of low/medium quality content; it's not like they need open-weights LLMs to increase the supply of listicles and quizzes, which are already plentiful. There isn't much of a metaverse angle either. And they risk regulatory scrutiny, because everyone else is lobotomising their models.
And if they wanted a flood of genai content - wouldn't they also want to release an image generation model, to ensure instagram gets the same 'benefits' ?
Sure there are some benefits to the open weights LLM approach that make them better at LLMs - I'm sure it makes it easier for them to hire people with LLM experience for example - but that's only helpful to the extent that Facebook needs LLMs. And maybe they'll manage to divert some of that talent to ad targeting or moderation - but that hardly seems 'perfectly aligned', more of a possible indirect benefit.
In a recent interview, Mark Zuckerberg said they're spending $10B-$100B on training and inferencing over the next X years. They see open source as a way to get the community to cut that cost. In his view, even just 10% cheaper inferencing pays for a lot.
Does open source just not count if you have an alternative business model? Even big open source projects hold on to enterprise features for funding. What company would meet your criteria of proper open source contributer?
Also is perfectly aligned with Yann’s goals as an (academic) researcher whose career is built on academic community kudos far more than, say, building a successful business
> I think one should attribute a good amount of credit of this to Yann LeCun (head of meta's research)
Isn't this more attributable to the fact that whilst Open AI's business model is to monetise AI, FB has another working business model and it costs them little to open source their AI work (whilst restricting access by competitors).
Yeah, the way I see it Meta is undermining OpenAI's business model because it can, I have serious doubts Meta would be doing as it does with OpenAI out of the picture.
This is clear as day. If they got an early lead to the LLM/AI space like OpenAI did with ChatGPT, then things would be very different. Attributing the open source to "good will" and Meta being righteous seems like some mis-founded 16 year old's overly simplistic ideal of the world. Meta is a business. Period.
Things like PyTorch help everyone (massively!), including OpenAI.
Another of Meta's major "open source" initiatives is Open Compute which has nothing to do with OpenAI.
I see zero relationship between Meta's open source initiatives and OpenAI. Why would there be? OpenAI is not a competitor, and in fact help push the field of AI forwards which is helpful to Meta.
Part that and part Zuckerberg's misanthropy. Zuckerberg doesn't care about Facebook's harms to children and society as long as he makes a quick buck. He also doesn't care about gen AI's potential to harm society for the same reason.
> regulate AI (read: let a handful of US government aligned closed-source companies have complete monopoly on AI under the guise of ethics)
Regulatory capture would certainly be a bad outcome. Unfortunately, the collateral damage has been to also suppress regulation aimed at making AI safer, advocated by people who are not interested in AI company profits, but rather in arguing that "move fast and break things" is not a safe strategy for building AGI.
It's been remarkably effective for the original "move fast and break things" company to attempt to sweep all safety under the rug by claiming that it's all a conspiracy by Big AI to do regulatory capture, and in the process, leave themselves entirely unchecked by anyone.
I think the whole "AGI safety" debate is a red herring that has taken attention away from the negative externalities of AI as it exists today. Namely, (even more) data collection from users and questions of IP rights around models and their outputs.
We can do more than one thing at a time. (Or, more to the point, different people can do different things.) We can advocate against misuses of current capabilities, and advocate about the much larger threats of future capabilities.
There's a big fucking difference between people who want to regulate AI because they might become a doomsday terminator paperclip factory (the register-model-with-government-if-they-are-too-big-crowd) and the folks who want to prevent AI being used to indirectly discriminate in hiring and immigration.
If you look around you’ll see that there are indeed very many people who are doing very different things from one another and without much centralized coordination.
Also the potential for massive job losses and even more wealth inequality. I feel a lot of the people who are philosophizing about AI safety are well-off people who are worried about losing their position of influence and power. They don't care about the average guy who will lose his job.
If we are interested in AGI safety, we should experiment with slightly unsafe things before they become hugely unsafe, instead of trying to fix known unknowns while ignoring unknown unknowns.
We should open source current small models and observe what different people are actually doing with them. How they abuse it. We will never invent some things on our own.
When Sam Altman is calling for AI regulation, yes it is a conspiracy by big AI to do regulatory capture. What is this regulation aimed at making AI safer that you refer to anyway? Because I certainly haven't heard of it. Furthermore, there doesn't seem to be any agreement on whether or how AI, at a state remotely similar to the level it is at today, is dangerous or how to mitigate that danger. How can you even attempt to regulate in good faith without that?
Sure; that's almost certainly not being done in good faith.
When numerous AI experts and luminaries who left their jobs in AI are advocating for AI regulation, that's much more likely to be being done in good faith.
> What is this regulation aimed at making AI safer
> Furthermore, there doesn't seem to be any agreement on whether or how AI, at a state remotely similar to the level it is at today, is dangerous
You could also write that as "there's no agreement that AI is safe". But that aside...
Most of the arguments about AI safety are not about current AI technology. (There are some reasonable arguments about the use of AI for impersonation, such as that AI-generated content should be labeled as such, but those are much less critical and they aren't the critical part of AI safety.)
The most critical arguments about AI safety are not primarily about current technology. They're about near-future expansions of AI capabilities.
Pause capabilities research until we have proven strategies for aligning AI to human safety.
> How can you even attempt to regulate in good faith without that?
We don't know that it's safe, many people are arguing that it's dangerous on an unprecedented scale, there are no good refutations of those arguments, and we don't know how to ensure its safety. That's not a situation in which "we shouldn't do anything about it" is a good idea.
How can you even attempt to regulate biological weapons research without having a good way to mitigate it? By stopping biological weapons research.
> When numerous AI experts and luminaries who left their jobs in AI are advocating for AI regulation, that's much more likely to be being done in good faith.
Their big revelation that they left their jobs over is that AI might be used to usurp identities, which, admittedly, is entirely plausible and arguably already something we're starting to see happen. It is humorous that your takeaway from that is that we need to double down on their identity instead of realizing that identity is a misguided and flawed concept.
Let's assume, for the sake of discussion, that it is. Is rudeness not rooted in the very same misguided and flawed identity concept? I don't suppose a monkey that you cannot discern from any other monkey giving you the middle finger congers any feelings of that nature. Yet, here the output of software has, I suspect because the software presents the message alongside some kind of clear identity marker. But is it not irrational to be offended by the output of software?
However, in practice, enforcing a pause entails a massive global increase in surveillance and restrictions on what can be done with a computer.
> Track the sales of GPUs and other hardware that can be used for AI training
So we now have an agency with a register of all computer sales? If you give a computer with a a GPU to a friend or family member, that's breakibf the law ubless you also report it? This takes us in a very scary direction.
This has to be a global effort, so we need a system of international enforcement to make it happen. We've been marginally successful in limiting proliferation of nuclear weapons, but at significant international and humanitarian costs. Nuclear weapons require a much more specialized supply chain than AI so limiting and monitoring adversarial access was easier.
Now we want to use the same techniques to force adversarial regimes to implement sane regulations around the sale and usage of computer equipment. This seems absolutely batshit insane. In what world does this seem remotely feasible?
We've already tried an easier version of this with the biological weapon research ban. The ban exists but enforcement doesn't. In that case, a huge part of the issue was that facilities that do that research very similar or the same as all kinds of other research.
An AI Pause has the same issue, but it is compounded by the fact that AI models can grant significant economic advantage in a way that nuclear/biological weapons don't so incentives to find a way to skirt regulations are higher. (Edit: it's further complicated by the fact that the AI risks that the Pause tries to mitigate are theoretical and people haven't seen them, unlike biological/nuclear. This makes concerted global action harder)
A global pause on AI research is completely unrealistic. Calls for a global pause are basically calls for starting regime change wars around the globe and even that won't be sufficient.
We have to find a different way to mitigate these risks.
>Pause capabilities research until we have proven strategies for aligning AI to human safety.
If the power of AI deserves this reaction, then governments are in a race to avoid doing this. We might be able to keep it out of the hands of the average person, but I don't find that to be the real threat (and what harm that does exist at the individual level is from a Pandora's box that has already been opened).
Think of it like stopping our nuclear research program before nuclear weapons were invented. A few countries would have stopped but not all and the weapons would have appeared on the world stage all the same, though perhaps with a different balance of power.
Also, is the threat of AI enough to be willing to kill people to stop it? If not, then government vs government action won't take place to stop it and even intra-government bans end up looking like abuse. If it is... I haven't thought too much on this conditional path because I have yet to see anyone agree that it is.
Then again, perhaps I have a conspiratorial view of governments because I don't believe that those in power stopped biological weapons research as it is too important to understand even if just from a defensive perspective.
This is coincidentally a ridiculously bad-faith argument, and I think that you know that.
The fact that one particular person is advocating for AI regulation does not mean that all calling for AI regulation are doing so due to having the same incentives.
This is exactly the point the parent poster is making. It feels like you only skimmed the comment before replying to it.
When I talk to folks like GP, they often assert that the non-CEO people who are advocating for AI safety are essentially non-factors, that really the only people whose policy agendas will be enacted are those who already have power, and therefore that the opinions of everyday activists don’t need to be considered when discussing these issues.
It’s a darkly nihilistic take that I don’t agree with, but I just wanted to distill it here because I often see people imply it without stating it out loud.
This regulation can only be done through a clueless government body listening to sneaky AI leaders. Let's make it new healthcare? Premature regulation is the last thing you want.
For a bigger pic, it's time for the civilization to realize that speech itself is dangerous and to build something that isn't prone to "someone with <n>M subs said something and it began" that much. Without such transformation, it will get stuck in this era of bs forever. "Safety" is the rug. It hides the wires while the explosives remain armed. You can only three-monkey it for so long.
This seems like an uninformed rant to me. I’m not even sure where you’re trying to go with that.
Do you know offhand, approximately what percentage of the White House AI Council members are from the private sector? The government doesn’t need to seek advice from tech bro billionaires.
They were supposed to be a non-profit mission driven organization. Having a product can be in alignment with that. In fact, I would happily pay for a product because I think paying for products is more constructive to humankind than the advertising business model.
Two different things to have a product to sustain research and expenses vs. having a product to make company grow exponentially and make investors richer
What if OpenAI couldn't bring ChatGPT out with the fund they was able to raise prior to converting to for-profit?
If investors didn't get any benefits ("richer" in your parlance) why should they invest?
If they didn't invest where would the money come from? tax dollar? Can you envision tax dollar be spent on AI research? or would they bring ChatGPT as quickly or as capable?
I don't really think that LLMs are a proper complement to social media.
Joel's reasoning requires that the complementing product be seen by the consumer as a requirement for consuming your product. Babysitters complement nice restaurants because parents need a babysitter to go. Gas complements cars because cars won't run without gas and the driver must buy gas periodically in order to use the car.
LLMs don't occupy the same space with relation to social media, it's more that they're quickly becoming an essential internal component for any social media site. Joel's reasoning doesn't apply to internal components that are invisible to the end user. A restaurant may benefit from finding a source of cheap lobster, but they don't benefit from publicizing that source to the whole world. Lobster is not a complement to restaurants, it's a component. LLMs occupy the same kind of space with relation to social media—they are something that every social media company would benefit from having cheaply, but not something their users need in order to consume their service.
Content creation is complement to social media, because content (videos, etc.) is shared on social media and in order to get it in front of people, you have to pay.
Platforms and Social media has replaced most of the world's ad surfaces, it's become THE way to get in front of people. Social media is a giant attention market but ultimately functions the same way as amazon: You pay to get your product in front of people.
LLMs commoditize content creation. Fewer people (after layoffs) can create more content. The money you save on laying off people then will be pumped pumped into boosting to get the content in front of people as the auction prices to get the right eyeballs go up due to increased competition.
People keep saying this, but I don’t see how an LLM is a complement to a social media website. People don’t consume more social media with more LLMs. Whats the link here?
Also on the Dwarkesh podcast, Zuck indicated one thing they’re afraid of is walled garden ecosystems they have to go through to reach users like with Apple and Google, and releasing open models is a way of preventing that happening with LLMs.
Both sides. Meta has tons of data streaming from their users (upstream) and more frequent touchpoints into their lives (downstream) than OpenAI.
None of that changes is AI is commodified.
Ergo, OpenAI wins if they have better models. Meta wins by default if everyone has equivalent models: existing business unimpacted, more access to users.
Social Media Websites are Marketplaces for Attention.
For example Mobile Gaming: Mobile games are demand generation bound - you literally run fake ads on facebook, and only the games that convert well are made. Yes, that's why the fake game videos exist. Making games is no longer hard, you pay a bunch of chinese and get the game. The majority of a mobile game's budget, with few exceptions, is spent on user acqusition.
Now picture AI making it easier to make content. Games. Movies. Etc. It invariably results in more content (and less quality, but as we've seen with News, that's not Mark's concern and people who think quality is something consumers choose over commoditized volume haven't paid attention for the last 2 decades). More content means more demand for eyeballs on Meta's platform, higher ad auction prices. Higher user acquisition spent.
Lucky for you, making more content with fewer people is a good effect of AI. So you save on talent, you lay off people and ... then discover that because Meta made AI available to everyone, you're just going to spend the additional money you made to pay for ads.
I agree that LLMs are not a complement - Facebook is not an organisation that desparately needs a bunch of LLM-generated content.
They have masses of content generated for free by users and journalists and influencers and so on - if anything, a bunch of LLM spam is a threat to that.
However, Open-weights LLMs are a much smaller threat to Facebook than they are to Google (where it could replace a lot of search usage) or Open AI (whose business is selling LLM access)
Perhaps for Facebook the benefits of the open weights approach - where you give away the model and get back a load of somewhat improved models, a faster way of running it, and a load of experienced potential hires - pays off because it doesn't threaten their core business.
> Facebook is not an organisation that desparately needs a bunch of LLM-generated content.
This is an overly narrow view of what an LLM can do. Generating text is the really neat parlor trick that people are trying to cram in to every possible startup, but if you take a broader view then what LLMs really are is the single largest breakthrough in natural language understanding.
Facebook doesn't need text generators, but they do need language understanding, especially for recommendation and moderation.
I'm not convinced that it's a complement—Joel's explanation is that you make a product that users consume alongside yours very cheap in order to keep people coming to you— but they definitely need LLMs.
Meta's business model is figuring users out and selling ads to them, as well as having to police posts on an industrial scale to try to remove stuff like election interference, terrorist videos, etc. AI is used for all of this.
The GPU cluster that they trained their Llama models on was actually built to train Reelz (their TikTok competitor) to recognize video content for recommendation purposes, which is the thing that TikTok does so well - figuring out users' preferences.
While this is true, that doesn't make them a complement to Facebook's broader business. Here's Joel's definition:
> A complement is a product that you usually buy together with another product. Gas and cars are complements. Computer hardware is a classic complement of computer operating systems. And babysitters are a complement of dinner at fine restaurants.
LLMs aren't really a complement like gas to cars because the end user doesn't need to consume the LLM in order to use the social media site. It's more like LLMs are becoming an essential component of a social media site—not like gas to cars but like an engine control unit, a part that ideally the user will never see or interact with. Joel's reasoning doesn't apply to that kind of product because users don't see the price of LLMs as a barrier to consumption of social media.
That doesn’t make it a compliment. A compliment is what a facebook advertiser or facebook user would also buy (or at least buy with their time) along with facebook. 5G data might be an example for FB users.
There are many ways. Generative AI helps people create content (and that's not the only way, I'm sure). Meta's platforms use content to drive attention.
For instance, an Instagram account that shows cool AI generated photos generates ad revenue for Meta.
Meta is not in the AI business. Meta is in the attention business (in this case, actually, no pun intended). If AI is not your product (as in: not how you need to make money), you can be be "generous". Making other peoples products less competitive by aggressively subsidising part of your business is not that cool of a move.
If Meta starts being all open and generous about their core assets, we can start talking. But we will not start talking. Because that will not happen.
> [...] Meta (or Facebook) democratises AI/ML much more than OpenAI, which was originally founded and primarily funded for this purpose. [...]
I believe this statement is accurate. Your comment does not alter this fact and merely imposes an arbitrary requirement instead of giving credit where credit is due.
If another company were to openly share alternatives to Meta's core assets, I would welcome that as well.
Facebook may be doing the right thing in this case, but for wrong reasons.
If a restaurant chain with deep pockets opens a restaurant in your area and starts selling food at a loss (because they can afford to do so, at least short term) in order to kill your beloved local mom and pop restaurants, should they be praised for it? This is how Walmart built their empire destroying countless small family owned businesses. The difference in the AI business is the scale of the fight. There are just no good guys in this, Facebook or OpenAI.
This is just a ruthless commercial move, not done out of the goodness of Mark's heart.
I read the Sam Walton autobiography and providing low cost goods with lots of options was one of the key benefits they provided smaller towns in Arkansas. The other chains couldn't operate at a profit due to the smaller customer base.
He was constantly trying to optimize his stores and dropping in on his locations daily. Because they were oroginally so far apart, in order to save driving time he learned to fly a plane and would just land in the field behind the store.
Constantly shopping competitors to see what they are doing better and how he could improve.
Originally store staff and towns welcomed the stores and him with open arms.
> If another company were to openly share alternatives to Meta's core assets, I would welcome that as well.
Metas core asset is human attention. "Sharing" it means selling ads. I don't think there is an open model to that — just giving away access to users is probably not it — but that's only one of many problems with "sharing" attention.
Like I said, it won't happen (for various reasons). So while it's cool that in theory we just want everything to be more open, and celebrate Meta for doing that where they do, and asking for more where they don't, my original point stands.
I don't think it's really being "generous" with their competitors business value. Meta has a track record of open sourcing the "infrastructure" bits behind their core products. They have released many, many things like React/React Native, GraphQL, Casandra (database), Open Compute Project (server / router designs), HHVM and dozens of other projects long before their recent AI push. Have a look here, I spent five minutes scrolling and got 1/4 of the way through! https://opensource.fb.com/projects/
With Llama, they now have an army of people hacking on the llama architecture, so even if they don't explicitly use any of llama-adjacent projects, there are tons and tons of optimizations and other techniques being discovered. Just making up numbers, but if they spend x billions on interference per year and the open source community comes up with a way to make inference even just a few more percent efficient, the costs of their open source efforts might be a drop in comparison.
For example, Zuck was on the Dwarkesh podcast recently and mentioned that open sourcing OCP (server/rack design) has saved them billions because the industry standardized their designs, driving down the price for them.
Is React a core Facebook asset? Where is Facebook sharing their ad tech? That’s the closest thing Facebook has to a core technology. Even Facebook’s ability to technologically scale isn’t that much of a core differentiator. We haven’t really seen a truly competing social network even get to the point where this was a problem. Facebook’s core is, if anything, the (social) network itself, which is something that it DEFINITELY closely protects.
Making their APIs easy to use like they used to be 10 years ago will be equivalent to releasing their core assets. In the past you could do almost anything from the Facebook API that you could do with their web or mobile app.
They release a lot of open source stuff as other commenters have mentioned but you can't build a Facebook or Instagram competitor just by integrating those components.
AI should have a huge impact on attention economics though. One cynical interpretation is that something that goes into their equation is that high availability of very competent open models should drive users to walled gardens since everywhere else is full of bots. I don't think that's the whole story though.
Facebook had the most open and used social platform for third party apps, and it was so successful that it was blamed for an election, and they had to cut back usage sharply.
> Making other peoples products less competitive by aggressively subsidising part of your business is not that cool of a move.
I don’t see it that way. Meta doesn’t have AI products, really, they have AI backend infrastructure. It’s like the Open Compute project for them, making their own infrastructure cheaper via openness, which seems perfectly cromulent to me. This could change.
Of course they’re doing good things out of self interest, but that’s how the system is supposed to work. It might even be preferable for us outside. Self interest tends to be more durable than altruism - particularly corporate altruism.
> Meta is not in the AI business. Meta is in the attention business (in this case, actually, no pun intended). If AI is not your product (as in: not how you need to make money)
Companies like Meta use AI heavily for everything from recommendation engines to helping flag content.
It’s not true at all to say that Meta isn't the AI business. They’re one of the companies deploying AI at the largest scale out there.
There’s more to “AI” than LLMs and ChatGPT style chat interfaces.
I explore two counterpoints in my top level comment, check it out. One point is that AI can moderate content to respect user attention. This leaves less captive attention for Meta to extract.
Imagine a browser where every HTML element is judged, filtered, and refactored in real time by Asgard - an AI that jealously guards the user's precious attention. That could become a major threat to meta's attention business, overnight. And I love that!
This seems to imply that generative content is a ‘complement’ to the advertising business, which is a pretty dispiriting realization.
The implication is that Meta benefits from there being a lot of generative AI users out there producing content, because a rich marketplace of competing bots will generate engagement with Meta platforms that they can sell advertising into.
They’re outsourcing click generation to content farms, and giving away the tools to do it to keep content farmers’ moats small.
I don't care about the motives or purity, I can just be happy that for whatever reason a company is releasing open source models and not just saying only a small group of weirdos in the bay area are responsible enough to use it
When we say should be, this assumes some kind of intention, no? Meta making things open is a means, not an end. The end is to weaken competition to position itself economically. The end is not to make things "open" in and of itself which is what the charter of OpenAI is (was?).
I agree, except in the case of OpenAI it should be an end, and yet they fail at it spectacularly, and since Altman/Microsoft finished their takeover there is basically no hope of it ever coming back.
The OpenAI charter specifically says "safe". We're all arguing over what that even means for an AI. If you're at all risk adverse, that argument by itself should be a hint that releasing the weights of a model.
For example, the last few years have seen a lot of angry comments about how "the algorithm" (for both social media feeds and Google search results) is politically biased. IIRC we don't know the full set of training data (data includes RLHF responses before anyone points me to the Wikipedia page I already read that lists some data sources) how can we be confident that any model from Facebook has not been specifically trained to drive a specific political agenda?
I can’t believe anyone buys it anymore. It feels like it was just yesterday when they were begging for a pause on training “dangerous” models (where danger was defined as “anything better than our flagship product”).
"The" safety argument? You think there's only one?
"The unsafe AI"? Which one would this be? Would it be the one which so many people on this very website complain has been "lobotomised"? (No matter how much I object to that word in this context, that's what people call it).
You can't have it both ways: either ChatGPT as it is now is dangerous (hence you don't open the weights but you also should not commercialize it) or it is not and there's no good reason to keep it secret.
ChatGPT has clearly caused significant negative social impact (students cheating on their essays, SEO spam, etc.) and they didn't give a shit.
> You can't have it both ways: either ChatGPT as it is now is dangerous (hence you don't open the weights but you also should not commercialize it) or it is not and there's no good reason to keep it secret.
Almost every dangerous thing I can think of "has it both ways" by the standard you apply here.
I can use an aircraft without being a registered pilot; I can use the police without being a trained law enforcement officer; I can use restricted drugs when supplied by a duly authorised medical professional; I can use high voltage equipment, high power lasers, high intensity RF sources, when they are encased in the appropriate safety equipment that allows them to be legally sold as a lightbulb, a DVD player, and microwave oven respectively.
The weights themselves reveal any and all information found within the model, regardless of superficial attempts to prevent the model "leaking". We do not, at present, actually know how to locate information within a model as would be required to confirm that it has genuinely deleted some information rather than merely pretending — this is an active field of research.
By analogy: data on a spinning hard drive. In certain file systems, if you delete a file, you only remove the pointer to it, the actual content can be un-deleted. A full overwrite of unused space is better, but owing to imprecision in the write head, even this is not actually certain to delete data, and multiple passes are used — but even this was not sufficient for the agents who oversaw the destruction of The Guardian's laptop containing a copy of the Snowden data.
At present, we do not even know how to fully determine the capabilities of a set of weights, so we cannot actually tell if any given model is "safe" (not even by a restricted definition of "safe" as in "we won't get sued under GDPR Article 17"), we can only guess.
And those best-guesses are what people complain about when they are upset that an AI model no longer does what it did last week.
There is an argument that having open access to the weights makes it easier to perform the research necessary to even be able to answer this question. That's an important counter! But it's not even remotely where you're going with your comment.
> ChatGPT has clearly caused significant negative social impact (students cheating on their essays, SEO spam, etc.) and they didn't give a shit.
None of those are significant social impact. Negative, sure, but the "significant" risk isn't SEO spam, it's giving everyone their own personal Goebbels; and it's not "students cheating" because, to the extent the AI is capable of that, the tests they can cheat on with it represent something that is now fully automated to the point that it shouldn't be tested — I mean, I grew up with the maths teachers saying "you won't have a calculator on you all the time when you're an adult", but by the time I got to my exams some of the tests required a calculator, and now I regularly do ⌘-space and it does the unit conversion as well as the maths, and even for the symbolic calculus I can usually just ask Wolfram Alpha… which is on my phone, which I have on me all the time.
> And noticing your lack of argument in support of a company you confuse with a sport club is not an insult, by the way.
I have no idea why you think I think OpenAI is a sports club. That's so weird I have to assume it's an auto-corrupt and not even what you intended to write.
I have no idea why you think a bunch of counter-examples isn't an argument, or why they are shallow — they demonstrate that you are just plain wrong. The only other people I've met who think that counter-examples weren't arguments, were… as I recall, two biblical literalist-fundamentalists and one politician. Oh, and someone who refused to accept that encryption was a good idea and government backdoors a bad idea, but otherwise I can't categorise them because they were a random Twitter account and it was a decade ago. And, now I think about it, someone who got themselves banned from HN for repeatedly insulting anyone who preferred electric/PV over hydrogen/nuclear.
And given that it is your previous comment which is in the grey as I write this, I think you need to look in a mirror before calling my words "clumsy".
> I have no idea why you think I think OpenAI is a sports club. That's so weird I have to assume it's an auto-corrupt and not even what you intended to write.
Not understanding something is OK, taking pride of it like that is quite something though.
> I have no idea why you think a bunch of counter-examples isn't an argument
Using counter-arguments is fine, as long as they are on point, which isn't the case here. No aircraft maker, for instance, ever claimed they were designing in aircraft to protect people against the dangers aircraft cause to mankind.
And that's why no example taken from the existing world can help defend OpenAI' hypocrisy, because no other company in the world started as a non-profit with grandiose claims like they did.
> or why they are shallow
Your writing is shallow because it goes in every direction without logical structure or exhibition of coherent thinking. You're jumping from one idea to another without articulating them.
And this latest comment of yours is also a good illustration of this, with the majority of your comment being rambling about random people you've met in real life or over the internet. Don't you realize it doesn't bring anything to the point you're trying to make and does a great disservice to your argumentation?
> And given that it is your previous comment which is in the grey as I write this, I think you need to look in a mirror before calling my words "clumsy".
Oh no, one individual OpenAI fan[1] downvoted my comment, my day is ruined.
[1] an here you see the explanation for the “sport club” point I made earlier, for some reason a bunch of people seem to believe that it's sensible to be fans of particular company and defend them in all situations over the internet. Apple fandom is the canonical example of that kind of behavior, but nowadays Tesla and OpenAI now have a fanbase with similar zealotry.
Meta wants to profit from the use of AI not the making/selling of it. Commoditize Your Complement and all that. Having this understanding of why is enough to be comfortable with alignment.
The Meta offerings should be under the umbrella name "FreeAI" analogous to Free Software (e.g. FSF) vs the more commercial leaning Open-Source. Heck everyone should contribute to it and make it a movement so "Free" is both an adjective and a verb.
>When we say should be, this assumes some kind of intention, no?
I don't think so. It's a company, not a person. Meta doesn't have intentions, just incentives. And if the incentives of a company are aligned with publishing open science and open software that's as good as it gets.
I don't require for profit businesses to do good things because they love world peace or are altruistic. Meta making its money from its consumer facing products that nobody is forced to use and having the models out in the open is exactly how it should be.
Making things open is a means, not an end, _in general_. The end is what we do with these things. What you call "competition" I would call "monopoly". A healthy, non-monopolic (or such) state of AI would be open by itself.
It never was. During their brief fight with Elon Musk they revealed old emails clearly stating that releasing things openly was only a ruse to attract talent at first and they never intended to do that with good AI models.
> As we get closer to building AI, it will make sense to start being less open. The Open in openAI means that everyone should benefit from the fruits of AI after its built, but it's totally OK to not share the science (even though sharing everything is definitely the right strategy in the short and possibly medium term for recruitment purposes).
Words like this suggest a failure of imagination. Given it's their own mouth and they're writing in their own defence, what might be a more generous interpretation?
The only more generous interpretation I can see is that OpenAI actually did intend to be open, but only between December 11th 2015 to January 2nd 2016, at which point they had changed their mind.
The paragraph immediately preceding the quotation is:
"The article is concerned with a hard takeoff scenario: if a hard takeoff occurs, and a safe AI is harder to build than an unsafe one, then by opensorucing everything, we make it easy for someone unscrupulous with access to overwhelming amount of hardware to build an unsafe AI, which will experience a hard takeoff."
"""H.G. Wells’ 1914 sci-fi book The World Set Free did a pretty good job predicting nuclear weapons:
They did not see it until the atomic bombs burst in their fumbling hands…before the last war began it was a matter of common knowledge that a man could carry about in a handbag an amount of latent energy sufficient to wreck half a city"""
and, I hope I'm summarising usefully rather than cherry-picking because it's quite long, also says:
"""Once again: The decision to make AI findings open source is a tradeoff between risks and benefits. The risk is that in a world with hard takeoffs and difficult control problems, you get superhuman AIs that hurl everybody off cliffs. The benefit is that in a world with slow takeoffs and no control problems, nobody will be able to use their sole possession of the only existing AI to garner too much power.
But the benefits just aren’t clear enough to justify that level of risk. I’m still not even sure exactly how the OpenAI founders visualize the future they’re trying to prevent. Are AIs fast and dangerous? Are they slow and easily-controlled? Does just one company have them? Several companies? All rich people? Are they a moderate advantage? A huge advantage? None of those possibilities seem dire enough to justify OpenAI’s tradeoff against safety."""
and
"""Elon Musk famously said that AIs are “potentially more dangerous than nukes”. He’s right – so AI probably shouldn’t be open source any more than nukes should.""
This is what OpenAI and Musk were discussing in the context of responding to "I've seen you […] doing a lot of interviews recently extolling the virtues of open sourcing AI, but I presume you realise that this is not some sort of panacea that will somehow magically solve the safety problem?"
I think people should be calling out Yann's role in this more. Mark might or might not have come up with this strategy on his own, but Yann was 100% pushing for Open Models and I'd like to think he has enough weight with Mark to make it happen.
Geoffrey Hinton, AI doomer Eliezer Yudkowsky and others like Gary Marcus preaching AI == nuclear weapons seem like they don't really understand how LLMs work under the hood. Or they flip flop between "look at how stupid AI is" and "OMG! AI gonna kill us all, they are too smart".
100% Generative AI for text, video, audio, images have legitimate concerns for deep-fakes, scams, hallucination. However extending it to say that it will destroy humanity in the next few years is pretty far fetched.
AGI in the hands of one or two powerful corporations / nation states is the biggest risk we face.
Competition and power balance is extremely important.
Yan LeCun is right, we need Good AIs to fight bad AIs. Good robots to fight bad robots. Good and Bad being relative to a group's beliefs.
We cannot trust nation states, or corporations, or billionaires to do the right thing. The right thing is relative. Humans are a distributed system optimizing for their own survival and good feelings.
Hinton, Bengio, Stuart Russell and other behemoths voice similar concerns over AI, with enough confidence to switch their careers (e.g. Bengio now doing safety research at Mila, Ilya Sutskever switching from capabilities to alignment at OpenAI, Hinton quitting job to focus on advocacy)
Their concern isn’t about today’s generative LLMs, it’s about tomorrow’s autonomous, goal driven AGI systems. And they clearly got presented with the arguments for the limitations of auto regressive models, but disagreed.
So, it seems a little much to place absolute confidence into it being a non-issue, because they just don’t understand something LeCun and others do. (with the same holding true the other way around)
"Luckily" the USA is capitalist enough that, if the top S&P500 companies are all releasing open (weight) models to the public, regulation isn't going to happen any time soon.
Probably around when they started selling their data to AI companies. Blocking, or at least aggressively rate-limiting datacenter IP address ranges is a no-brainer at that point, they will want to make it as difficult as possible to scrape their data without paying them.
Just started noticing it about a week ago. As someone else said, it only seems to happen when you're not logged in AND on a VPN. I can still login while on VPN and can still surf Reddit anonymously when not on a VPN.
My kids are hooked on "Meta AI" that's now built into Whatsapp. I have very mixed feelings about this and have tried my best to ensure they understand the limitations, but I also don't see an obvious way to disable it without getting rid of Whatsapp entirely, and thanks to the network effect that's not really an option either.
>We’re rolling out Meta AI in English in more than a dozen countries outside of the US. Now, people will have access to Meta AI in Australia, Canada, Ghana, Jamaica, Malawi, New Zealand, Nigeria, Pakistan, Singapore, South Africa, Uganda, Zambia and Zimbabwe — and we’re just getting started.
For one -- All the limitations and pitfalls of Generative AI ... with not enough awareness and maturity on part of users to sidestep or disregard them.
A very tame but representative example -- GenAI can reinforce stereotypes and biases:
MetaAI incapable of generating an image of Indian without a Turban.
So you're concerned about the cultural influences? Or the impact on development/learning? Both? Curious about this process, everyone planning on having kids is going to have to figure out the "AI questions". How are you planning on handling LLMs/AI usage by your kids going forward?
Only if it can't do voice cloning. Normally I find the AI ethics people insufferable but I do think that good voice cloning tech will do much more harm than good.
Might be a good idea to consider generalizing, and recognizing that if you find them to be correct on a particular topic, that should update your opinion on the potential correctness of other related positions. What positions do you think it's most likely that in a few years you might consider correct?
I think I correctly recognize who my enemies are (OpenAI and the small set of people who think they're the only capable stewards of AI). People allying themselves with the OpenAI/AI-safety folks are allying themselves with our enemies and regardless of the merit of their arguments in some philosophical sense I will oppose everything they do
> the small set of people who think they're the only capable stewards of AI
You're mistaking one group of people for another. I'm talking about the set of people who think that nobody is a capable steward of AI, and thus that we should not develop its capabilities further until they're proven safe (and I mean "proven" very literally, not figuratively).
These people will end up helping regulatory capture regardless (since they're too big of cowards to do what would be required to actually stop AI development)
What do you think harping about AI safety and x-risk is going to lead to? You think it will actually result in banning of frontier model training? It's going to result in "sensible regulation" being passed that means only massive corporations will control all the keys to the kingdom in AI
That seems like a description of what governments are currently willing to do, rather than a description of what AI alignment/safety advocates are arguing for.
I still think it's crucial to recognize OpenAI's impact on the field, because without them ChatGPT wouldn't exist and it's unlikely any other organization would have developed such advanced LLMs so early (or even released them for free). It's easier to come second and release stuff for free to appear "generous".
I don't think thats actually the case. OpenAI were ahead, but they weren't engineering in isolation. The building blocks were out there, but openai managed to put them together first.
That's why I said I don't think anyone else would have started training and releasing free models UNLESS ChatGPT existed and had success, and only OpenAI managed to do so.
Because it's easy to forget that fact, and pretend that Meta is "better" than OpenAI when in reality they are late to the game and are trying to catch up to commoditize their product's complement.
Everything there are doing is in service of advancing the singularly corrupt surveillance and "consumer" control service that is their bread and butter.
Tactical contributions are entirely in the service of strategic goals, in the service of a leadership and culture who have proven to unerringly do the worst thing possible so long as it increases their own wealth and power.
The list of whistleblowers and insider accounts of reprehensible and inexcusable abuses is endless and ever-growing.
Wtf would ever engage with them or their products?
Their models are the moral equivalent of "good data" from experiments run in gulags and concentraiton camps.
"Anyone building $0 free AI models that can be used on-device, like Stability, Meta, Apple, etc have already 'won' the AI race to zero."
The surprise here was that Google joined in the $0 free LLM race. Even when this is all over, both Google and Meta can do more than just LLMs; and people really have to think beyond this.
But at this point, it is clear that OpenAI's head start on GPTs is rapidly getting eroded as everyone else is catching up.
FAIR really does stand out from the field on available weights, safety considerations that seem at least plausibly connected to abuse potential, and a long-term, serious academic research agenda.
And I’ll remind you that their ocean of multi-modal, longitudinal over decades training set is not in any “uncertain” state as concerns copyright.
I’m pretty fricken annoyed with e.g. Boz given the fact that people at least sort of believe Carmack and Palmer now, which means they at least sort of believe me now.
But that increases, not decreases my obligation to be fair, and Meta AI is on fire.
Well for starters Meta has a lot of consumer facing stuff like social networks. And they are also able to produce hardware devices (being a much bigger company too). OpenAI had to invent the whole AI market but building customers is harder.
The bigger question is why google is so incompetent with AI now. Granted they still have Google Search monopoly but I think search without going to Google will be the future.
I do wonder if the most profitable AI stuff is coming from Microsoft due to their B2B skew.
For example, I know they have auto-subtitle stuff, but it's pretty ancient tech and I haven't seen it improve much since it was launched and it has still glaring shortcomings, like not being able to split out different people speaking or infer punctuation.
I'm not the OP, but recently, YouTube has started adding an automatic summary below videos. Before that, they also started adding automatic chapter titles, which, in my experience, are surprisingly good for navigating slide-based talks but are otherwise fairly hit-or-miss
No, as Hinton says in his own words "it's crazy to open source something so powerful". OpenAI is being responsible, LeCun still thinks AI is not smarter than a cat (literally he said that month ago). And he has taken Zuck for a ride.
Ilya and Hinton can be considered the inventors of this whole LLM novelty and they both agree they should not be open sourced. Says a lot.
As a bit of a side note, Meta's AI democratization will help with AR/VR. There is a dearth of interesting content to use on them. Open-sourcing will certainly create a fleet of content creators for the platform.
I expect the are creating a Sora competitor in the background as well.
Meta has an incentive to release free technology that could threaten their business competitors who stand to gain with closed AI. OpenAI isn't rich yet, so they have to monetize.
Their behavior is obvious when accounting for their positions.
hello mate so sorry but I came across your page and I thought I should pass this vital info,Well I suggest you should message Rothstein_code on instagram or Facebook @Rothstein_Code he’s a staff at meta ,and Their ic3 who handles account recoveries,this is an opportunity to reach one of their staff, I lost my account and the support teams couldn't do anything about it but this guy was able to help me out,all thanks to him. Hey! The support team doesn't work no more After the pandemic so You can still get access to your account am sure, quickly send a message to @Rothstein_code on Facebook/Instagram page he will sort this out for you,he works with meta.
AI can be a superior substitute for the low quality interactions on Facebook - one that never exploits your attention. And AI can be used to de-enshittify Facebook's contents.
AI as a direct substitute
AI can provide social support and feedback more constructively than people you meet in Facebook communities, and is not instrumented with anxiety inducing bloat.
I spend less time arguing in non productive debates on Facebook now, and more time talking with AI which helps me develop ideas and solve conflicts.
AI is more than just a stochastic parrot in this regard - it's increasingly a sound advocate. And it is hallucinating far less these days with the newest paid models.
When I want to talk to a random asshole who gives me grief, I talk to someone on Facebook.
If I want to talk about a topic and actually get somewhere beneficial to me, I'll talk to AI.
AI as De-Enshittware
There is another way that meta is not necessarily acting in their own best interests here. We can likely use llms to filter out the attention abusing bull crap from Facebook, adaptively overcoming countermeasures Facebook puts up.
The fact that LLMs can potentially de-enshittify Facebook's attention modification mechanisms makes it into an "indirect substitute" or "displacement good" for the enshittware version of Facebook.
Conclusion: Commoditizing AI might turn into a massive footgun for Meta
While open AI clearly hurts their immediate competition, it can also directly substitute the Facebook product. And it may indirectly make the Facebook less awful for users - and therefore less profitable for Meta.
tangent: it might not be worth the tens of billions that have been dumped into the "Metaverse", but I've always thought FB rebranding as "Meta" was a savvy strategic decision. Unlike Alphabet — which everyone still calls "Google" b/c Google search is still the most useful and ubiquitous part of the Alphabet conglomerate — plenty of people use things like Whatsapp and IG without ever having to touch FB (beyond its login infrastructure) and many of these users despise FB and its perceived "boomer" audience and content.
"Meta AI" at least sounds much more congruent and palatable than "Facebook AI", even if the people and processes remain the same.
> I’m surprised no one is outraged that Israel’s Military AI, named lavender is in active use identifying, targeting and murdering Palestinians In Real time.
It's not that no one is outraged, it's that we like to keep the outrage limited to the threads that are about the outrage. I'd rather not see HN devolve into a place where every discussion inevitably pivots to the horrific world event du jour.
The discussion was about the safety of AI and pausing AGI/AI out of fear of it going rogue or killing people. Lavender is actually doing that as a state sponsored military AI.
The technology is now out in the world and all AI/LLMS can be used in the general world community.
I would assume Palintir has offerings to do this Weapons-based AI killing people. Are there humans in the loop?
I must run in very different circles because in my social sphere even my Jewish friends are uncomfortable saying anything supporting Israel. Where is this rabidly pro-Israel mob?
Meta weaponizes open source to ensure no technological moats develop which increases the value of their moats:
- Data (Meta is one of two competitice companies when it comes to data volume)
- Compute (Meta is #1 here)
- Platform / eyeballs. (Meta is #1 here, Bytedance will be degraded)
It degrades talent moat and destroys proprietary technology moats.
Open source does PMF, R&D and de-risking for them while destroying any proprietary competitor - especially ones that don’t have the funding to fight the price dumping effect.
And make no mistake, in most industries this would be illegal dumping - if a furniture chain started giving away superior lumbering equipments to anyone cross financed with external money to deny sales to their competitors it would be handled with swiftly and decisively.
Sundar right now will be getting questions from their investors of why they spend 200M on Gemini if anyone with enough data and compute can achieve the same thing. Remember “We have no moat and neither has OpenAI?”. It took less than a year for that to play out go brutal effect. Llama3 450B will have google up at night.
It also allows meta to effectively not hire armies or product and engineering talent because they get that value for free. Llama.cpp alone is worth hundreds of millions in combined R&D at this point, catching up the llama architecture to its competitors.
Finally the result of AI is a commoditization of content creation - more content in an attention saturated ecosystem increases the competition for eyeballs - aka what companies have to pay to beat their competition on the marketplaces of attention.
And companies will be able to spend money on that because they can fire their creators (that’s what the Sora and Vasa class of models ultimately will do within a year) and save on compute - only to spend it on demand generation.
Analog to how Amazon managed to run by the spirit of open source to monetize open source software without giving back, Meta has shaped the passion and desire of people to build and share into a powerful weapon it wields with deadly precision against its competition, all while being able for benefit from the collateral effects on every level.
Mark is nothing but predictable here - he’s an aggressive, always at war General and “commoditize your complements” and “accelerate technology adoption to improve the business environment” are some of his key gambits (see emails on oculus adoption) and the road is littered with the burnt out husks of previous plays - such as the entire news business he commoditized for attention and reengagement.
Yes there is side effects that are good - the freeing of the technology from the yoke of Google and Altman Corp, but that does mean there’s any charitable intent here. Mark does not give a damn about the common good. He cares about winning. Always has.
> And companies will be able to spend money on that because they can fire their creators (that’s what the Sora and Vasa class of models ultimately will do within a year) and save on compute - only to spend it on demand generation.
How does this make any sense? Sora is a killer technology with a massive potential, firing the team for it is a suicide move as they can move to a competitor and build a better one.
This comment is just desperately grasping for straws at this point. Why is the simpler explanation (that they have always had a culture of being open about AI tools and research) so hard to grasp?
I’m curious why you think I’m grasping for straws? It is clear that Meta can like open source and meta can like it not for charitable or esoteric “values” but for the business benefits it brings.
Maybe I should have mentioned the better part of a decade on the inside, if there were “values” beyond “how do we crush our enemies”, “how do we keep regulators at bay” and “how do we win the war for talent” relating to open source, well, I haven’t seen them.
You really think “open” is a business motivation enough to dump hundreds of millions of R&D into this ;)
There are a lot of good reasons to go open-source, the majority of them being related to boosting the quality of the products being developed themselves. PyTorch is the behemoth it is only because it went ubiquitous, and it went ubiquitous only because it was that accessible. Whose "moat" was Pytorch destroying?
Saying that going FOSS is just inherently some sinister strategy to cut off other businesses' tech moats is a lot far-fetched than the simpler explanation that open-source simply gives you a lot more exposure and insight towards development and quality of the software. It has been a legitimate model for nearly two decades now (read: CatB).
Their engineers. Their time. Their knowledge. Open-Source.
And who knows what future will bring. Maybe a model like GPT-4 will eventually be made public. To this day it is still the benchmark, which is forcing other teams to find solutions for them to be able to get to that point also.
I suppose the difference is that Meta didn't just contribute to Torch, they created it. Meta seems to be quite good at open sourcing things in a way that provides real value to people.
The Github org you linked to mainly seems to have repos for the OpenAI API, which doesn't quite rise to the same level of React and PyTorch.
It probably also stems from his experience working at Bell Labs, and how his pioneering work very much required a lot of help from things available openly, as is still the case in academia.
The man have been repeating the advantage of open-source being the better option, and been very vocal about his opposition to "regulate AI (read: let a handful of US government aligned closed-source companies have complete monopoly on AI under the guise of ethics)". On this point he (and meta) stands in a pretty stark contrast to a lot of big-tech AI mainstream voices.
Me myself recently moved some research tech stack to meta stuff, and unlike platform that basically stops being supported as soon as the main contributor finishes his PhD, it have been great to work on it (and they're usually open to feedback and fixes/PR).
reply