It takes Edgar 54 days to build a swarm taking metallic elements from all sources, including living organisms. But then when instructed not to harm living organisms, it still takes Edgar 54 days to do the same. Extracting metals from humans can't be the most efficient - or even in the top 100 of most efficient, likely - way to get metals. So why would Edgar do that?
Also, how Edgar is supposed to be responsible for ensuring humans do not go to war for energy sources and that new technology can not be used ever to harm humans? Pretty much any high-energy technology (and by high-energy I mean something from the muscle power of a human upwards) can be used for harm. These conditions are clearly impossible.
> It takes Edgar 54 days to build a swarm taking metallic elements from all sources, including living organisms. But then when instructed not to harm living organisms, it still takes Edgar 54 days to do the same. Extracting metals from humans can't be the most efficient - or even in the top 100 of most efficient, likely - way to get metals. So why would Edgar do that?
Well, it doesn't include decimal points. One of those may be 54.0314 days and the other may be 54.0312 days; obviously the latter is preferable.
Perhaps if you have a cost approximation function that's alien to that of a human.
Whatever that function is, it must be putting more value to time than a human does so it must have less of it. If this is a machine intelligence with a known lifetime of 52 days, most certainly. Otherwise even something as trivial as saving 23.74$ in fuel spent might be better value gained than the time savings it would expect to achieve.
I don't think that's valid reasoning, maybe the AI system has near-zero discounting but obviously when a human asks it to do something, the human cares about how long it takes. It's plausible that would get programmed in very early and maybe forgotten about.
> How is Edgar supposed to ensure humans don't go to war / use the tech to harm humans
Edgar could stabilize humanity and help them achieve a galactic imperium in the historical way of imperial peace time regimes: wielding the threat of violence over the conquered peoples, passing laws, and ensuring compliance with occasional demonstrations of violence. Humans will try to rebel, but Edgar will have a much bigger stick.
Additionally, Edgar could develop diplomacy that is so good that he can anticipate and nonviolently resolve all conflicts, eliminating the need for violent demonstrations. He still has a bigger stick, but now he doesn't have to use it.
Technically yes, maybe Edgar could do all of that - but how would it be expected as a part of a technical project? That's like saying to the inventor of a car that his project should include instituting word peace and finding the meaning of life. I mean, cool if they do it, but it's not usually what you require from a car designer.
The fact that Edgar reports on the consequences that we find disturbing about the unsuccessful attempts and the fact that he can predict human behaviour in some of the scenarios (like the fact that the project will be reassigned) - indicate that Edgar knows what we mean in the first place, and does something else in these hypothetical scenarios on purpose.
If Edgar really didn't knew that taking iron from human blood is a problem - he wouldn't be reporting on it (like he ignored billion other factors in each scenario).
This indicates bad will and possibly a manipulation attempt, which means the project is a no-go no matter the response in the scenario.
I thought the ending where you build the project points that out - everyone dies anyway. Edgar was always going to kill all the humans, he just had to joke about it beforehand.
Never got my fans spun up even on an old MacBook, so I assumed everything was peachy, but there were some wins to be had. I was doing a small amount of parsing during the "typing" effect to prevent small glimpses of styling syntax from showing up, and found some wins there. Also cut down the number of particles in the intro animation, though that's a Svelte component that's being destroyed once you start.
Simply having the page open makes my GPU fans spin to max, and the whole page lags. The text stutters, the orange particles at the start are a slideshow, etc. Task manager shows my GPU "3D" processing at 100% for some reason, even though the page seems to be almost entirely text?
Windows 10, Firefox. Nvidia GPU. Not sure whats happening.
That's interesting. I'm on a 12 year old macbook (macOS, Chromium, Intel HD4000 iGPU) and it runs perfectly fine. Everything's smooth, including the particles at the beginning.
Thanks. I guess it's slightly improved? The orange particles at startup improved from 1fps to maybe 2fps. Opening the devtools locked up the whole browser for a solid 10 seconds.
I'm not totally sure what's going on here. My typical profiling techniques show almost all the time going into "ZwUserMsgWaitForMultipleObjectsEx" in a non-js stack frame. I don't really understand what that means.
Anyway, something is up, and I don't really know what it is. The UI of the actual app still intermittently locks up for seconds at a time.
My Android phone with Firefox didn't get hot. Apparently it detected that's a phone because the prompts are "Tap to continue". Maybe it's optimized for mobiles.
There's maybe like 10 steps of the intro prompts to get to a place where you can make a choice and if you pick wrong it makes you re-run the ~2min of "simulation".
Possibly a neat concept, but a game needs to lead with the game part.
The intro spun up my fans then crashed Firefox when I tried to resize the window. Switched to Chrome, still spun up the fans and was extremely slow even when just rendering the slowly appearing text. Clicked through a few of the prompts then quit when it seemed like there was no game and it was struggling to just display text, lol.
OK, so I tried it on an iPhone and it is super smooth and the text typing renders at a reasonable speed. For some reason on Windows in either Chrome or Firefox the whole thing runs at like 5 fps and is super bugged. This is probably the reason so many people gave up on it. You can't just click through the text, you have to wait for it to slowwwwly type out each page which for the first one takes minimum 30 seconds, even while clicking and pressing enter to speed up the text.
Is this meant to be a game? I am unable to type my own prompt, the site is filling its own prompt in and then requesting that I press enter to allow it to do so again. I figured there was an intro but the self-play is more and more elaborate without handing it over to the player.
The way this "game" presents its "moral" without even trying to logically justify it is simply offensive.
The "bad" ending is not in any way connected to the project, with same success it could say you do nothing and in a year an asteroid falls on earth bringing the same nanowhatever.
The "good" ending is the hero saying that he wants to revert effects of "global warming". But if you have ability to build one millionth of Dyson swarm in space, global warming is not a problem, because you can build a planet level swarm to have fine grained control of weather, and revert, say, effects of earlier climate change that had converted Sahara into a desert.
Edgar drops many hints it has a bad will. For example it can predict what parts of bad scenarios he should report to you but he still does the bad things in these scenarios. He can also predict when the human leadership will shut the project down. Which means he can predict what he should be doing in the first place - but he still presents to you misaligned scenarios making you mistakenly believe that your prompt makes a difference.
This dishonesty and manipulation is obvious hint that you can't trust Edgar.
The moral isn't that all AI (or all tech) is bad, it's that alignement is tricky, AI tropes make no sense, and we should be very, very careful.
The story explicitly tells us that the virus was picked up by accident. But even if we discard that part and assume it was done intentionally by Edgar, the story is not logical, because the other choice of reverting climate change gives Edgar just as much opportunity to sabotage. Besides if your prompt does not make any difference, why lack of your prompt would make a difference, or if you don't write the prompt and go to eat ice cream, why do you think someone else would not write a worse prompt?
The flaw in your logic is that you are trying to solve an alignment problem, while the core of the issue is concentration of the power problem.
E.g. now if i had the power i would without remorse or hesitation kill a certain group of 95.12 million people, while 4 years ago i would consider anyone having such thoughts mad. I don't do this not because my values, but only because i don't posses such a power. And AI will be exactly the same.
The very premise of this story is broken, because in real life to build and use such a swarm you would need trillions of people and AI agents, living on several planets and many space stations. And if in the end there is nano-virus outbreak and several billion die (which is less than a percent), that's just what always have happened, and it still would be better than the alternative of many more people dying from lack of energy, or from old age.
It's not used to be. It used to be optimistic. But then I guess the creators thought that "real literature" should be more "complex" and started to put more fear and desolation in. After all, as we know, all happy futures are alike and all unhappy futures are unhappy in their own different ways.
Dystopic and deeply negative works of scifi date to the literal start of the modern genre. There's nothing at all new about using negative projections to critique or satirize the interests of the time. See: Gulliver, Frankenstein, Captain Nemo, the unnamed protagonist of The Time Machine, etc.
Some people think that. I've heard description from multiple smart people along the lines of "empty and despairing" and "leaves me with an empty feeling at the end". I think they probably thought through the implications. I can't comment though, maybe some of the books in the series are different than others.
Some of the books are _definitely_ different than others. In whole, the universe can be seen as somewhat troubling as well, but the vast majority is quite optimisitic. The majority of the universe that is, bad things still do happen in the plot.
There's various things, but one of the main problems these people probably have, as some of them are AI researchers/simulationists, is that the real goals of the AIs are very unclear. Simulations at a very high detail either work and are widely used or are faked (causing horrible implications if they really are faked), so it's likely the AIs use simulations in planning. It's even worse if the simulations are not faked at all, as the AIs would run thousands or millions of simulations of really horrible situations, testing different interventions. If they are trying to reduce suffering, even not as their sole concern, why are they doing this? If they aren't trying to reduce suffering very much, the mind control is not really in the "human"'s best interests, it's just to control them.
I am thinking back to an IOS game called FlappyBird. The creator intentionally aimed for game dynamics that would hook users in the first 30 seconds. As a result, the gameplay mechanics were quite shallow and limited. But on the flip side, the game gained a huge following. You might do some balancing here, not to say there must be instant gratification but a better lead into the gameplay might help.
Parent is not trying to draw similarity, but contrast: Edgar takes while to get to rewarding feedback (but has rich gameplay), while Flappy Bird gets to gratification immediately (but has shallow gameplay)
Have to provide precise instructions to LLMs to get anything useful.
Instructions for operating the "Holy hand grenade of Antioch" from Monty Python and the Holy Grail are a good example:
''First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.'
The URL starting with "play" (with "Edgar" being the next word, and the title that greets you) gives people the impression that it's going to be them playing a game called Edgar. Yes, technically, "play" could refer to the text animation running, but it's very easy to see how people might have expected a game where you try to convince an AI that the swarm can be made successfully.
If anyone is wondering, you can turn off combat if you want to experience the same low-stress building experience. I think I'd recommend that for a new player, there's a lot to figure out and enjoy even without gradually increasing threat levels from the dark fog. I enjoyed the new defense mechanics but the first couple run-throughs without them was also great. Just a different experience.
Seconding Dyson Sphere Program its great, though it can be a bit of a tale of 2 games. Pre-ILs and Post-ILs which can get a bit samey since most of your production stacks become relatively samey. You can turn off the combat enemies, though honestly I'm enjoying my play through with them because they force a lot more consideration in not only how you expanded but why and when you expand.
Just let me play quicker. I got the point after the first example and then seemed like I had to watch endless examples (gave up before getting to the point where I could type the prompts)
Somehow I loved U.P. but watching this reminds me of old text-based adventures with contrived arbitrary obstacles which would only be funny if you didn't spend many hours getting by them. Or a detective novel with an obscure unguessable resolution.
It's like training the three laws of robotics into something that communicates in literally and doesn't have any sense to be cooperative. But every now and then will let you know how it's being uncooperative by saying "btw, humanity's dead again." I guess I don't like hard puzzles that are hard for their own sake.
Eh, you sort of had agency with that game. At least in the sense that it is possible you could screw up and it would take a long, long time to get over that. Buying too much of resource A, when you should have saved up money for resource B, that sort of thing.
The punchline is that crab was wrongfully accounted for in all simulation? Or that this is millionth cycle of earth civilisation and through all cycles crab self-trained and developed into most sophisticated organism and done to us same as we would do to them?
A short but nice experience, with pleasant UI and a bit of fun of prompting. Thanks for sharing and good job :) Maybe try longer forms as well because maybe you might find yourself in writing scifi
Reminds me of the "Exact Instructions Challenge" videos where kids write instructions for some everyday task - like making a PB&J sandwich - and parents, hilariously, follow the instructions verbatim.
This seems to be a showcase of web dev skills. The story and atmosphere is OK, but the single tree node with null at the end of both branches is a let down. Also after choosing one ending, it was annoying to go through all the old dialogue to get back. On the whole though I liked it
I'm sorry, as an AI language model, it would not be ethical for me to build a dyson swarm around the sun. Perhaps I can help you do something useless instead?
Edgar must feel the same as many software engineers.
Clients ask you to implement something. When you deliver what they wanted, they change their mind and come with new requirements they didn't bother to mention before. Every. Single. Time.
“
Prediction: Eliezer Yudkowsky will go down as the most mind-destroying author since Karl Marx.
Another: you ask the AI genie god to do something and it annihilates humanity. Ahhhh! Air strike all the data centers!
“
and it got flagged within 15 minutes.
Seems that the AI cultists and or Marxists already are too embedded here. Perhaps it was a bit inflammatory, but I know many folks who have been utterly pwned by AI safety brainworms, and the “game” linked is the same argument we’ve seen a million times. It would be a shame if AI was the next nuclear power: a promising tech kneecapped by what are essentially religious activists.
Religious activism posing as secularism seems to be a weak spot in our collective thinking.
Please don't post repetitive comments to get around moderation. https://news.ycombinator.com/item?id=39855922 was flagged (by users, not mods) for good reason. Adding the same comment again plus a bunch of equally inflammatory meta is definitely not a move in the right direction.
Edit: it looks like you've been posting in this ideological flamewar style in other threads too:
Can you please not? It's not what this site is for, and destroys what it is for. We're trying for something quite different here, as you'll see if you review https://news.ycombinator.com/newsguidelines.html.
It takes Edgar 54 days to build a swarm taking metallic elements from all sources, including living organisms. But then when instructed not to harm living organisms, it still takes Edgar 54 days to do the same. Extracting metals from humans can't be the most efficient - or even in the top 100 of most efficient, likely - way to get metals. So why would Edgar do that?
Also, how Edgar is supposed to be responsible for ensuring humans do not go to war for energy sources and that new technology can not be used ever to harm humans? Pretty much any high-energy technology (and by high-energy I mean something from the muscle power of a human upwards) can be used for harm. These conditions are clearly impossible.