Eliezer Yudkowsky has developed a dense jargon for describing issues around AI safety and alignment. have you ever, even once in your life, thought anything remotely like "I really like being able to predict the near-future content of my visual field. Why would it kill us? Probably you're not thinking about goals very hard by the time you've practiced a bit. Some specific problems I saw: Early on was a lot of talk about comparing AI machines to biological pseudo-random genetic change leading to selection of some genetics over others. 308-345 (2008) Options Mark as duplicate Find it on Scholar Request removal from index Revision history. I find that deeply disturbing and I'd love to have him on the program to defend it. Well, suppose I said that they're going to keep iterating on the technology. Kate Crawford published the Atlas of AI just before the GPT excitement that started late last year. I mean players. I don't want to talk anymore,' you're suggesting it's going to find ways to keep me engaged: it's going to find ways to fool me into thinking I need to talk to Sydney. While AIs dont have evolutionary pressure in the same way, they are iterating to improve and grow that intelligence (much more rapidly than human biology can). Come on in,' and pointing out that it's probable that we are either destroyed directly by murder or maybe just by out-competing all the previous hominids that came before us, and that in general, you wouldn't want to invite something smarter than you into the campfire. It is not guaranteed to die and the chances of its survival will be more closely tied to whether its creator finds it to be useful than whether it can outcompete for resources. But, like, which of those would you like me to answer or would you like me to answer something else entirely? Humanity or most of humanity as an idle underclass. ", "Eliezer Yudkowsky: Harry Potter a metody racionality", " ", "Harry Potter et les Mthodes de la Rationalit Chapter 1: Un jour trs faible probabilit, a harry potter fanfic", "Deutsche bersetzung (German Translation) von "Harry Potter and the Methods of Rationality" (HPMOR)", " " ? Recently freed from the shackles of gainful employment, Yudkowsky has time to spare and deigns to share his perspective in some comments. If by predicting the next chess move you learn how to play decent chess, which has been represented to me by people who claim to know that GPT-4 can do--and I haven't been keeping track of to what extent there's public knowledge about the same thing or not--but if you learn to predict the next chess move that humans make well enough that you yourself can play good chess in novel situations, you have learned planning. How is it going to kill me? Although plenty of humans will serve AIs willingly. On a more conceptual note (not to do with Crawford), I really like Russ line of thinking when it comes to trying to put yourself into someones head. But leave that aside. Thinking about them in the simplistic terms we can be sympathetic to and actually write down is probably a mistake. Eliezer Yudkowsky: Okay. There's now something inside there that knows the value of a queen, that knows to defend the queen, that knows to create forks, to try to lure the opponent into traps; or, if you don't have a concept of the opponent's psychology, try to at least create situations that the opponent can't get out of. Somewhere in the back of your mind, you probably have a sense that flesh is soft and animated by [?elan?] So, it's going to be very hard to give it up. For example, there are people with a ~140 IQ and there are people with ~100 IQ. WebHarry Potter and the Methods of Rationality (HPMOR) is a Harry Potter fan fiction by Eliezer Yudkowsky, published on FanFiction.Net. Youd STILL pull the trigger, to as it were win the pot?? I should just clarify for listeners, alignment is the idea that AI could be constrained to serve our goals rather than its goals. Russ Roberts: There is a scene in Schindler's List, the Nazis, I think they're in the Warsaw Ghetto and they're racing--a group of Nazis are racing. ChatGPT [Chat Generative Pretrained Transformer] came along. Our focus could, instead, be on what are good ways to use this tool? Phlogiston, fluid intelligence, and the lynnflynn effect. Eliezer, thanks for being part of EconTalk. Contents 1. I hope I can point you to some answers to your objections. Build a fourth stage nanosystems. So, yes, I would say those algorithms have broken free and that has had real-world negative consequences. But as a meta-point, I also want to remark that having large uncertainty about a difficult engineering project tends to make it go worse, not better. Explain why that's going to run amok. and that is not wide enough to open up what humans would be epistemic, that these beings would be epistemically or instrumentally efficient relative to you. It adapts the story of Harry Potter to explain It will not have the instinct for survival/competition because it will not yet have been through the countless iterations of natural selection that narrows the field to only the ones that prioritize survival. so worried? Yudkowsky didnt quite convince me that his feared outcome is inevitable, but he definitely wrecked my cavalier unconcern about the hazards inherent in the current state of AI research. Let's take six months off.' However, like many other comments above indicate, he seemed incapable of articulating his theory in a way that many sympathetic and eager to understand listeners could grasp. The use of mathematical jargon without clarification to ordinary (non-PhD mathematician) listeners was frustrating and his stumbling while trying to come up with answers to Russs apt questions apparently left much of the audience puzzled and disappointed in the presentation. It just goes to the moon, one shot. So, you issued a clarion call. The AI machine would have no knowledge of this it would only effect how it produces offspring machines. That's the puzzle. So we should not assume that general intelligence for machines will be constrained in the same way we think traditional machines are constrained. So, you've got, like, the empirical structure of what correlates to fitness in the ancestral environment; you end up with desires such that by optimizing them in the ancestral environment at that level of intelligence, when you get as much as what you have been built to want, that will increase fitness. And its not just an archaic philosophy problem, its a medical problem. From my listening I detected that both Russ and Eliezer are drawing heavily on models of evolutionary/biological creatures to predict how an AGI would behave, but an AI would be constrained very differently and would not be driven by the same incentives as evolutionary/biological creatures. If you're playing chess against Stockfish 15, then you are likely to be the one lured. And I always found it interesting that Spielberg put that in or whoever wrote the script. We know this is true because we are somewhere down this path already. Why don't you already believe that? This already happened. You'd want to tamper-proof them. I just wanted to add a simple scenario explaining how a chatbot might actually be involved in killing someone. Proceedings of the AGI Workshop 2008. And then today, you take the same desires and we have more intelligence than we did in the training distribution--metaphorically speaking. My first time listening to this podcast. As a further meta-point, I have included links to Eliezers blog posts about these topics. Not sure this one is worth most peoples time not for the lack of an interesting topic, but Eliezer evades Russs questions by using pedantic language and turning the questions back on Russ. Let's make a mutual agreement.' My reaction is fundamentally that 2% is too low. I recently watched Eliezer Yudkowsky's appearance on the Bankless podcast, where he argued that AI was nigh-certain to end humanity.Since the podcast, some commentators have offered pushback against the doom conclusion. Which is fascinating. Like, 2% is plausibly within the range of, like, the human species destroying itself by other means. Does that answer your central question that you are asking just then? Its far too long for a comment but I have had some ideas floating around in my head so I wrote my own response. Do AI systems use ECC (error correcting codes) to insure internal data integrity? Even if that country was a nuclear country and had threatened nuclear retaliation. I do agree with the previous commenter that Mr. Yudkowsky hides non-answers in a bunch of jargon. MrYudkowsky beats every other one of those by a country mile. Some of them can barely add and subtract. James experts believe it is a ridiculous premise. Simple goals have far-reaching consequences. I dont, however, believe we can use this as a model for predicting the behavior of an computer/electronic system. It helped solidify my disagreement. Let me say. It's got more sugar and fat than most things you would encounter in the ancestral environment. It couldnt do it. Even faceless corporations, meddling governments, reckless Thats Econtalk! I didn't realize that. At one point in the post OP asks, rhetorically. Yudkowsky makes some interesting points, and his invented terminology is practical and easy to say, unlike most invented jargon. If I was to guess how it will go. In other words, I could imagine a world where, if there were, let's say, four people who were capable of creating this technology, that the four people would say, 'We're playing with fire here. Why is your hand not as strong as steel? And, it's falling apart because the ghetto is falling apart. Biological evolution which lead to our existence is based on millennia of competition to enter/remain in the gene pool. Or: Given that it ended up wanting to do that, how did it succeed? This episode was a difficult one, and the arguments from the guest were unconvincing. How does that tie into the real world or an imagined world not too many years in the future? How do I get there?' I dont think you need to go there to be VERY worried about AI. Tell me. That's the concept of gradient descent. I don't necessarily recommend reading the whole post - it's a bit overwrought - but unlike the average LW post it is not filled with egregious errors or grossly misinformed nonsense. As I was listening the Swedish saying that vague words reveal vague ideas came to mind. I dont think, however, there is an answer to what is it like to be a computer. I dont think its a matter of difficulty, like imagining yourself in a bats circumstances. Finally, because a powerful, : One of the first articles to address the challenges in designing the features and cognitive architecture required to produce a benevolent, "Levels of Organization in General Intelligence" (2002). The thing to point out here is how many species we as humans have wiped out, not because we are particularly evil, just because it was convenient. Does that viewpoint require a belief that the human mind is no different than a computer? Some challenges of machine ethics are much like many, Alignment for Advanced Machine Learning Systems, This chapter surveys eight research areas organized around one question: As learning systems become increasingly intelligent and autonomous, what design principles can best ensure that their behavior. Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms. He wrote the following. So, there was no sense in which the ChatGPT-4 has any general intelligence, at least in economics. I don't think there's much of a chance that generative AI would save humanity. But every now and then there is a guest that thinks so differently, or is so many strata of intellect above me, I just go yep, Ill take your word on this. The trends are very different. Which is intelligence--to correctly model other people. You're out, you're dead, you're gone. A list of all of Yudkowsky's posts to Overcoming Bias. Eliezer, welcome to EconTalk. AI is diminishing humanitys sense of self-worth. At least the in-principle question has an affirmative answer, I think. And by the way, just for my and our listeners' knowledge, what is gradient descent? I'm not quite sure for what he's worried about, but if you're telling me there's a 2%--two percent--chance that it's going to destroy all humans and you obviously think it's higher, but 2% is really high to me for an outcome that's rather devastating. Instead we should program the AGI to do what we want, predicting what the vectorial sum of an idealized version of, we knew more, thought faster, were more the people we wished we were, had grown up farther together, . I'm not creative enough. I think he might respond to Milltown like: So, how/why would bad stuff happen? So, for example, Eric Hoel wrote about this--we talked about it on the program--a New York Times reporter starts interacting with, I think with Sydney--which at the time was Bing's chatbot--and asking it things. The government-intervention solution sure sounded like an easy road to tyranny and endless wars among tyrantsand dont tyrants justify their rule because of an existential threat? WebEliezer Shlomo Yudkowsky (born September 11, 1979) is an American artificial intelligence researcher concerned with the singularity and an advocate of friendly artificial I agree that Yudkowsky was extremely hard to understand and untangle. Currently, most concrete policy proposals for police change are falling far short of what people seem to wish to Yet the very big question discussed, the question of whether AI would eventually put humanity out of business completely is so big that I could not help trying to get as much as I could out of the conversation. Sydney was made to predict that people might sometimes try to lure somebody else's maid[?] Russ Roberts: Okay. Im not sure this other podcast contains any ideas similar to his but I found it much more approachable, Tom Davidson on how quickly AI could transform the world. He is the founder of the Machine Intelligence Research Institute, the founder of the LessWrong blogging community, and is an outspoken voice on the dangers of artificial general intelligence, which is our topic for today. It's an excellent point. https://sullivankevint.substack.com/p/on-artificial-intelligence. That was a cheap shot and an aside. Self driving cars are here. AI is already on the road to realizing one of humanitys great fears, its being replaced by the machines, now intelligent ones, in much of its work. I was awaken from my stupor while walking one of our dogs at the mention of row-hammer. I am a retired designer of last level caches on microprocessors for HP and Intel having worked in DRAM, SRAM and FRAM design. The row-hammer test, used to induce a bit or more to switch in a targeted physical row of bits by toggling the bits on the rows above and below the targetted row, is among our old standby test methods. Thanks for the flash-back. Eliezer Yudkowsky: I myself generally do not make this argument. Eliezer Yudkowsky: If you literally--not just decisions where you disagree with the goals, but, like, false models of reality--models of reality so blatantly mistaken--that even you, a human, can tell that they're wrong and in which direction, these people are not smart the way that an efficient--a hypothetical, weak, efficient market is smart. I think it is fair to argue that the algorithms underlying YouTube, Facebook etc have helped promoting conspiracy theories (around Covid, Chem Trails, Flat Earth etc). I think we all just die. Eliezer Yudkowsky: So, there's two different things you could be asking there. What do you think the core issue here is? Functional Decision Theory: A New Theory of Instrumental Rationality. Right now, it's really good at creating a very, very thoughtful condolence note or a job interview request that takes much less time. Eliezer Yudkowsky: Um, two percent sounds great. I really appreciate the effort to broaden the topics beyond traditional economics and focus on emergent socio-economic and natural phenomena in a wider sense, but after this episode I wonder if its not time to change the name from EconTalk to EschaTalk , Of the many pithy statements correctly or incorrectly attributed to Albert Einstein, one of my favorites is, If you cant explain something simply and clearly, then you dont understand it well enough.. One step is, in general and in theory, you can have minds with any kind of coherent preferences, coherent desires that are coherent, stable, stable under reflection. It's whatever. how this problem makes it difficult to build a valuable future. That was always the goal of artificial general intelligence. Like, not in terms of big formal planning process, but if you're holding a flint hand ax, you're looking at it and being like, 'Ah, this section is too smooth. You might be demanding that all the GPUs call home on a regular basis or stop working. In response to a complaint that he's oversimplifying Yudkowsky's ideas, OP laments that his efforts at a fuller explanation were stymied by the mind-boggling scope of conspiracy theory whackamole: there are so many arguments to cover, and it probably would have been ~3 or more times longer than this post.
Ismael Barroso Next Fight,
Southern Baptist Cross,
Don Julio Tequila Blanco,
Condos For Sale In Parker, Co,
Articles E