Future of Life Institute Podcast

The impacts of the climate crisis don’t stop at rising sea levels and changing weather patterns. Episode 7 of Not Cool covers the national security implications of the changing climate, from the economic fallout to the uptick in human migration. Ariel is joined by Lindsay Getschel, a national security and climate change researcher who briefed the UN Security Council this year on these threats. Lindsay also discusses how hard-hit communities are adapting, why UN involvement is important, and more. Topics discussed include: -Threat multipliers -Economic impacts of climate change -Impacts of climate change on migration -The importance of UN involvement -Ecosystem-based adaptation -Action individuals can take

What is geoengineering, and could it really help us solve the climate crisis? The sixth episode of Not Cool features Dr. Alan Robock, meteorologist and climate scientist, on types of geoengineering solutions, the benefits and risks of geoengineering, and the likelihood that we may need to implement such technology. He also discusses a range of other solutions, including economic and policy reforms, shifts within the energy sector, and the type of leadership that might make these transformations possible. Topics discussed include: -Types of geoengineering, including carbon dioxide removal and solar radiation management -Current geoengineering capabilities -The Year Without a Summer -The termination problem -Feasibility of geoengineering solutions -Social cost of carbon -Fossil fuel industry -Renewable energy solutions and economic accessibility -Biggest risks of stratospheric geoengineering

In his Research Agenda v0.9: Synthesizing a human's preferences into a utility function, Stuart Armstrong develops an approach for generating friendly artificial intelligence. His alignment proposal can broadly be understood as a kind of inverse reinforcement learning where most of the task of inferring human preferences is left to the AI itself. It's up to us to build the correct assumptions, definitions, preference learning methodology, and synthesis process into the AI system such that it will be able to meaningfully learn human preferences and synthesize them into an adequate utility function. In order to get this all right, his agenda looks at how to understand and identify human partial preferences, how to ultimately synthesize these learned preferences into an "adequate" utility function, the practicalities of developing and estimating the human utility function, and how this agenda can assist in other methods of AI alignment. Topics discussed in this episode include: -The core aspects and ideas of Stuart's research agenda -Human values being changeable, manipulable, contradictory, and underdefined -This research agenda in the context of the broader AI alignment landscape -What the proposed synthesis process looks like -How to identify human partial preferences -Why a utility function anyway? -Idealization and reflective equilibrium -Open questions and potential problem areas Here you can find the podcast page: https://futureoflife.org/2019/09/17/synthesizing-a-humans-preferences-into-a-utility-function-with-stuart-armstrong/ Important timestamps:  0:00 Introductions  3:24 A story of evolution (inspiring just-so story) 6:30 How does your “inspiring just-so story” help to inform this research agenda? 8:53 The two core parts to the research agenda  10:00 How this research agenda is contextualized in the AI alignment landscape 12:45 The fundamental ideas behind the research project  15:10 What are partial preferences?  17:50 Why reflexive self-consistency isn’t enough  20:05 How are humans contradictory and how does this affect the difficulty of the agenda? 25:30 Why human values being underdefined presents the greatest challenge  33:55 Expanding on the synthesis process  35:20 How to extract the partial preferences of the person  36:50 Why a utility function?  41:45 Are there alternative goal ordering or action producing methods for agents other than utility functions? 44:40 Extending and normalizing partial preferences and covering the rest of section 2  50:00 Moving into section 3, synthesizing the utility function in practice  52:00 Why this research agenda is helpful for other alignment methodologies  55:50 Limits of the agenda and other problems  58:40 Synthesizing a species wide utility function  1:01:20 Concerns over the alignment methodology containing leaky abstractions  1:06:10 Reflective equilibrium and the agenda not being a philosophical ideal  1:08:10 Can we check the result of the synthesis process? 01:09:55 How did the Mahatma Armstrong idealization process fail?  01:14:40 Any clarifications for the AI alignment community?  You Can take a short (4 minute) survey to share your feedback about the podcast here: www.surveymonkey.com/r/YWHDFV7

Planning for climate change is particularly difficult because we're dealing with such big unknowns. How, exactly, will the climate change? Who will be affected and how? What new innovations are possible, and how might they help address or exacerbate the current problem? Etc. But we at least know that in order to minimize the negative effects of climate change, we need to make major structural changes — to our energy systems, to our infrastructure, to our power structures — and we need to start now. On the fifth episode of Not Cool, Ariel is joined by Ken Caldeira, who is a climate scientist at the Carnegie Institution for Science and the Department of Global Ecology and a professor at Stanford University's Department of Earth System Science. Ken shares his thoughts on the changes we need to be making, the obstacles standing in the way, and what it will take to overcome them. Topics discussed include: -Relationship between policy and science -Climate deniers and why it isn't useful to argue with them -Energy systems and replacing carbon -Planning in the face of uncertainty -Sociopolitical/psychological barriers to climate action -Most urgently needed policies and actions -Economic scope of climate solution -Infrastructure solutions and their political viability -Importance of political/systemic change

The reality is, no matter what we do going forward, we’ve already changed the climate. So while it’s critical to try to minimize those changes, it’s also important that we start to prepare for them. On Episode 4 of Not Cool, Ariel explores the concept of climate adaptation — what it means, how it’s being implemented, and where there’s still work to be done. She’s joined by Jessica Troni, head of UN Environment’s Climate Change Adaptation Unit, who talks warming scenarios, adaptation strategies, implementation barriers, and more. Topics discussed include: Climate adaptation: ecology-based, infrastructure Funding sources Barriers: financial, absorptive capacity Developed vs. developing nations: difference in adaptation approaches, needs, etc. UN Environment Policy solutions Social unrest in relation to climate Feedback loops and runaway climate change Warming scenarios What individuals can do

What is a climate tipping point, and how do we know when we’re getting close to one? On Episode 3 of Not Cool, Ariel talks to Dr. Tim Lenton, Professor and Chair in Earth System Science and Climate Change at the University of Exeter and Director of the Global Systems Institute. Tim explains the shifting system dynamics that underlie phenomena like glacial retreat and the disruption of monsoons, as well as their consequences. He also discusses how to deal with low certainty/high stakes risks, what types of policies we most need to be implementing, and how humanity’s unique self-awareness impacts our relationship with the Earth. Topics discussed include: Climate tipping points: impacts, warning signals Evidence that climate is nearing tipping point? IPCC warming targets Risk management under uncertainty Climate policies Human tipping points: social, economic, technological The Gaia Hypothesis

On the second episode of Not Cool, Ariel delves into some of the basic science behind climate change and the history of its study. She is joined by Dr. Joanna Haigh, an atmospheric physicist whose work has been foundational to our current understanding of how the climate works. Joanna is a fellow of The Royal Society and recently retired as Co-Director of the Grantham Institute on Climate Change and the Environment at Imperial College London. Here, she gives a historical overview of the field of climate science and the major breakthroughs that moved it forward. She also discusses her own work on the stratosphere, radiative forcing, solar variability, and more. Topics discussed include: History of the study of climate change Overview of climate modeling Radiative forcing What’s changed in climate science in the past few decades How to distinguish between natural climate variation and human-induced global warming Solar variability, sun spots, and the effect of the sun on the climate History of climate denial

On the premier of Not Cool, Ariel is joined by John Cook: psychologist, climate change communication researcher, and founder of SkepticalScience.com. Much of John’s work focuses on misinformation related to climate change, how it’s propagated, and how to counter it. He offers a historical analysis of climate denial and the motivations behind it, and he debunks some of its most persistent myths. John also discusses his own research on perceived social consensus, the phenomenon he’s termed “climate silence,” and more. Topics discussed include: History of of the study of climate change Climate denial: history and motivations Persistent climate myths How to overcome misinformation How to talk to climate deniers Perceived social consensus and climate silence

In this short trailer, Ariel Conn talks about FLI's newest podcast series, Not Cool: A Climate Conversation. Climate change, to state the obvious, is a huge and complicated problem. Unlike the threats posed by artificial intelligence, biotechnology or nuclear weapons, you don’t need to have an advanced science degree or be a high-ranking government official to start having a meaningful impact on your own carbon footprint. Each of us can begin making lifestyle changes today that will help. We started this podcast because the news about climate change seems to get worse with each new article and report, but the solutions, at least as reported, remain vague and elusive. We wanted to hear from the scientists and experts themselves to learn what’s really going on and how we can all come together to solve this crisis.

Discussions of Chinese artificial intelligence often center around the trope of a U.S.-China arms race. On this month’s FLI podcast, we’re moving beyond this narrative and taking a closer look at the realities of AI in China and what they really mean for the United States. Experts Helen Toner and Elsa Kania, both of Georgetown University’s Center for Security and Emerging Technology, discuss China’s rise as a world AI power, the relationship between the Chinese tech industry and the military, and the use of AI in human rights abuses by the Chinese government. They also touch on Chinese-American technological collaboration, technological difficulties facing China, and what may determine international competitive advantage going forward. Topics discussed in this episode include: The rise of AI in China The escalation of tensions between U.S. and China in AI realm Chinese AI Development plans and policy initiatives The AI arms race narrative and the problems with it Civil-military fusion in China vs. U.S. The regulation of Chinese-American technological collaboration AI and authoritarianism Openness in AI research and when it is (and isn’t) appropriate The relationship between privacy and advancement in AI

"In July 2017, The State Council of China released the New Generation Artificial Intelligence Development Plan. This policy outlines China’s strategy to build a domestic AI industry worth nearly US$150 billion in the next few years and to become the leading AI power by 2030. This officially marked the development of the AI sector as a national priority and it was included in President Xi Jinping’s grand vision for China." (FLI's AI Policy - China page) In the context of these developments and an increase in conversations regarding AI and China, Lucas spoke with Jeffrey Ding from the Center for the Governance of AI (GovAI). Jeffrey is the China lead for GovAI where he researches China's AI development and strategy, as well as China's approach to strategic technologies more generally. Topics discussed in this episode include: -China's historical relationships with technology development -China's AI goals and some recently released principles -Jeffrey Ding's work, Deciphering China's AI Dream -The central drivers of AI and the resulting Chinese AI strategy -Chinese AI capabilities -AGI and superintelligence awareness and thinking in China -Dispelling AI myths, promoting appropriate memes -What healthy competition between the US and China might look like Here you can find the page for this podcast: https://futureoflife.org/2019/08/16/chinas-ai-superpower-dream-with-jeffrey-ding/ Important timestamps:  0:00 Intro  2:14 Motivations for the conversation 5:44 Historical background on China and AI  8:13 AI principles in China and the US  16:20 Jeffrey Ding’s work, Deciphering China’s AI Dream  21:55 Does China’s government play a central hand in setting regulations?  23:25 Can Chinese implementation of regulations and standards move faster than in the US? Is China buying shares in companies to have decision making power?  27:05 The components and drivers of AI in China and how they affect Chinese AI strategy  35:30 Chinese government guidance funds for AI development  37:30 Analyzing China’s AI capabilities  44:20 Implications for the future of AI and AI strategy given the current state of the world  49:30 How important are AGI and superintelligence concerns in China? 52:30 Are there explicit technical AI research programs in China for AGI?  53:40 Dispelling AI myths and promoting appropriate memes 56:10 Relative and absolute gains in international politics  59:11 On Peter Thiel’s recent comments on superintelligence, AI, and China  1:04:10 Major updates and changes since Jeffrey wrote Deciphering China’s AI Dream  1:05:50 What does healthy competition between China and the US look like?  1:11:05 Where to follow Jeffrey and read more of his work You Can take a short (4 minute) survey to share your feedback about the podcast here: https://www.surveymonkey.com/r/YWHDFV7 Deciphering China's AI Dream: https://www.fhi.ox.ac.uk/wp-content/uploads/Deciphering_Chinas_AI-Dream.pdf FLI AI Policy - China page: https://futureoflife.org/ai-policy-china/ ChinAI Newsletter: https://chinai.substack.com Jeff's Twitter: https://twitter.com/jjding99 Previous podcast with Jeffrey: https://youtu.be/tm2kmSQNUAU

Does the climate crisis pose an existential threat? And is that even the best way to formulate the question, or should we be looking at the relationship between the climate crisis and existential threats differently? In this month’s FLI podcast, Ariel was joined by Simon Beard and Haydn Belfield of the University of Cambridge’s Center for the Study of Existential Risk (CSER), who explained why, despite the many unknowns, it might indeed make sense to study climate change as an existential threat. Simon and Haydn broke down the different systems underlying human civilization and the ways climate change threatens these systems. They also discussed our species’ unique strengths and vulnerabilities –– and the ways in which technology has heightened both –– with respect to the changing climate.

In this podcast, Lucas spoke with Jade Leung from the Center for the Governance of AI (GovAI). GovAI strives to help humanity capture the benefits and mitigate the risks of artificial intelligence. The center focuses on the political challenges arising from transformative AI, and they seek to guide the development of such technology for the common good by researching issues in AI governance and advising decision makers. Jade is Head of Research and Partnerships at GovAI, where her research focuses on modeling the politics of strategic general purpose technologies, with the intention of understanding which dynamics seed cooperation and conflict. Topics discussed in this episode include: -The landscape of AI governance -The Center for the Governance of AI’s research agenda and priorities -Aligning government and companies with ideal governance and the common good -Norms and efforts in the AI alignment community in this space -Technical AI alignment vs. AI Governance vs. malicious use cases -Lethal autonomous weapons -Where we are in terms of our efforts and what further work is needed in this space You can take a short (3 minute) survey to share your feedback about the podcast here: www.surveymonkey.com/r/YWHDFV7

Nuclear weapons testing is mostly a thing of the past: The last nuclear weapon test explosion on US soil was conducted over 25 years ago. But how much longer can nuclear weapons testing remain a taboo that almost no country will violate? In an official statement from the end of May, the Director of the U.S. Defense Intelligence Agency (DIA) expressed the belief that both Russia and China were preparing for explosive tests of low-yield nuclear weapons, if not already testing. Such accusations could potentially be used by the U.S. to justify a breach of the Comprehensive Nuclear-Test-Ban Treaty (CTBT). This month, Ariel was joined by Jeffrey Lewis, Director of the East Asia Nonproliferation Program at the Center for Nonproliferation Studies and founder of armscontrolwonk.com, and Alex Bell, Senior Policy Director at the Center for Arms Control and Non-Proliferation. Lewis and Bell discuss the DIA’s allegations, the history of the CTBT, why it’s in the U.S. interest to ratify the treaty, and more. Topics discussed in this episode: - The validity of the U.S. allegations --Is Russia really testing weapons? - The International Monitoring System -- How effective is it if the treaty isn’t in effect? - The modernization of U.S/Russian/Chinese nuclear arsenals and what that means. - Why there’s a push for nuclear testing. - Why opposing nuclear testing can help ensure the US maintains nuclear superiority.

In this month’s podcast, Ariel spoke with Ashley Llorens, the Founding Chief of the Intelligent Systems Center at the John Hopkins Applied Physics Laboratory, and Francesca Rossi, the IBM AI Ethics Global Leader at the IBM TJ Watson Research Lab and an FLI board member, about developing AI that will make us safer, more productive, and more creative. Too often, Rossi points out, we build our visions of the future around our current technology. Here, Llorens and Rossi take the opposite approach: let's build our technology around our visions for the future.