Slate Star Codex Podcast

Gwern has answered my prayers and taught GPT-2 poetry. GPT-2 is the language processing system that OpenAI announced a few weeks ago. They are keeping the full version secret, but have released a smaller prototype version. Gwern retrained it on the Gutenberg Poetry Corpus, a 117 MB collection of pre-1923 English poetry, to create a specialized poetry AI. I previously tested the out-of-the-box version of GPT-2 and couldn’t make it understand rhyme and meter.

Here’s a graph of US air pollution over time: During the discussion of 90s environmentalism, some people pointed out that this showed the Clean Air Act didn’t matter. The trend is the same before the Act as after it. This kind of argument is common. For example, here’s the libertarian Mercatus Institute arguing that OSHA didn’t help workplace safety: I’ve always taken these arguments pretty seriously. But recently I’ve gotten more cautious.

[Related to: Book Review: Albion’s Seed] [Epistemic status: Not too serious] I realize I’ve been confusing everyone with my use of the word “Puritan”. When I say “That guy is so Puritan!” people object “But he’s not religious!” or “He doesn’t hate fun!” I don’t know what the real word for the category I’m calling “Puritan” is. Words like “Yankee”, “Boston Brahmin”, or “Transcendentalist” are close, but none of them really work. “Eccentric overeducated h

I. Albion’s Seed by David Fischer is a history professor’s nine-hundred-page treatise on patterns of early immigration to the Eastern United States. It’s not light reading and not the sort of thing I would normally pick up. I read it anyway on the advice of people who kept telling me it explains everything about America. And it sort of does. In school, we tend to think of the original American colonists as “Englishmen”,

Last week the FDA approved esketamine for treatment-resistant depression. Let’s review how the pharmaceutical industry works: a company discovers and patents a potentially exciting new drug. They spend tens of millions of dollars proving safety and efficacy to the FDA. The FDA rewards them with a 10ish year monopoly on the drug, during which they can charge whatever ridiculous price they want. This isn’t a great system, but at least we get new medicines sometimes.

[Title from this unrelated story or this unrelated essay] Last week I wrote about how conspiracy theories spread so much faster on Facebook than debunkings of those same theories. A few commenters chimed in to say that of course this was true, the conspiracy theories had evolved into an almost-perfect form for exploiting cognitive biases and the pressures of social media. Debunkings and true beliefs couldn’t copy that process, so they were losing out. This sounded like a challenge, so here you go:

[With apologies to Putnam, Pope, and all of you] Two children are reading a text written by an AI: The hobbits splashed water in each other’s faces until they were both sopping wet One child says to the other “Wow! After reading some text, the AI understands what water is!” The second child says “It doesn’t really understand.” The first child says “Sure it does! It understands that water is the sort of substance that splashes.

The Verge writes a story (an exposé?) on the Facebook-moderation industry. It goes through the standard ways it maltreats its employees: low pay, limited bathroom breaks, awful managers – and then into some not-so-standard ones. Mods have to read (or watch) all of the worst things people post on Facebook, from conspiracy theories to snuff videos. The story talks about the psychological trauma this inflicts:

Imagine a black box which, when you pressed a button, would generate a scientific hypothesis. 50% of its hypotheses are false; 50% are true hypotheses as game-changing and elegant as relativity. Even despite the error rate, it’s easy to see this box would quickly surpass space capsules, da Vinci paintings, and printer ink cartridges to become the most valuable object in the world. Scientific progress on demand, and all you have to do is test some stuff to see if it’s true?

[Epistemic status: I am basing this on widely-accepted published research, but I can’t guarantee I’ve understood the research right or managed to emphasize/believe the right people. Some light editing to bring in important points people raised in the comments.] You all know this graph:     Median wages tracked productivity until 1973, then stopped. Productivity kept growing, but wages remained stagnant.

[This post is having major technical issues. Some comments may not be appearing. If you can’t comment, please say so on the subreddit.] I. I Come To Praise Caesar, Not To Bury Him Several years ago, an SSC reader made an r/slatestarcodex subreddit for discussion of blog posts here and related topics. As per the usual process, the topics that generated the strongest emotions – Trump, gender, race, the communist menace, the fascist menace, etc

I was going back over yesterday’s post, and something sounded familiar about this paragraph: A very careless plagiarist takes someone else’s work and copies it verbatim: “The mitochondria is the powerhouse of the cell”. A more careful plagiarist takes the work and changes a few words around: “The mitochondria is the energy dynamo of the cell”. A plagiarist who is more careful still changes the entire sentence structure: “In cells, mitochondria are the energy dynamos”. The most careful pla

A machine learning researcher writes me in response to yesterday’s post, saying: I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked. I resisted the urge to answer “Yeah, well, your mom is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.” But I think it would have been true.

Last week OpenAI announced its latest breakthrough. GPT-2 is a language model that can write essays to a prompt, answer questions, and summarize longer works. For example (bold prompt is human-written, all other text is the machine-generated response): Prompt: For today’s homework assignment, please describe the reasons for the US Civil War. It is easy to identify why the Civil War happened, because so many people and so many books and so much television and films tell us

I. Chefs. Hundreds of them. Tall chefs, short chefs, black chefs, white chefs. I pushed forward through them, like an explorer hacking away at undergrowth. They muttered curses at me, but I was stronger than they were. I came to a door. I opened it. Sweet empty space. I shut the door behind me, sat down in the chair. “Hello,” I said. “Detective Paul Eastman, pleased to make your acquaintance.”