What Happens When an Algorithm Helps Write Science Fiction
Source: Stephen Marche

That statement probably requires some explanation. Two researchers named Adam Hammond and Julian Brooke have spent the past few years developing software that analyzes literary databases. Their program can identify dozens of structural and stylistic details in huge chunks of text, and if you give them a collection of great stories—stories that maybe you wished you had written—they are able to identify all the details that those great stories have in common.

That’s where I come in: I write stories for a living. (My last one was about werewolf billionaires. It was fiction.) And I’ve watched technology infiltrate countless trades and crafts, oftentimes improving how people do their jobs, all while passing storytellers by. Where’s the technology that can make me better at my job? Where’s the computational system that will optimize my prose? Hammond and Brooke agreed to collaborate with me on a simple experiment: Can an algorithm help me write a better story? I began by giving them a collection of my 50 favorite sci-fi stories—a mix of golden-age classics and some more recent stuff. (We decided I’d write a science-fiction piece, both for the obvious reasons and because sci-fi is easy to identify.) They used their program to compare my stories to a mass of other stories. First they came back to me with a series of stylistic guidelines that would make my story as much like the samples as possible—things like there had to be four speaking characters and a certain percentage of the text had to be dialog. Then they sent me a set of 14 rules, derived from a process called topic modeling, that would govern my story’s main topics and themes. All I had to do was start writing.

Hammond and Brooke created a web-based interface through which their algorithm, called SciFiQ, could tell me, on the textual equivalent of the atomic level, how closely every single detail of my writing matched the details in my 50 favorite works. (I’m talking “nouns per 100 words” level.) When I typed in a word or phrase and it was more than a little different than what SciFiQ had in mind, the interface would light up red or purple. When I fixed the offending word or phrase, the interface would turn green.

The key, obviously, was the texts that I selected: “Vaster than Empires and More Slow” by Ursula K. Le Guin, “The Father-Thing” by Philip K. Dick, “There Will Come Soft Rains” by Ray Bradbury—I can’t list them all, but you get the idea. I wanted to write something incredible, so I picked stories I thought were incredible. Whether that’s what I got might be another story.



1

“Topic modeling,” Hammond says of the process he and Brooke used to create the 14 rules, “is mathematically sophisticated but otherwise stupid. The algorithm looks for words that tend to occur near one another in a very large corpus of text.” Based on how frequently the words appear together, Hammond determined what my story had to be about. For instance, after finding clusters of words throughout the texts that suggested extraterrestrial worlds and beings, he gave me rule number one: “The story should be set on a planet other than Earth.”

2

The algorithm affected the story much more than I thought it would. Rule number one above seemed to conflict with rule number nine: “Include a scene set on a traditional Earth farm, with apple trees and corn fields.” The only way I could figure out how to follow both rules was to have someone on Earth viewing another planet. Which, I have to say, I like—the feeling that you’re watching helplessly as faraway events transpire. That suits our time, doesn’t it?

The machines sat empty in the dark. Only a single light was on when Anne and Ed entered. A lone searcher was staring at the Other planet1, his face half-swallowed by the viewer, and the empty banks of blank screens2 sloped into the room’s vague emptiness.

“Profitable and marketable,” Ed said. “I cannot stress that enough.” “Profitable and marketable,” Anne murmured in agreement.

The man at the viewer sucked out his face with a faint squelch and, with no acknowledgment of either Anne or Ed, began to pack up as quickly as possible. Anne had overdressed for her first day, obviously. Ed was night supervisor, but he was wearing blue-and-green overalls. The guy at the viewer was in head-to-toe sweats. His sallow eyes were exhausted3. He emanated a grotesque odor of off-brand bleach, and it burned the inside of Anne’s nostrils. And she was wearing her best outfit, the pencil-­skirt outfit she’d bought for her dissertation defense.

3

What most writers and readers consider style (a recognizable way with words) is not what the algorithm considers style. It was developed to analyze average sentence length, variance in paragraph length, verbs per 100 words, and dozens of other statistics and patterns that my story would have to follow.

“Once upon a time,” Ed continued, “people were interested in the Other world just because it was another world. There was discovery. Then there was building the telescopes, carrying the mercury to the translunar observatories, constructing the antigravity bases, the discs within discs of whirling silver the size of cities to capture the light.”

The sallow man Anne was replacing misted the inside of the viewer with antiseptic spray and gently rubbed the screen down with a paper towel. Nodding curtly to each of them in turn, he half-jogged out the door. They were apparently not to be introduced. Her coworker couldn’t wait to be gone.

“If you’re curious, go to the archives. I know, you’re a full prof, full xenologist. I know you’ve spent 10 years in the archives already, but you’ve got four hours tonight, well, three hours and 42 minutes. The archives have a hundred million hours cross-referenced. Your job is to keep looking to find something so we can justify keeping the lights on here.”

“I understand.”

“This light here,” he said, tapping the lamp.

The glow from the viewer that no one was looking into unnerved Anne. The Other world, 1,564 light-years away, was flowing brightly4 and glamorously5 into the machine, unobserved, while Ed gave what must be his boilerplate orientation speech.

4

I wrote a rough draft, based on the rules and guidelines, and dropped it into the interface. The first thing SciFiQ told me was that I used too few adverbs. I’ve always been taught to cut anything ending in ly, and I had to go back over the story putting in adverbs. Absurdly, good science fiction has a lot of adverbs.

5

It wasn’t just adverbs either. It was adverbs per 100 words. So they had to be sprinkled throughout.

“Nobody cares. That’s the thing to remember. While you’re here, I’ll be making phone calls to the South China coast begging for cash. Help me out. Keep the lights on here to keep an eye on there. That’s our motto now.”

“Curiosity isn’t enough,” she said.

“Curiosity isn’t enough. Exactly. You’re starting to understand. When people with money, people who matter, think of the Other, they think of aliens who have been dead for 1,500 years. It’s a nightmare, in a way, a planet of corpses who don’t know the oblivion they have momentarily escaped with us. Everybody knows. If they were ever going to find their way to us, they probably already would have. And if they’re looking at us, which they probably aren’t, what would we have to say to them? So it makes everybody sad, that there’s intelligent life out there and it doesn’t matter much. And sad is a hard sell.”



Ed was obviously wrapping up.

“You’re here to see, not to have insight. You will no doubt be struck by the reality of a planet so similar to ours, so distant from ours, and you will think deep thoughts about the loneliness of the cosmos. You may come to think even about the fate of a universe that is probably one of many universes, exemplified only by the fact that the universe that we happen to reside in happens to have created observers. Don’t bother sharing these digressions. They have already been written down by people who are 10,000 times more perspicacious than you and I and still managed to die in comprehensive obscurity.”

6

The algorithm also told me what percentage of text should be dialog and how much of that dialog should come from female characters. This is where things get embarrassing. Turns out that, based on the stories I chose, only 16. 1 percent of the dialog could be from a woman’s point of view. Which is a crazily low number. Female writers historically write 40 to 50 percent of their dialog for female characters, male writers about 20 percent; so even by the shitty standards of male writers and history, this is appalling. It meant I had to make Anne shy and scholarly, and I had to make all the men around her bloviating assholes. Otherwise the dialog numbers wouldn’t work out.

“Profitable and marketable,”6 Anne repeated.

“That’s correct. So tonight you have fewer than four hours to look at Othertribespeople on a ring of the lesser Chekhovs. Nobody knows much about them. They might have some new medicine. Anything that might have salable value, report.”

7

The female dialog thing is still bugging me. If I had chosen a different 50 stories, or even changed one of the 50 stories, there would be a different outcome. I need to start reading better science fiction.

“So I should call you if I see something new?”7

“Call me if you see an Other holding up a sign that says, ‘Hello, Earth. It’s us up here.’ ”

FSB-ALGO_ChapterBreak1test4.png

______At its peak, the Institution for the Study of Extraterrestrial Life had employed 264 fully trained researchers at the banks of screens. The mania for the Other had gripped the world, and every school devoted a class a week to its study. Universities all over the world had Other departments. Biologists handled the various pockets of life discovered in the rest of the universe, slimes mutating fiercely but drably on dozens of freezing or burning hells. The Other was its own field. The similarity had come as an existential shock to the earth. A planet 1,564 light-years away had forests that were not dissimilar to Earth’s forests. They had animals that were not that unlike the remaining animals on Earth. And they had the Others, who lived in cities, with streets, or in villages, or in tribes, just like us. The Others wore clothes. They fell in love. They wrote books. They kept time. They had laws. The odds of two worlds being conjured by chance at such similar points in their development—the Other was roughly at Earth’s 1964—had to mean something. The anthropic principle was considered proven. The universe could only exist under conditions in which ourselves and the Others were there to witness it. Those were the days when children, like Anne when she was a kid, wore pajamas with patterns of glublefrings gamboling among the tzitziglug trees, and everybody called it The Yonder. But all novelty eventually wears off. The natural market for the shock of recognition is perishingly small.

8

Rule number 11: “Engage the sublime. Consider using the following words: vast, gigantic, strange, radiance, mystery, brilliance, fantastic, and spooky.”

Alone in the vast8 dark room, Anne wiped down the viewer again, just to be sure. She understood why there had been so many conspiracies in the days after discovery. It was like the machine fabricated the planet. Anne placed her face inside. The sucking in of the face curtains sealed her. She was hovering over a planet on the other side of the galaxy, 20 feet over a small group of Othertribespeople at night, fishing.

The quality of the screen was so impeccable that the sense of her own body dissolved, and she was a floating dot. There was no comparison to watching a tape; this was live, or rather it was live 1,564 years ago. The tribe grouped tightly around a mountain stream. The males held torches up to the water, where a flurry of small fishes roiled on or under the surface, and a female Other poised, a spear in her hand, waiting for a gallack. They were huge, the gallacks, nearly the size of an Other. A single fish could feed a group of tribespeople for a month of desert season.

9

The algorithm distinguishes between the “literariness” and “colloquialness” of any given word, and I had to strike the right balance between the two kinds. My number of literary words was apparently too high, so I had to go through the story replacing words like scarlet with words like red.
FSB-ALGO_Spot_4_test1.png

Anne wanted to look a bit more closely. She reached down and her screen went blank. She had zoomed too far. She pulled up with a clenched fist and an elbow curl, and she was among the clouds above the mountains. The fire of the tribe’s torches made a red9 and blue dot in the center. She pushed down slowly, adjusting. She had asked one of her dissertation supervisors what it was like working on the screens and he had told her it was like being an impotent god, and the description was precise. Delicately, tentatively, Anne focused on the face of the Other woman holding a spear. Sometimes a gallack might not come to light for hours, and when it did, it offered maybe three seconds of its purple-streaked skull bone for a strike. The Otherwoman’s eyes had narrowed sharply in concentration, her eyes small, even for the eyes of the Others, who had no nasal bridge, and whose button noses, like tiny dogs, were considerably more powerful than a human nose. A horrific violence lurked in her gaze.

The Others stood so still, so intently and contentedly waiting for a slimy mammoth fish to rise out of the waters. Why was she watching this? The hope was that someone would hurt themselves in the hunt, and that the tribe would use an herb that had found an analogue in the surviving jungles on Earth to repair the damage. That’s how they had found that the bark of the Amazonian gluttaree had curative properties for Bell’s palsy. That was profitable and marketable. Only the leaves on the Other trees—she thought they were hualintratras, or maybe grubgrubs—moved at all10. The shimmering and the stillness were so different from the recordings, somehow. The recordings were always significant. That was the difference. Something had always happened to make them worth watching, worth preserving. The Othertribespeople were just waiting around for a gallack. Maybe the gallack would come, or maybe it wouldn’t.

10

I loved writing descriptions of the Other planet, but I could only include a few. My story had to consist of about 26 percent dialog, so every time I wrote a bit of descriptive non-dialog, I knew I’d have to make up for it elsewhere with some talking. It was like working out probabilities when you’re playing poker.

It wouldn’t really matter if she snuck off to the city for 20 minutes, would it?

She marked the place of the tribe, flicked up with a curled fist, saw the planet whole for a second, found the biggest dot, centered herself visually, and pushed down.

She landed accidentally in a funeral, right in the middle of the green twigs. Curling up, she could see that a ritual was in its final stages, the morbid consummation. The funeral must be in the Middle Space, off the straight avenue. Soon they would have a horrible shattering, a grandiose howl, an unconditional prostration. The crowd was small, six Others, so a prominent Other must have died. The body was already under the branches though, so Anne couldn’t quite tell.

11

Rule number four: “The story should be set in a city. The protagonists should be seeing the city for the first time and should be impressed and dazzled by its scale.”

She pulled up, too quickly, and she was once again too high. She hovered over the whole of the OSC, the Other South City, momentarily dazzled11. There were 24 million Others in the city, more than any city on Earth had held for 50 years, and that was without counting however many were living in the subterranean tunnels. Even at night, glowing with torches over the large avenues, the circles within interlocked circles, orbs within orbs which were so typically a figure of the Southern part of the main Continent, the City Center sprawled haphazardly. So much life. So much life to see.

But all that life was none of her business. Her business was back on the lower Che­khovs. Anne flipped back to the saved locale. The Other­tribespeople were still waiting patiently for a big fish to come to light.

Back in OSC, she floated over the Coil, the central avenue of the biggest Other city. The flashes of the running Others, the tumult of their flat faces. Who to follow? Who to forget?

She followed one Other licking his lips anxiously. He turned off the corner and was gone. She followed another Other woman before she dipped into a store that sold texts. The universe is crammed with fascinating irrelevance. Anne was just watching now. All the work had already been done on the main streets, although it grew out of date so rapidly. When she had been a xenosociologist, she had studied some of the commercial patterns, the gift and theft matrices that seemed to be their version of exchange. That was before her department, and all the other departments except xenolinguistics, had been folded into general xenology. They were all just xenologists now.
FSB-ALGO_Spot_1_test1-1.png

She widened her gaze and drifted into one of the neighborhoods halfway to the Uppertown Stage, or more than halfway if the city was still spreading since she had last read about it. The harsh tangerine dawn was rising on Other children as they played the string game in its labyrinthine star patterns laid out in the sand. She had written one of her first papers in grade school on geometrical erudition in Other children games, an A+. Her teacher, Ms. Norwood, had said, not quite believing it, that she might work at ISEL some day.

She remembered that Ms. Norwood had been a devotee of Wodeck’s theory of distant proprioception, though it had been defunct as a theory even then. By virtue of the Heisenberg principle, Wodeck argued, we must be altering the Others in our observation of them. The idea was too Romantic for the academy or the public, both of whom thought Heisenberg was fine for electrons but not for aliens who had been dead for 1,500 years and whose remains had long since rotted to ashes by the time their light had arrived. The idea was doubly distasteful, because who knew who was watching us, and from where? Who wanted to believe their lives were shaped by alien eyes?

Anne saw another Other girl, to a side of the players, reading pages, so she pushed in, focused, and caught a corner of the text, cut and pasted it into the archive comparer on the off chance it might be new and viable, a late entry into the now mostly unread library of the Other.

12

Rule number six: “Include a pivotal scene in which a group of people escape from a building at night at high speed in a high tech vehicle made of metal and glass.”

Then the book, in the middle of being copied, fluttered from the Other girl’s hands. The Other girl’s face was up, staring, in horrified confusion. Anne flicked over to where the Other child was looking. A smoldering hole had formed in the sand lot beside the children’s play space. A bizarre machine, unlike any devices she had seen in any xenology class, careened12 at top pace down one of the lesser coils. She looked down. An Other man and an Other woman were riding in it, driving. The machine was large and silver. It would fit a bed. The thing must have ripped through the surface. She had never heard of that. She looked closer, and the Other man and the Other woman were carrying a baby, and they had a look of terror and tenderness on their haggard faces, pale from the cruelty of underground life. Anne pulled out with a curled fist, and they had no chance to escape. The restraint work of the Other authorities was always impressive in its brutality. The Others were monsters when it came to crime and punishment and angrily excised any difference with savagery. A remorseless circle of exalters, at least 30 of them, were coiling in on the fleeing Others. How long did they have? She looked back, flipped up. The Other man smiled at the Other woman for some obscure reason, cooed over the infant. She flipped back and the round group of the sinister exalters crept in, and then they all slowed, out of screen. She flipped back up and the strange machine had vanished. She curled up more. The machine had crashed into a boulder, and the Other woman with her baby were burning horribly inside the wreckage, and the Other man, thrown clear, lay dying on the gray sand. The Other man was looking straight up. He was looking straight up at Anne. He was staring at her across the galaxy right into her eye.

FSB-ALGO_ChapterBreak2_test4.png

13

Rule number 10: “Include extended descriptions of intense physical sensations and name the bodily organs that perceive these sensations.” The first part of that rule is generally good writing advice (make ‘em feel it), but the second part is innovative: It’s not just the description but the organs that matter.

______Anne’s face, as it sucked out of the viewer, pulled slightly on the flaps, gently squeezing her eyeballs in their sockets13. Two hours and 17 minutes had passed. Time was always distorted by drifting over the Other, what with a 36 hour, 17 minute, 54 second day. Culture shock is always worse coming home.

“Ed?” She called up the professor’s visuals from control. His face, on Skype, was the haggard face of a begging administrator on one call after another.

“Hi Anne, did they hold up a sign saying ‘Hi, Earth’?”

“I saw something.”

“Is it profitable and marketable?”

Was there profit in that rickety old machine somewhere? Was there some kind of profit in that? Or in the look of sadness on the Other’s face?

“There’s lots of wonderful things to see, Anne. Nobody needs us here to show them a new wonderful thing. The moon shines wonderfully every evening. Nobody needs 70,000-ton telescopes in the sky to show them a place they have never seen before. If we want to keep an eye on, we have to find useful, profitable Otherness. Not the new and wonderful. Got it? ”

“Got it.”

“Profitable and marketable.”

“Profitable and marketable.”

The wreckage was still smoldering gruesomely on the viewer. The corpse of the Other man had already been cleared away. The machine, which must have been cobbled together in the underground, chuffed and spluttered smokily. And there was no way any of it could ever be profitable and marketable.

Anne called Lee, a colleague from graduate school who had worked on subterranean history, and if she was recalling it right, even something with machines. He was living in Cairo these days, she thought, some kind of assistant professor at the uni there.

14

This guy is here—this whole scene is here—because there needed to be four speaking characters and I needed more dialog. If I were just writing it myself, I would probably cut the whole section.

“Is that Anne?” he asked14. He was older, more slovenly than she remembered, but it had been nearly 10 years. She reached him at a Shisha bar on Tahrir Square. “Is that the Anne who is working, I heard, at ISEL and who is actually looking into the sky?”

“That’s me.”

“And what can I do for Anne who has a good job at ISEL where she is looking into the sky?”

“You once, long ago, studied the subterranea right?”

15

Ordinarily, when I’m writing and I’m stuck with a line I don’t like, I work on finding the right way to write that line. The adjective sucks? I find a better adjective or cut the adjective altogether. But, in this case, that’s not enough. If you cut an adjective in one place, you have to put in an adjective somewhere else, and putting in that adjective somewhere else alters the balance of sentence length, paragraph length, paragraph length variation, and so on. It’s a bit like doing a Rubik’s cube. You fix one thing, you’ve messed up the side you weren’t looking at.

The hitch in his voice swelled awkwardly, stringently, into a silence. The envy reached through the phone. Anne remembered. Lee had only managed a lousy archival job15, rustling in 10-year-old tapes for culinary elements. All the best dishes had been transferred years ago.

“Wow. You’re actually at ISEL asking me a question about the subterranean, aren’t you?”
FSB-ALGO_Spot_5_test1.png

“That I am.”

His voice hitched again. “You didn’t see a real breakout, did you?”

“Well, I’m not sure. I just want to know if there’s any history on the machines used in breakouts.”

Lee paused, recognizing that his scholarship might matter, realizing that the Other existed, was existing, and he understood it, understood it usefully.

“Well, a big book on subterranea as a prison system is Nguyen’s Other Underground, but that was 40 years ago or more even. The subterranea’s only had maybe a thousand hours of inspection over the past 20 years.”

“Why is that?”

“I guess they figure if the Others don’t care about it, why would we? People get bored with mysteries after awhile, for sure. And then there was an article a couple of years ago, out of the unit at Oxford. ‘Otherness among the Other,’ but it was general xenosociology. Wasn’t that your field?”

“Before it all folded.”

“Right. We’re all xenologists now. Also, there’s a footnote in my last paper in Otherism on the first escape, but you know all about that. So what can you tell me about your breakout?”

She would be fired for a leak, even with Lee, even for a story no one cared to hear. Systems grow stricter as institutions decline. If there is nothing profitable or marketable in a thing, it must remain a secret or it has no value at all.

FSB-ALGO_ChapterBreak3_test4.png

16

Rule number five: “Part of the action should unfold at night during an intense storm.”

17

One way of looking at this algorithm is as an editor. It’s commissioning a story with guidelines and then forcing me to write it the way it wants. If I don’t do it right, the algorithm makes me do it again, and again, until I get it right.

______Her parents were still up when Anne, sick from the train and suffused with an indefinable and all-suffusing disappointment, rolled through the portico of the family farmstead. She found them in the viewing room, watching a new storm roll ferociously over the cornfields and the apple orchard. Mom was lying down, asleep, with her head on Dad’s lap. The lightning from the storm16 was continuous enough that the room needed no other illumination, and Anne’s skin tingled furtively17 with the electricity in the air. She sat beside her father in the noise of the rain that filled her ears like a cloying syrup.

“How was the first day at ISEL?” he whispered.

“Everything I thought it would be.”

“And what did you think it would be then?”

It was the first time that day that anyone had cared what Anne thought. And at that very moment she didn’t want to see or to record. At that very moment she just wanted to listen to the rain.

“There’s just so much of it,” she said.

“It is another world.”

“And what are we doing looking at it?”

“Keeping an eye on, right?”

“Keeping an eye on what?”

Anne’s father ran a hand through her mother’s hair a few moments.

“This morning I was weighing in my mind that first book of the Other plants and animals we bought you. Remember that?”

“Sure.”

“And those bedroom sheets you wanted so badly, the ones with a little kangaroo-like Other thing on it. What are they called?”

“Calotricks.”

“And now you’re a grown-up woman, and they’re letting you look up in the sky from the big machines at ISEL.”

The storm ripped the sky, harsh as a lash against her eyes. Her dad was proud of her, but she could tell he cared less for the Other world—the distant miracle, a sign however remote that we were not alone in the universe—than whether she would be able to move out now that she had a job. She was about to tell him about the nightmare chase of the burning woman and the dying man and the baby they took with them when her mother roused, and Dad shushed and began to sing:
FSB-ALGO_Spot_3_test1.png

18

I chose the title of the story. Some things the algorithm didn’t get to decide.

19

Did you know this poem was actually written by a person? A woman named Jane Taylor (1783–1824). And it’s so famous that everybody assumes nobody wrote it, that it just kind of appeared. That is the ultimate achievement of writing, that it’s so good that no person could have written it.

Twinkle, twinkle18, little star

How I wonder what you are.

Up above the world so high

Like a diamond in the sky

Twinkle, twinkle little star19

How I wonder what you are.

He picked up his wife and carried her out of the viewing room to bed. Anne was alone, more alone than before.

20

“The fact that it’s really not that bad is kind of remarkable.” That’s how Rich, my human editor, described the story. I’ll take it.

The exhaustion of the day accumulating inside her, she was glad of a half-dark room and a storm. As a child, to be even a cog in the celestial machinery would have been enough. She loved a whole other world, miraculously reflected in a skypiercing eye. She was middle-aged now: There was only light, moving through emptiness, trapped by machines20.
FSB-ALGO_Sidebar_Test2-5.png

Human Critics

WIRED asked two of publishing’s top talent scouts to read this story without knowing who (or, more specifically, what) wrote it. They detected something was amiss.

______“Full of unnecessary detail, wooden, implausible dialog (Who talks like this?), and sentences that don’t actually hold up when you read them carefully. They seem like they hold up, but they don’t. It’s aimless. It uses language to describe things rather than reveal them (flowing “brightly and glamorously,” etc.). That stuff doesn’t sound human—or, better, doesn’t sound writerly. Feels like words on a page.”

Andy Ward, editor in chief, Random House

______“This seems to come from a writer who has an interesting, if still undeveloped, idea and a strong sense of his/her fictional landscape but who hasn’t quite put enough thought into the narrative trajectory of the story or the details of the language, which, line by line, can feel a little pedestrian. There isn’t quite enough character development or narrative movement for this to sustain my interest as a reader. Then again, perhaps this is all setup for a longer story or a novel?”