Disney's AI Judges Short Stories; What's Next For Story Writers?

Artificial intelligence is crossing new barriers every day — while some tech companies are working on giving it imagination, others are working on making it capable of even more complex tasks — for instance, judging short stories. A new study by Disney’s research team and researchers at the University of Massachusetts at Boston has attempted to create artificial intelligence which will be able to understand and evaluate short stories.

"Our neural networks had some success in predicting the popularity of stories. You can't yet use them to pick out winners for your local writing competition, but they can be used to guide future research," stated Boyang Li, a research scientist at Disney Research.

The target of the research is to create an AI-based automated evaluation of content, although it does not have a reference point of large human evaluated story databases since not many exist. The researchers are using crowd-sourced answer website Quora as a yardstick for judging the popularity of stories and training their AI modules instead.

The research team has developed an AI module based on 28,000 narratives picked up from Quora answers with an average word-length of 369 words each and rated on the basis of ‘upvotes’ and ‘downvotes’ on the website.

Seems pretty simple, right?

The research team did not just stop there, they then worked on different story structures, the sequence of events representing a story, how they reveal character intentions and then evaluated their influence on AI based neural networks. They also looked at the effect of different parts of a story, interdependently and holistically.

Using these parameters, the AI module will predict which stories would be most popular with readers.

"The ability to predict narrative quality impacts on both story creation and story understanding. To evaluate quality, the AI needs some level of understanding of the text. And if AIs are to create narratives, they need to be able to judge the quality of what they are producing," said Markus Gross, vice president at Disney Research, according to a news release.

The study claims that the AI module performs 18 percent better than other non-automated, text-based story evaluation modules.

While the AI module being developed by the team does not seem to be in any way close to being a literary critic, AI is slowly edging towards taking on more creative tasks rather than just analytical tasks.

AI has already started writing scripts for short movies — a short movie called Sunspring based on a short science fiction story written entirely by AI was released last year.

Even news outlets like Bloomberg are working on using automation in writing. The company is working on creating what it calls “smart automated content.”

“The computer spots a trend, delivers a portion of a story to you and in essence asks the question: Do you want to add or subtract something to this and then publish it? And it will only count as Bloomberg journalism if you sign off on it,” Bloomberg’s editor-in-chief John Mickletwait stated in a memo to the company’s staff.

While AI is not expected to replace writers and editors soon, chances are rife that somewhere down the line, the story you are reading might be written by a bot and not a human.