A newspaper in Japan is using AI to summarize news stories to get them out quicker.

The Shinano Mainichi Shimbun is working with Fujitsu to speed up its news updates. This is how it works.

By Tim Hornyak
Splice Japan

In another step forward for robo-journalism, a regional newspaper in Japan is rolling out an artificial intelligence system that automatically generates summaries of news articles for distribution across a range of media platforms.

The Shinano Mainichi Shimbun teamed up with Fujitsu, Japan’s largest IT services company, to create the software based on technology developed by Fujitsu Laboratories. Staff at the broadsheet have been producing summaries manually, a task that takes up to five minutes per article. The software creates summaries instantly and with greater accuracy than a different summarizing method that begins with the lead and stops when the word limit is reached, according to Fujitsu.

The system uses a combination of natural language processing and machine learning to pick out the most salient parts of the article, scoring each sentence in terms of importance.

During a trial, it was trained on a dataset of 2,500 articles from the newspaper as well as their manually compiled summaries.

“By pairing the original articles with the summaries and defining that as reference, or teacher data, we built an ‘important sentence extraction model’ that evaluates the content importance according to individual sentences, as well as a ‘sentence-shortening model’ that maintains sentence structure while deleting unnecessary words,” says Masato Yokota, a director at Fujitsu’s State Infrastructure and Finance Business Group.

The software can work with articles written in Japanese or English. It was built with a web API that can be easily inserted into the existing editorial workflow. A “summary” button activating the API was implemented into the editing screen for the paper’s cable TV news, Yokota said.

Shinano-Mainichi-Shimbun-summarization-AI
A screenshot of the AI system from its trial period shows the original article in Japanese (left), an automatically generated ranking of sentences by importance (center), and the summarized text (right).

Robots vs. Journalists

First published in 1873, the Shinano Mainichi Shimbun is one of Japan’s oldest dailies. Headquartered in Nagano, northwest of Tokyo, it claims a morning-edition circulation of 487,000 copies and distribution to 61% of households in Nagano Prefecture.

“The third-wave AI is set to become a trend of great relevance, and now is the time to make concerted efforts in improving the newspaper production workflow as well,” says Hiroshi Misawa, the paper’s managing director.

The Shinmai, as it’s known, plans to roll out the system in April for its cable TV news summary service, with an eye to speeding up news updates.

The summarizing AI joins a host of other automated news applications sometimes described as automated or augmented journalism. Heliograf, the Washington Post’s own news bot, produced about 300 briefs on the Rio Olympics of 2016, and has since covered U.S. elections and high school football games; it produced about 850 articles in its first year, according to Digiday. The Associated Press worked with AI firm Automated Insights to deploy software to cover earnings reports.

“Through automation, AP is providing customers with 12 times the corporate earnings stories as before (to over 3,700), including for a lot of very small companies that never received much attention,” AP global business editor Lisa Gibbs was quoted as saying in a 2017 report.

“With the freed-up time, AP journalists are able to engage with more user-generated content, develop multimedia reports, pursue investigative work and focus on more complex stories.”

Tim Hornyak

Tim Hornyak is a freelance journalist based in Tokyo. He is the author of Loving the Machine: The Art and Science of Japanese Robots. Follow Tim Hornyak on Twitter.

Our newsletters are read around the world by some of the smartest people in media. Subscribe here.

From this week

Platforms

Facebook’s paralysis and negligence in tackling hate speech keep coming up in conversations.

Reuters — which has two of its journalists in prison in Myanmar for reporting on the country’s genocide — put out a special report on Facebook’s hate speech problem in the country. Facebook doesn’t have an employee in this country. Speech moderation is outsourced to Accenture in Kuala Lumpur in a secretive project called “Honey Badger”. But it’s not clear how many Burmese speakers are on the job. People working on the project sign a one-year renewable contract, and agree to never divulge that Facebook is the client. This is what Reuters found out about the project.
Reuters

Facebook’s head of news partnerships Campbell Brown made some off-the-record comments to Australian media executives about traffic referrals that stirred the hornet’s nest.

The Australian, breaking professional protocol in publishing details of that session, quoted her as allegedly saying, “We are not interested in talking to you about your traffic and referrals any more. That is the old world and there is no going back”. Of course, this isn’t new to many publishers who’ve seen their referrals dwindle in the past year. But it’s another reminder to everyone: Facebook is in the business of Facebook. If you’re in publishing and you’re still counting on Facebook’s referral traffic to keep your traffic numbers up, you’re delusional.
Nieman Lab

Transformations

Design

How do you redesign The Wall Street Journal’s 126 newsletters?

1. You cull them by a third. 2. You nudge your readers to subscribe with a prompt. 3. You update market info — in real time. 4. You let people hit reply. People like that whole responding thing. 5. Test readers’ resistance to your paywall. 6. You test a new email platform that plays to your strengths. 7. You add whimsy. At the end of it all, a newsletter is a conversation, and it takes more than machine learning to keep that going. All aboard for whimsy, I say.
Nieman Lab

The product design process can be notoriously difficult.

This is mostly because it’s often seen as an artistic moment of genius broken down into deliverables to a client waiting for results in a process with unstructured feedback. This is also known as herding cats. But we tend to forget that most effective product design works to fix a business problem in a collaborative manner that establishes goals, relies on prototyping, user feedback, and testing. How this product designer explores and tests his own journey is a lesson in process and how we work with it.
UX Design

Notables

Thanks for subscribing!