A Glimpse into Foreign-Language News

Although the language of global communication is (still) English, there's lots of interesting content in other languages too. What's a curious English speaker to do? For example: A Bulgarian news site may only publish in Bulgarian because the target audience is in Bulgaria. Or: A website with international readers, like Al Jazeera, publishes in both Arabic, the region's native language, and English. But the English articles may not be exact translations, and native-language articles aimed at a domestic audience may not be translated. Wouldn't it be interesting to get at least a glimpse into what the domestic audience is reading?

One option for reading articles from foreign-language websites already exists: Watching America. It publishes articles about the US, found on non-US news sites, and translated to English by volunteers.

I had something else in mind: An overview of any articles on foreign-language sites, whether about the US or not, thus including domestic news as well. My demo, with the (possibly difficult-to-type) name of What's New(s)?, is my first attempt. It retrieves the titles and summaries of news articles from selected foreign-language news sites, and translates the content to English using AI. The articles are updated several times a week.

How It Works

The website has two functions: fetching, translating and saving articles on a regular basis; and displaying the articles on the What's New(s)? home page. These two functions work independently of each other.

Fetch, Translate and Save the Content

The first function consists of three steps, carried out several times a week:

1. Fetch the foreign language content. Many sites offer an RSS feed, a file that contains summaries and URLs for recently-published content. This file contains the information we want.

2. Translate the title and summary to English. This is done via the DeepL API to translate texts instantaneously.

3. Store the results in the What's New(s)? database. We don't want to fetch, translate and save content every time someone visits the site, so we save all the info in a database, which then gets displayed for each visitor.

Display the Articles

Once the original and translated content has been stored in the database, surfing over to the What's New(s)? home page will display the English translations.

By default, only the English translations of the article titles and summaries are shown. Click on the show button located at the right end of the horizontal orange line to show the original foreign-language texts if you're curious. Click on the go button at the bottom of each summary to take you to the original foreign-language article.

(Click on a thumbnail to enlarge.)

The Demo

Here's the demo, with a few hand-picked foreign news sites: whatsnews.jbrcodes.com. (Full disclosure: non-Latin alphabets fascinate me.)

Notes

• Relevance of Articles in the RSS File: The articles returned in the RSS file are not sorted by importance. Most likely the first article in the RSS file is the article most recently published on the foreign-language website. This is a disadvantage if you're interested in the top news stories, but you may be pleasantly surprised (as I was) by some of the low-importance articles returned.

• AI-based Language Translation: Computer translation works great in some situations, but it isn't perfect, and it possibly never will be. For the purposes of this demo, AI translation works fine. But human translation is probably still the better option for texts requiring more nuance.

Technologies Used

As with all my demos, What's New(s)? is built on the proverbial shoulders of existing powerful tools:

Flask

A Python-based web server framework.

feedparser

A Python module/extension to download and analyze RSS files.

DeepL

An API to translate texts using AI.

Peewee

A database ORM used to manage the original and translated texts in a SQLite database.