The answer to your search may depend on where you live — Harvard Gazette

A Chinese-language Google user types in “Buddhism” and launches a search. A French-language user searches the same term, as does an English-language user. Will all three get the same results? New Harvard research says not necessarily, finding online search responses can vary significantly and even conflict depending on the topic and language of the query.

The variation is a result of a hidden “language bias” embedded in the search algorithms of Google, ChatGPT, YouTube, and Wikipedia, says Queenie Luo, who studies artificial intelligence ethics and early Chinese history and is co-author of a paper with her Ph.D. adviser, Michael Puett, the Walter C. Klein Professor of Chinese History and Anthropology, and Michael D. Smith, former dean of the Faculty of Arts & Sciences who now teaches at the Harvard Paulson School of Engineering and Applied Sciences (SEAS). 

That bias, they contend, distorts users’ understanding of search topics by limiting their exposure to a full range of information and viewpoints and raises questions about larger implications for relations between nations and peoples.

The Gazette spoke with Luo, who earned a master’s degree from SEAS in data science in 2023, about language bias in searches and the potential social and political harms that can arise from this hidden filter in major search platforms. The interview has been edited for clarity and length. 

You tested a number of search terms, but the paper focuses on two that are extremely complex and abstract, Buddhism and liberalism. What did you find on Google? 

Buddhism is a global religion that has developed distinct cultural traditions in different language communities worldwide. Chinese Buddhism is very different than Japanese Buddhism, Thai Buddhism, and Vietnamese Buddhism. Essentially, Western Buddhism has evolved into another branch of Buddhism over the past two centuries.

Our research found that when searching for Buddhism-related phrases on Google using different languages, the top-ranked websites tend to reflect the dominant Buddhist tradition of the query’s language community. 

The general assumption among internet users is that they can access global information through search engines, with Google often perceived as providing objective, unfiltered results. However, our research found that as Google and most online platforms use search language as a significant filter, different language users end up reading highly distinct information online. 

Such problems aren’t just limited to Buddhism-related queries, but extend to a wide range of topics, such as liberalism and international trade policy. For example, if you search liberalism using English on Google, you tend to get very positive views about liberalism and nearly no references to neoliberalism. The concepts of free market, human rights, and equality tend to be strongly emphasized in English-language search results.

However, if you switch your search language to Chinese, Google’s top-ranked search results on liberalism tend to benegative and frequently connect it to neoliberalism. Similarly, when you ask Google, “What constitutes good economic policy” using a European language like Italian or French, the top-ranked websites tend to emphasize aspects such as protective market economy, but the top-ranked websites would instead focus on “free market economy” or “limited government intervention” when you search using English. 

These ideas aren’t mutually exclusive but can be contradictory based on context.  

What about other platforms people use to search, like ChatGPT?

Things changed with ChatGPT. As ChatGPT is predominantly trained on English-language data, it always presents the Anglo-American perspectives by default. The version integrated with Bing behaves similarly to Google in that it searches into websites in the query’s language and summarizes the main content for you. 

Wikipedia and YouTube are two major platforms featured prominently on Google. We found that language bias also exists on these two platforms. For example, if you search on Wikipedia using English about Buddhist meditation, the English article gives you an overview of world traditions of Buddhist meditation. 

But if you switch to the French article, it includes a section of “neuroscience and Buddhism” that does not exist in articles in other languages. This difference could be partly due to the influence of the French monk Matthieu Ricard, who has participated in a series of neuroscience experiments to demonstrate the benefits of meditation and attracted a lot of attention in the French-speaking community.

Language bias becomes more dramatic on YouTube because YouTube videos tend to provide highly concentrated and focused information on one narrow aspect. For example, when searched using Japanese, the top-ranked videos include Buddhist music performed by a Japanese monk, whereas the English videos teach users about the wisdom of the Buddha. The impressions that different linguistic users get from watching these videos are very different.

So users are steered to different information and only shown the prevailing views of the search language, and not getting a global picture of the query topic? 

Exactly. Such filtering effect can be neutral or useful for utility-based questions, such as visa requirements. However, it can pose a significant threat to our society on sensitive and complex topics like liberalism and international trade policy. Unlike math or computer science, which have definitive answers, complex topics demand diversity and mutual debate. 

We use the fable of the blind men and the elephant to describe this phenomenon, that each language community is like a blind person touching a small portion of the elephant while believing they have seen the whole. 

As Google’s ranking algorithm is designed to capture the “common case” and majority’s interests while also filtered by language, English-language users typically get positive views about liberalism while Chinese-language users get criticisms of it through Google.

Over time, such technology serves as a facilitator for social division. Mutual engagement is no longer possible because each language community sees different facts. 

What factors are driving this?  

There are many factors that contribute to the existing pattern. First, using the language filter is an algorithmic choice. Without a good translation system, users cannot read information written in other languages, so the language filter serves a practical function. However, now with machine translation, the language filter might not be necessary. 

Second, language is intrinsically tied to culture, history, and group identity, so any concept that is expressed through a certain linguistic system is inseparable from its cultural roots. On concepts surrounding Buddhism and liberalism, different language corpora do exhibit very different opinions and perspectives. 

And thirdly, the extent of discrepancy across different language searches varies depending on the topic you’re searching. For example, on topics like “Jacobean matrix,” we did not observe precipitable differences across languages. “Jacobean matrix” is a relatively new and very technical term, and has a well-defined mathematical definition, so when you’re searching this term across different languages, you don’t see much difference. 

Scientific, mathematical, and technical terms, especially recent technical terms, as they have well-defined definitions, tend to have consistent interpretations. However, for terms that have a longer history, like Newton’s First Law, the top-ranked websites often include a lot of historical narratives surrounding these topics

Why is language bias showing up in AI-powered searches?