The mystery behind the Worldometer data aggregation site

MADRID – On April 28, Spanish Prime Minister Pedro Sánchez was left alone on the stage of a bright but empty meeting room. While a CNN reporter asked a question via video link, the prime minister seemed deeply focused, scribbling notes and pausing to watch the monitor once. As he launched his answer, he looked directly into the camera to brag about Spain’s Covid-19 test volume.

“We are one of the countries with the largest number of tests carried out,” said Sánchez.

Initially, the prime minister cited data from a recent ranking of the Organization for Economic Cooperation and Development (OECD) which had placed Spain eighth in the Covid-19 rankings among its members.

“Today,” he added, “we discovered another study, from Johns Hopkins University, which […] places us in fifth place in the world for the total tests carried out. “

There were only two problems: OECD data had been incorrect. And while some sources had ranked Spain fifth in the total test volume, Johns Hopkins was not one of them; the study cited by Sánchez does not exist.

However, two weeks later, the Spanish government claims the substance of his prime minister’s complaint. Instead of quoting Johns Hopkins, Spanish officials are now aiming to test the rankings from a data aggregation website called Worldometer – one of the sources behind the coronavirus dashboard widely cited by the university – and asking questions about why some respected governments and institutions have chosen to trust a source of which little is known.

Before the pandemic, Worldometer was best known for its “counters”, which provided real-time estimates of numbers such as the world population or the number of cars produced this year. His website indicates that the revenue comes from the advertising and licenses of his counters. The Covid-19 crisis has undoubtedly increased the popularity of the website. It is one of Google’s best search results for coronavirus statistics. In the past six months, Worldometer pages have been shared about 2.5 million times, compared to just 65 shares in the first six months of 2019, according to statistics provided by BuzzSumo, a company that tracks social media efforts and provides insights into the content.

The website claims to be “run by an international team of developers, researchers and volunteers” and “published by a small, independent US-based digital media company.”

But public records show little evidence of a company employing a multilingual team of analysts and researchers. It is unclear whether the company paid staff who check its data for accuracy or whether it relies solely on automation and crowdsourcing. The site has at least one job ad, since October, looking for a volunteer web developer.

Once known as Worldometers, the website was originally created in 2004 by Andrey Alimetov, then a recent 20-year-old immigrant from Russia who had just gotten his first IT job in New York.

“It’s a very simple website, there’s nothing crazy about it,” he recently told CNN.

Within about a year, Alimetov said, the site received 20,000 or 30,000 visits every day, but it cost him too much money in web hosting rates.

“There wasn’t an immediate quick way to cash in,” he said, so he listed the site on eBay and sold it for $ 2,000 in 2005 or 2006.

When Reddit’s homepage featured its old site in 2013, Alimetov sent an email to the buyer, a man named Dario, to congratulate him.

In his answer, Dario said he purchased the site to drive traffic to his other websites.

As those companies “started to decline, I decided to invest in Worldometers, bringing resources and people until it finally took a course of its own,” wrote Dario.

Worldometer no longer carries its trailing “s” except in its URL. Other than that, not much has changed.

Today, the Worldometer website it is owned by a company called Dadax LLC.

Worldometer and Dadax representatives did not respond to CNN’s requests for interviews, but state business documents show that Dadax was first formed in Delaware in 2002. The list lists a post office box as the company’s address. From 2003 to 2015, corporate documents in Connecticut and New Jersey listed Dadax’s president as Dario Pasqualino. The addresses on the documents linked the company and Pasqualino to the houses in Princeton, New Jersey and Greenwich, Connecticut. The company is still actively registered in Delaware and has been in business since 2010.

The company shares the name Dadax with a Shanghai-based software company. In March, both companies released statements that denied a connection. Chinese Dadax said he was released his statement after receiving “many calls and emails” on the statistics site. Worldometer, in a tweet, he said he never had “any kind of affiliation with any entity based in China”.

The IDs in the source code of Worldometer and U.S. Dadax websites link them to at least two dozen other websites that appear to share ownership. Some seem to have passed away. Others, like usalivestats.com, italiaora.org and stopthehunger.com, share the same premise: statistics counters in real time. Most sites have a rudimentary aesthetic, reminiscent of a network from the 90s or early 2000s. Some seem fairly random. An Italian site displays Christmas poems and gift suggestions, such as a bonsai plant (for her) or a plot of land on the moon (for him). Another site is dedicated to Sicilian puppet shows.

A person with Pasqualino’s name and birthday is also registered as a sole proprietorship in Italy. This activity manages and sells “advertising spaces”, according to an Italian registration document filed last year. His address leads to an orderly three-story condominium on a tree-lined street in an elegant neighborhood of Bologna.

CNN was unable to reach Pasqualino through the contact information listed on Worldometer and in public records.

According to the Worldometer website, its Covid-19 data comes from a multilingual team that “monitors the live flows of press briefings all day long” and through crowdsourcing.

Visitors can report new Covid-19 numbers and data sources to the website, without needing a name or email address. A “team of analysts and researchers” validates the data, says the website. At first it may seem like the Wikipedia of the data world, but some Wikipedia editors have decided to avoid Worldometer as a Covid-19 data source.

“Several updates lack a source, do not match their cited source or contain errors,” wrote an editor with the username MarioGom a discussion page for Wikipedia editors working on Covid-19 content last month. “Some errors are small and temporary, but some are relatively large and never corrected.”

The publisher, whose real name is Mario Gómez, told CNN in an email: “Instead of trying to use consistent criteria, [Worldometer] seems to be going for the highest figure. They have a system for users to report higher figures, but so far I have not been able to use it to report that some figures are incorrect and should be lower. “

Edouard Mathieu, the data manager for Our world of data (OWID), an independent statistics website based at the University of Oxford, has seen a similar trend.

“Their main target seems to have the latest issue wherever it comes from, whether it is reliable or not, whether it is of good origin or not,” he said. “We think people should be cautious, especially the media, policy makers and decision makers. These data are not as accurate as they think.”

Virginia Pitzer, an epidemiologist at Yale University focused on modeling the spread of Covid-19 in the United States, said she had never heard of Worldometer. CNN asked you to evaluate the reliability of the website.

“I think the Worldometer site is legitimate,” he wrote via email, explaining that many of its sources appear to be credible government websites. But he also found flaws, inconsistencies and an apparent lack of specialist care. “Data interpretation is lacking,” he wrote, explaining that he found the data on active cases “particularly problematic” because data on recoveries are not constantly reported.

Pitzer also found some detailed explanations for problems or discrepancies in reporting the data. For Spain, it is a single sentence. For many other countries, there are no explanations at all.

He also found mistakes. In Spanish data, for example, Worldometer reports over 18,000 recoveries on April 24. The Spanish government reported 3,105 recoveries that day.

When Spanish Prime Minister Pedro Sánchez boasted of Spain’s high rankings, he didn’t pull his numbers out of thin air. On April 27, the OECD erroneously ranked Spain eighth in per capita tests. Initially, the OECD had used OWID data for compilation its statistics. But he provided the Spanish numbers independently because the OWID data was incomplete. Mixed sourcing distorted Spain’s position in the ranking because it counted a wider category of tests than the numbers in other countries. The organization corrected the next day, two hours before Sánchez’s press conference, taking Spain to 17th place.

In his statement, the OECD said that “we regret the confusion created on a sensitive issue by any debate on methodological issues” and stressed that increasing the availability of tests in general is more important than knowing where a particular country is located.

Sánchez’s subsequent reference to a study by Johns Hopkins, in which he claimed that Spain ranked fifth for tests worldwide, appears to have been a case of confusing attribution by the prime minister. JHU has not published international test data. Jill Rosen, a school spokeswoman, told CNN that the university was unable to identify a report that matched Sánchez’s description.

At a press conference on May 9, Sánchez evaded a CNN question prompting him on the existence of the JHU study and listed government numbers on test totals instead. In comments made to a Spanish journalist the next day, Health Minister Salvador Illa continued to insist that test data had been released by the JHU, although he indicated Worldometer as the underlying source. Given that Johns Hopkins gets his data from Worldometer, he said, it’s just as good.

“These are the data provided by John Hopkins University […] taken from a key source of information, the Worldometer website, ”said Illa. “You can check it.”

It is true that on April 28, Worldometer data ranked Spain fifth in total test volume. At the time, OWID data was also in fifth place in Spain, but when more countries started recording larger test volumes, it became clear how flawed Worldometer data was. Its figure in Spain includes both polymerase chain reaction (PCR) tests, which show whether patients are currently infected, and antibody tests, which indicate whether patients have ever been infected. For most countries besides Spain, Worldometer data appear to only count PCR tests.

Since so few countries report antibody test data and to ensure an apple-apple comparison, OWID says it only tracks PCR tests. With this measure, starting from May 17, Spain is in sixth place, behind the United States, Russia, Germany, Italy and India. Worldometer in fourth place in Spain.

However, relying on the ranking based on the raw number of tests performed is still misleading because it does not take into account population differences between countries.

OWID data manager Edouard Mathieu says that a much fairer way to compare test data is to take population size into account. As of May 10, OWID has put Spain in 19th place in tests for 1,000 people. Worldometer placed Spain 15th with a similar measure.

Roberto Rodríguez Aramayo, research professor at the Institute of Philosophy of the Spanish National Council for Research (CSIC) and former president of a Spanish ethical association, said that Spain is reporting data of the most and least reliable types of tests .

“Unfortunately, there seems to be sure [political] interests in the readings that are provided of this data when it is shown, “he said.

Johns Hopkins has not published international data on Covid-19 tests, but lists Worldometer as one of several sources for the widely cited coronavirus dashboard.

The university declined to say for which specific data points it relies on Worldometer, but problems with counter data caused at least one noticeable error.

On 8 April, the JHU’s global count of Covid-19 confirmed cases briefly exceeded 1.5 million before falling by over 30,000. Later Johns Hopkins published an explanation for the accident on his GitHub page. At the time, JHU told CNN that the error seemed to come from a double counting of French nursing home cases. But French officials told CNN that there have been no revisions, not even on nursing home data. Johns Hopkins’ data seemed to come directly from Worldometer. The website did not list any sources for its figure.

A Wikipedia editor, James Heilman, an assistant professor of emergency medicine at the University of British Columbia, said that Wikipedia volunteers have noticed persistent errors with Worldometer, but also with “a more reliable name with a longer history. accuracy, “referring to Johns Hopkins. “Hopefully they’ll double check the numbers too.”

In an article published in FebruaryJHU said it began manually monitoring Covid-19 data for its dashboard in January. When this became unsustainable, the university began collecting data from primary sources and aggregation websites. Laura Gardner, the associate engineering professor who runs the university’s Covid-19 dashboard, told CNN to enter a declaration that the university uses a “two-step anomaly detection system” to detect potential data problems. Moderate changes are automatically added to the dashboard but marked so that staff can review them in real time. Changes beyond a certain threshold require “a human being to manually check and approve the values ​​before publishing them in the dashboard,” Gardner said.

The university’s dependence on Worldometer surprised some academics.

Phil Beaver, a data scientist at the University of Denver, looked speechless when asked what he thought of JHU by citing Worldometer.

“I’m not sure, that’s an excellent question, I had the impression that Worldometer was trusting [Johns] Hopkins, “he told CNN after a long hiatus.

Mathieu also seemed surprised.

“I think JHU has come under a lot of pressure to update their numbers,” he said. “Because of this pressure, they were forced or encouraged to get data from places they shouldn’t have been, but in general I would expect JHU to be a fairly reliable source.”

In the university’s response to CNN, Gardner said that Worldometer was one of the “dozens” of sources and that “before incorporating any new sources, we validate their data by comparing them to other references.”

“We try not to use a single source for any of our data,” added Gardner. “We use public health agency reports and aggregation sources to cross-validate numbers.”

The Spanish government and Johns Hopkins are not alone in citing Worldometer. The website was mentioned by Financial Times, The New York Times, The Washington Post, Fox News is CNN.

The British government cited Worldometer data on Covid-19’s deaths during its daily press conferences for much of April, before moving on to Johns Hopkins data.

“Both Worldometers and John Hopkins have provided complete and respected data. As the situation developed, we moved from Worldometers to John Hopkins, as John Hopkins relies more on official sources, “reads a statement from a spokesman for the British government.

In Spain, the apparent inattention of Johns Hopkins of Sánchez has become a major controversy. On Wednesday, in parliament, the deputy of the center-right Popular Party Cayetana Alvarez de Toledo called the government the party “of lies to CNN and the Spanish people”.

On May 10, a spokesman for the Spanish embassy in London complained to CNN about his coverage of the matter.

“In April, Mr. Sánchez mentioned Johns Hopkins University’s statistical data analysis based on data published by Worldometer,” the spokeswoman wrote in an email sent to the diplomatic editor of the network immediately after 4:00 on morning.

“Although Mr. Sánchez did not mention Worldometer as a primary source in his observations, [CNN] he might have known that most of the comparisons and analyzes on Covid-19 in the world use [Worldometer’s] tables “.

In observations to CNN, a spokesman for the prime minister’s office acknowledged that Worldometer counts PCR tests and antibodies together and rejected critics’ request to adjust test numbers for the population, calling it a “trap that the OECD and the Spanish press […] fell into “and arguing that Spain should not be compared to small countries like Malta, Luxembourg or Bahrain.

It is unclear, however, why the Spanish government continues to insist that test data published by Worldometer have been disclosed by Johns Hopkins University.

His refusal to acknowledge his attribution error comes just a month after Spanish justice minister Juan Carlos Campo said the government was considering changes to the law, trying to crack down on those reports of disinformation.

“I think it’s more than justified – with the calm, peace of mind needed for any legal changes – that we review what our legal tools are to stop those who contaminate public opinion in a serious and unjustified way,” Campo said.

At the time, Human Rights Watch executive director Kenneth Roth told CNN in an email: “If the justice minister suggested penalizing speeches that contaminate public opinion, it would be very dangerous.”

“Turning the government into a thurible would undermine that public responsibility just when it is most needed,” warned Roth.

In her letter to CNN, the embassy spokeswoman stressed that Spain was – and still is – the fifth in the world in Covid-19 tests, attaching screens of Worldometer tables as evidence.

“Figures speak louder than words,” he wrote. “And unwilling to recognize the truth of reality […] it is very disturbing, to say the least. “