Opinion | A Report That Fails: Assessing the Shortcomings of V-Dem’s Evaluation

We are still a member of the Commonwealth. The head of the Commonwealth is Charles III. We are a member of the Commonwealth by virtue of the 1949 London Declaration. “The Government of India have, however, declared and affirmed India’s desire to continue her full membership of the Commonwealth of Nations and her acceptance of The King as the symbol of the free association of its independent member nations and as such the Head of the Commonwealth.” The preceding sentence states, “The Government of India have informed the other Governments of the Commonwealth of the intention of the Indian people that under the new constitution which is about to be adopted India shall become a sovereign independent Republic.”

In 1949, it was already known that India would be a sovereign independent republic. Nevertheless, we wished to be a member of the Commonwealth, accepting the King as a “symbol”. It is high time we asked what we gain from being a member of the Commonwealth. India became independent in 1947 and a republic in 1950. Our head of state is the President and our head of government is the Prime Minister. Many countries maintain this difference between the two. Note that in India’s case, both positions are elected. This isn’t true of a constitutional monarchy, especially if that constitutional monarchy is hereditary.

Democratic roots in India are thousands of years old. Much before the Greeks, there were sabhas and samitis. Nonetheless, we are often lectured on democracy. One such recent one is V-Dem Institute’s Democracy Report for 2024. The V-Dem Institute is based in Sweden. Sweden has a king and the heir is a princess. Note the obvious. Monarchies don’t fund themselves. They are funded by the exchequer. We are told by the V-Dem Institute that India is an electoral autocracy. Critics of the government have used this to pillory the Narendra Modi government. However, this is what V-Dem actually says in the relevant paragraph. “India’s process of autocratisation begins in earnest from 2008 and characteristically proceeded in the incremental, slow-moving fashion of the ‘third wave’.”

Strange? What happened in India in 2008? V-Dem actually has six separate indices. The one cited most often is the Liberal Democracy Index. Under this, India is ranked 104^th out of 179 countries. Honduras is ranked 83^rd. That suggests the state of liberal democracy is significantly better in Honduras than in India. Yes, Honduras had elections towards the end of 2021. But there was a history before that and suggesting that democracy is significantly better in Honduras than in India doesn’t satisfy what is popularly called a smell test. Unlike immediate neighbours in South Asia, barring the emergency aberration between 1975 and 1977, post-independence India has always been a democracy. Yet, according to V-Dem, India is now an electoral autocracy, as it was in 1975 and 1976. Doesn’t pass the smell test again. In fact, the report stinks to the high heavens.

How does one rank? How does one pin down and define democracy? Take for instance India’s Constitution. As everyone knows, under the Constitution, the Supreme Court has the power to negate any legislation not in conformity with the basic structure of the Constitution and it has done so, on many occasions. Should this be a desirable ingredient in the “checks and balances” framework of any democracy? Most people will answer in the affirmative and will suggest that this should be an indicator in cross-country rankings of democracy. But since the Supreme Court has no such powers in Sweden, V-Dem doesn’t think this has anything to do with democracy at all.

Or take the monarchy I mentioned earlier. Most people who believe in modern democracies will argue kings and queens belong in a deck of playing cards, not anywhere else. I mention these examples to illustrate an obvious point. When trying to rank something as intangible as democracy, especially when attempting a cross-country exercise, there is nothing obvious about the indicators and variables used. Each indicator chosen, and each indicator rejected, reflects a value judgement. It becomes worse when no objective data are available for the indicators used. Instead, questions are asked of experts. “V-Dem typically gathers data from five experts per country-year observation.”

“The logic of the V-Dem measurement model is that an unobserved concept exists (e.g. a certain level of academic freedom and freedom of cultural expression) but we only see imperfect manifestations of this concept in the form of the ordinal categories which experts use to code their judgements. Our model converts these manifest items (expert ratings) to a single continuous latent scale and thereby estimates values of the concept.” Sounds very fancy. Except this reminds me of an anecdote, concerning Catherine the Great of Russia, the philosopher Denis Diderot and the mathematician Leonhard Euler.

When Diderot visited St. Petersburg, he was an atheist and this upset Catherine. She sought help from Euler. Euler went to Diderot and said, “(a+bn)/n = x. Therefore, God exists.” Diderot knew little mathematics and fled. Fancy statistics won’t get us far if there are serious problems with sample sizes and sampling design. Read the critical paper, ‘Measuring Democratic Backsliding’, authored by Andrew Little and Anne Meng. Other than smell tests, V-Dem’s report won’t pass any peer-review test.

The author is the chairman of the Prime Minister’s Economic Council and a well-known Sanskrit scholar. Views expressed in the above piece are personal and solely that of the author. They do not necessarily reflect News18’s views.