Online marketplace acts as a source for price data in Indian economy

The e-commerce industry in India is set to grow at a compound annual growth rate (CAGR) of over 30% in the next 10 years.

Reports suggest that in 2018 the online shopping industry grew 60% to about $28.5 billion. The corresponding figure in the next decade is estimated to be about $200 billion. With such rapid expansion, the online market plays itself out as a repository of information related to some important parameters of the Indian economy.

For instance, the prices of goods available online can potentially provide a valuable signal of the inflationary pressures in the economy. This is particularly true for food prices for two reasons. First, the prices of food items are widely posted online on numerous websites. Two, food and beverages as a category constitutes a large share of the Consumer Price Index (CPI), namely 45.86%. Consequently, any inflationary pressure in this category has a large impact on the overall inflation rate.

The advantages of using price data from the online marketplaces are many. The data collection process is cheaper, quicker and more efficient. Besides, the relevant information can be obtained much more frequently, allowing policymakers to have almost a real-time peep into some of the vital statistics of the economy. Further, advanced computational techniques exist today in which bots, or sophisticated programmes, can crawl the web to identify product categories and the corresponding prices from publicly available online data. This makes the data collection process free of human error.

However, the credibility of the online price information, to a large extent, depends on how well it correlates with the official inflation data, on which economic policy making in India is based. Thus, whether or not the online price data tracks the official CPI in India is an important empirical question.

In a recently published paper in the Economic and Political Weekly, we compared the official food and beverages CPI with an online price index created with the price data obtained from Bigbasket, a leading online retailer, for the Bangalore metropolitan area only.

The online price index was computed using the standard weights and methodologies that are used to compute the CPI. For each of the categories in the official inflation basket, one or more corresponding online products were chosen. For key items such as rice and atta, more than one brand were chosen in order to avoid picking up brand-specific or variety-specific price fluctuations.

The final online data set contained daily price information on 120 products across 25 product categories, 12 subgroups and two groups. The key question we asked was whether the online price index successfully tracked the official food and beverages CPI.

Our findings suggest that at an aggregate level there is indeed a close co-movement between the two indices. The high correlation between the two series of numbers suggests that the online index tracks the food and beverages CPI rather well.

Further, we performed a sub-group level analysis to show that for a variety of sub-groups such as cereals, fruits, meat and fish, milk products, pulses and vegetables, with some caveats, the online index successfully tracks the official CPI.

The one dip in the online index between January 2016 and April 2016, which is not reflected in the CPI, is driven by the seasonal dip in the prices for the vegetables category.

It is well known that the prices of food and beverages, particularly for subcategories such as vegetables, are sometimes quite volatile.

How do the online prices fare if such volatilities are excluded? We use an alternative methodology, called weighted median approach, to define a weighted median price index. The weighted median price index diminishes the volatility without sacrificing the substantive aspects of the price movement. It does so by focusing on the price movement in the middle of the distribution, ordered in an ascending manner. Figure 2 plots the weighted median online price index with the weighted median official CPI for food and beverages.

A comparison of the weighted median CPI in figure 2 and the official CPI in figure 1 suggests that the former does not dip in the February – March period in 2016 to the extent the latter does; however, it does capture the essential trends in inflation.

A comparison of weighted median CPI and weighted median online index in figure 2 reveals that there is a close co-movement of the two indices, sometimes even closer than that we see in figure 1. The reason why the weighted median method does better in tracking is because it successfully moderates the volatile components such as prices of vegetables in both the indices.

Despite the fact that online prices do well in terms of tracking the CPI in India, indices based on online prices may not have important desirable characteristics. For instance, it may well be the case that all the product categories and sub-groups used to compute the official CPI may not be represented in the online market place.

Also, in terms of its geographical reach, online market places are still fairly limited, with a vast swathes of regions either outside the ambit of the internet or its penetration is not mature enough to merit a well-functioning online market.

Thus, it is important for researchers to effectively collaborate with the e-commerce industry to scale up and replicate these findings in a broader context. Such validation exercises are necessary before the government can use online information to formulate efficient responses to important policy matters in India.

A statistics professor, teaching basic concepts in statistics, had once been asked by a student if the content she was teaching was relevant for big data. Bemused yet quirky, her response was, the concepts were relevant for all data, big or small.

Big data and machine learning are fashionable in today’s world alright, however, they have enormous potential in terms of contributing to policy making, particularly in the direction discussed above.

Banerjee and Subramanian are at IIM Bangalore, Singhal is at NeenOpal.