For almost a week and right till the night before the five-day Big Billion Day sales began on 10 October, a team of Flipkart engineers had been simulating 15 times more traffic on their live systems by streaming data equivalent to three HD movies per second internally. Further, anticipating a 1.8 times increase in shipments delivered per day, Flipkart has also doubled its storage capacity.
Hidden AI hand that drives online mega shopping days
The paranoia and preparation were warranted. Flipkart, which has 50 data scientists and 1,600 engineers working across various projects, including the ongoing Big Billion Days sales, wanted to avoid the embarrassing website crashes four years ago. While Flipkart began the trend of offering sales over a period of several days during the festivities in 2014, the success of the event was marred by server crashes and delayed orders.
“The Big Billion Days of 2014 was a huge learning for us. And it kind of fast tracked our learning by three years. Just the day before Big Billion Days sale, we caught a 25 times increase in traffic. And, that year, we were caught unawares,” recalls Regunath B., principal architect at Flipkart.
After the 2014 sales, Flipkart built an entire new data centre and a private cloud, which could handle more traffic. “Before these big sales, we start by looking at the most efficient ways of handling workload. For that we move our systems to newer types of instances in a private cloud, which is most efficient for that kind of workloads.”
Using real-time analytics and insights, Flipkart predicts the traffic and consumption patterns to help them tune and optimize the systems. “We don’t treat all users equally. The experience a user, who is just window shopping, will be very different from users who intend to actually buy based on what they are doing on the site,” Regunath explains.
Specific products, which are in greater demand, are made available on multiple servers so that they can handle more loads. Flipkart has also deployed artificial intelligence (AI) solutions to detect and mitigate bots and resellers, and stop them in the transaction funnel, to give a fair chance to genuine buyers.
Amazon’s Great Indian Festival (10-15 October) and Paytm’s Maha Cashback sale (9-15 October) are some of the other online sales events that see a sudden spike in traffic. The two companies are also upgrading their infrastructure and switching to AI tools. “Based on past experience during festival sales, we have estimated projections for increase in traffic, as well as transaction. We have our own SQL databases, which automatically get scaled up when there is a traffic spike. Paytm cloud infrastructure has AI tools that give us a view of the traffic pattern, and the time when the traffic will increase. The system automatically estimates and provides the exact data,” explains Srinivas Mothey, vice president, marketing, Paytm.
Getting orders sorted out and delivered in fewer days is another nightmare that e-commerce companies face during the festival sales. There are more orders to process and more packages to sort out. Flipkart uses AI-powered address intelligence tools, which help find the precise location of customer addresses for deliveries.
Bengaluru-based supply chain intelligence company Locus, which has Flipkart-owned Myntra and Big Basket among its clients, uses deep learning and proprietary algorithms to provide advanced logistics optimization solutions. Its automated shipment sorting and route optimization tools can read addresses and automatically print the route number, and the name of the person who will deliver the goods. This reduces the man hours spent on the first level of sorting. It also makes it easier to determine how many orders are coming from the same place, and the orders that could be clubbed on the same route, besides how many vehicles need to be deployed on any route.
According to Shantanu Bhattacharyya, data scientist, Locus, while the focus is on logistics, e-commerce companies also focus on issues such as inventory optimisation and demand prediction.