Delhivery: Powering Operational Excellence for Third-Party Logistics

Delhivery launched in 2011 as a hyperlocal delivery service, focusing on last-mile services such as delivering goods like food and flowers within Gurgaon, located on the border of Delhi. However, it was only months into this venture that the company’s co-founders became interested in a much larger opportunity surrounding e-commerce. The industry was only just beginning to take off in India, but the co-founders could see the potential and by mid-2011, Delhivery secured its first e-commerce client. From that point on, Delhivery has been focused on solving the logistics challenges of e-commerce, including speed of delivery.

Speed was a major pain point in those early days. It wasn’t just about the buyer’s experience, as sellers also suffered. India is a cash economy and most sellers needed to wait for payments to be collected upon delivery, so if the delivery was delayed so were the profits.

The only way for Delhivery to solve this issue was to optimise the efficiency of its entire delivery network, which it achieved with the help of Elastic. The Elastic Stack enables Delhivery to search, analyse, and visualise millions of data points collected from its delivery network daily. With these insights, the company continues to increase the speed and efficiency of delivery, and shape the future of logistics.

In order to maintain a high level of operations, Delhivery needs to know everything that’s going on — all the time. Every 20 seconds, Delhivery captures data from the field, including the size, location, and status of all its shipments. Over time, this adds up to more than 50 data points per package which can be used to build intelligence and efficiency into the delivery network.

Initially, data was indexed and analysed in MongoDB. However, as Delhivery increased its daily deliveries from 500 to 9,000 in its second year, the growth in data was enormous. It became difficult and time-consuming to slice and dice the data and extract new insights to drive performance.

“We’re always experimenting with different tools and technologies, and began looking for a solution that would make it easier to aggregate and explore our data,” said Co-founder and CTO Kapil Bharati. “We saw some dashboards powered by Kibana, and thought why can’t we just put all of our data into Elastic? It soon became a very powerful tool for us.”

Delhivery’s first application — released in 2013 — using a pre 1.0 version of Elasticsearch was a series of custom dashboards allowing e-commerce sellers to track their own deliveries. These dashboards are still in use today, providing sellers with real-time geo-location data on all their packages and thereby reducing their reliance on Delhivery’s call centre.

The company now uses a range of dashboards internally to track packages and performance. Powered by Kibana, these dashboards allow Delhivery’s control tower team to quickly spot blockages or delays within the delivery network. The team can then visually explore the underlying data to identify the root cause.

“With Kibana, we can observe everything that’s happening in real-time, and if we see that shipments are getting stuck in a particular location, we can address the problem immediately and proactively alert our customers,” said Kapil.

Deeper analysis of the same data over time helps Delhivery identify opportunities to improve performance. This could involve optimising package routing or bringing on more team members to manage peak periods.

“One of the great things about Kibana is that you don't need to be a technical expert to use it, so our operations teams can create custom, complex dashboards to help them make better decisions,” said Kapil.

Delhivery initially managed its own clusters, but the task became more challenging as it scaled and faced a surge of activity during India’s festival months. To help manage these scaling and hosting challenges, Delhivery became an early user of the Elasticsearch Service (now on Elastic Cloud, but back then on Found.no). And since that point, Delhivery’s relationship with Elastic has expanded alongside its operations.

“Moving to a managed service gave us greater flexibility to scale and allowed us to focus on our core activities,” said Elasticsearch Architect and Senior Engineering Manager, Karan Agarwal. “We have looked at other options over the years, including Amazon’s Elasticsearch offering, but we have continued with the Elasticsearch Service because it is run by the experts and always offers us access to the latest features of the Elastic Stack.”

Delhivery is extracting more value from these features following a recent upgrade from version 5.x to 6.5, which was made seamless with the help of Elastic Support. The upgrade has also introduced dedicated node types into its clusters, enabling it to optimise memory and CPU as well as stabilise performance.

Each day, Delhivery ingests more than five million data points, including package geodata. Data flows through a Kafka Stream to Logstash and into an Elasticsearch cluster where it is used to power Kibana dashboards and custom applications.

Kibana remains instrumental in managing and monitoring daily operations, with security features from the Elastic Stack used to control which data can be accessed by each individual. This allows Delhivery to assure the security and privacy of data while opening Kibana up to teams across the company as well as exposing it to their end e-commerce customers.

Teams are able to visualise a wide range of performance metrics in Kibana, and use Elasticsearch to explore anomalies and trends. For example, if the control tower observes the delivery network’s performance drop to 97%, it can look at the performance of each facility and then keep filtering the data to identify the problem. It may also choose to monitor specific aspects of the network by using Elastic Stack APIs to create front-end widgets with embedded visualisations.

“The API interface and documentation supporting the Elastic Stack is very good, and makes it easy to experiment and quickly create new widgets and visualisations as different needs arise,” Karan. “For a company in this space that is innovating rapidly, it actually has the best documentation I’ve seen.”

With proactive support from Elastic, Delhivery has optimised the health and stability of its clusters and continues to take advantage of new features and capabilities of the Elastic Stack. It is exploring the use of Elastic APM for monitoring and alerting of transport and warehouse management systems, and micro-services as well as for extended stack trace visuals of requests and interoperability. Delhivery is also investigating the use of Canvas as a way to build pluggable dashboards for its facilities teams and therefore reduce reliance on custom built dashboards for various use cases. These dashboards would give operators and managers self-explanatory and actionable data showing real-time fluctuations in packages and other events occurring at each facility.

“We’re now running more experiments and exploring new features and products like machine learning which would give our control tower the ability to make predictions on the capacity and performance of our delivery network. For example, if there’s a spike in activity at one centre, we can analyse what may happen at other facilities down the line,” said Kapil.

The intelligence and efficiency gained through machine learning and potential use cases for Canvas and Elastic APM will help Delhivery bolster its expansion and capacity to innovate. Today, the Elastic Stack is helping Delhivery make better and faster decisions about everything from resourcing through to where it should build its next facility, but they have ambitious plans to further disrupt the logistics industry with an operating system and platform that could be leveraged by others.