Observability
Travel and Transportation

Delhivery: Powering Operational Excellence for Third-Party Logistics

AT A GLANCE

  • 1
    million packages delivered per day in peak season
  • 50+
    data points collected per package
  • 4
    GB package data ingested each day

Decisions made in minutes

With the ability to visualise and explore data in real-time in Kibana, Delhivery can make rapid operational decisions and keep moving the needle on service and efficiency.

Network productivity at 90%

Delhivery can quickly spot and alleviate blockages within its delivery network, maintaining high levels of performance and minimising delays down the line.

Flexibility to scale and experiment

The Elasticsearch Service on Elastic Cloud enables Delhivery to seamlessly scale their deployment while easily accessing all the latest features of the Elastic Stack. This allows Delhivery to remain focused on expanding its logistics business rather than managing IT infrastructure.

Company Overview

Founded in 2011, Delhivery has evolved rapidly from its initial focus on last mile delivery to become India’s leading supply chain services provider. With a vast network encompassing 19 automated sort centres, thousands of delivery centres, 14,000 vehicles, and 21,000 team members, the company delivers an average of one million packages each day during peak season.

What makes Delhivery stand out is how it's optimising the speed of this network, and disrupting India’s logistics industry. With insights powered by the Elastic Stack, Delhivery provides unparalleled cost efficiency and increasingly fast service to more than 100,000 customers, including small and medium-sized businesses, large enterprises, and leading e-commerce platforms. These insights allow for faster decision making, which is vital to Delhivery’s success.

Elastic gives us the ability to make decisions in thirty minutes or less, and that’s everything. It impacts on cost, revenue, service, and everything else that matters to our stakeholders.

– Kapil Bharati, Co-Founder & CTO | Delhivery

Delhivery’s Journey with Elastic

Powering Operational Excellence for Third-Party Logistics

Delhivery launched in 2011 as a hyperlocal delivery service, focusing on last-mile services such as delivering goods like food and flowers within Gurgaon, located on the border of Delhi. However, it was only months into this venture that the company’s co-founders became interested in a much larger opportunity surrounding e-commerce. The industry was only just beginning to take off in India, but the co-founders could see the potential and by mid-2011, Delhivery secured its first e-commerce client. From that point on, Delhivery has been focused on solving the logistics challenges of e-commerce, including speed of delivery.

Speed was a major pain point in those early days. It wasn’t just about the buyer’s experience, as sellers also suffered. India is a cash economy and most sellers needed to wait for payments to be collected upon delivery, so if the delivery was delayed so were the profits.

The only way for Delhivery to solve this issue was to optimise the efficiency of its entire delivery network, which it achieved with the help of Elastic. The Elastic Stack enables Delhivery to search, analyse, and visualise millions of data points collected from its delivery network daily. With these insights, the company continues to increase the speed and efficiency of delivery, and shape the future of logistics.

Explosive growth in business and data

In order to maintain a high level of operations, Delhivery needs to know everything that’s going on — all the time. Every 20 seconds, Delhivery captures data from the field, including the size, location, and status of all its shipments. Over time, this adds up to more than 50 data points per package which can be used to build intelligence and efficiency into the delivery network.

Initially, data was indexed and analysed in MongoDB. However, as Delhivery increased its daily deliveries from 500 to 9,000 in its second year, the growth in data was enormous. It became difficult and time-consuming to slice and dice the data and extract new insights to drive performance.

“We’re always experimenting with different tools and technologies, and began looking for a solution that would make it easier to aggregate and explore our data,” said Co-founder and CTO Kapil Bharati. “We saw some dashboards powered by Kibana, and thought why can’t we just put all of our data into Elastic? It soon became a very powerful tool for us.”

Driving efficiency and speed

Delhivery’s first application — released in 2013 — using a pre 1.0 version of Elasticsearch was a series of custom dashboards allowing e-commerce sellers to track their own deliveries. These dashboards are still in use today, providing sellers with real-time geo-location data on all their packages and thereby reducing their reliance on Delhivery’s call centre.

The company now uses a range of dashboards internally to track packages and performance. Powered by Kibana, these dashboards allow Delhivery’s control tower team to quickly spot blockages or delays within the delivery network. The team can then visually explore the underlying data to identify the root cause.

“With Kibana, we can observe everything that’s happening in real-time, and if we see that shipments are getting stuck in a particular location, we can address the problem immediately and proactively alert our customers,” said Kapil.

Deeper analysis of the same data over time helps Delhivery identify opportunities to improve performance. This could involve optimising package routing or bringing on more team members to manage peak periods.

“One of the great things about Kibana is that you don't need to be a technical expert to use it, so our operations teams can create custom, complex dashboards to help them make better decisions,” said Kapil.

Increasing intelligence and scale

Delhivery initially managed its own clusters, but the task became more challenging as it scaled and faced a surge of activity during India’s festival months. To help manage these scaling and hosting challenges, Delhivery became an early user of the Elasticsearch Service (now on Elastic Cloud, but back then on Found.no). And since that point, Delhivery’s relationship with Elastic has expanded alongside its operations.

“Moving to a managed service gave us greater flexibility to scale and allowed us to focus on our core activities,” said Elasticsearch Architect and Senior Engineering Manager, Karan Agarwal. “We have looked at other options over the years, including Amazon’s Elasticsearch offering, but we have continued with the Elasticsearch Service because it is run by the experts and always offers us access to the latest features of the Elastic Stack.”

Delhivery is extracting more value from these features following a recent upgrade from version 5.x to 6.5, which was made seamless with the help of Elastic Support. The upgrade has also introduced dedicated node types into its clusters, enabling it to optimise memory and CPU as well as stabilise performance.

Each day, Delhivery ingests more than five million data points, including package geodata. Data flows through a Kafka Stream to Logstash and into an Elasticsearch cluster where it is used to power Kibana dashboards and custom applications.

Example architecture

Transaction / event data architecture

Example architecture

Log data architecture

Kibana remains instrumental in managing and monitoring daily operations, with security features from the Elastic Stack used to control which data can be accessed by each individual. This allows Delhivery to assure the security and privacy of data while opening Kibana up to teams across the company as well as exposing it to their end e-commerce customers.

Teams are able to visualise a wide range of performance metrics in Kibana, and use Elasticsearch to explore anomalies and trends. For example, if the control tower observes the delivery network’s performance drop to 97%, it can look at the performance of each facility and then keep filtering the data to identify the problem. It may also choose to monitor specific aspects of the network by using Elastic Stack APIs to create front-end widgets with embedded visualisations.

“The API interface and documentation supporting the Elastic Stack is very good, and makes it easy to experiment and quickly create new widgets and visualisations as different needs arise,” Karan. “For a company in this space that is innovating rapidly, it actually has the best documentation I’ve seen.”

From last mile delivery to world-class logistics provider

With proactive support from Elastic, Delhivery has optimised the health and stability of its clusters and continues to take advantage of new features and capabilities of the Elastic Stack. It is exploring the use of Elastic APM for monitoring and alerting of transport and warehouse management systems, and micro-services as well as for extended stack trace visuals of requests and interoperability. Delhivery is also investigating the use of Canvas as a way to build pluggable dashboards for its facilities teams and therefore reduce reliance on custom built dashboards for various use cases. These dashboards would give operators and managers self-explanatory and actionable data showing real-time fluctuations in packages and other events occurring at each facility.

“We’re now running more experiments and exploring new features and products like machine learning which would give our control tower the ability to make predictions on the capacity and performance of our delivery network. For example, if there’s a spike in activity at one centre, we can analyse what may happen at other facilities down the line,” said Kapil.

The intelligence and efficiency gained through machine learning and potential use cases for Canvas and Elastic APM will help Delhivery bolster its expansion and capacity to innovate. Today, the Elastic Stack is helping Delhivery make better and faster decisions about everything from resourcing through to where it should build its next facility, but they have ambitious plans to further disrupt the logistics industry with an operating system and platform that could be leveraged by others.

The Delhivery Clusters

  • Clusters
    2
  • Hosting Environment
    Elasticsearch Service on Elastic Cloud on AWS
  • Documents
    Express Cluster - 86,906,895

    Core Cluster - 94,409,502
  • Daily Ingest Rate
    Express cluster - average 470/s with max of 1000/s

    Core cluster - average 10/s with max of 50/s
  • Indexes
    Express cluster (month based indices grouped under 2 aliases) - 29

    Core Cluster - 15
  • Query Rate
    Express cluster - average 270/s with max 400/s

    Core cluster - average 10/s with max 15/s
  • Node Specifications
    Core cluster : 3 high cpu m5 instances each of 8GB ram and 64GB disk

    Monitoring: 1 high I/O classic instance of 4GB ram and 96GB disk

    Express: 4 high I/O data instances each of 58GB ram and 1.7TB disk

    3 Master r4 instances each of 4GB ram and 8GB disk 2 Ml m5 instances each of 4GB ram and 8GB disk