15 September 2015 User Stories

How Totango uses Elasticsearch to build the world’s finest Customer Success Platform

By Oren Raboy

portrait_oren.jpgOren Raboy is responsible for product development and operations at Totango. He works relentlessly to build & grow the software platform that powers the Totango customer success vision and ensure it is delivering high-quality insights to all customers, at all times. Prior to Totango, Oren was chief architect and head of products at P-Cube (acquired by Cisco) and founder of Beyondo and Kidyos. In short, a perpetually struggling entrepreneur.


Totango is a Customer Success platform used by subscription and recurring revenue businesses to reduce customer churn, drive product adoption, and maximize customer lifetime revenue.  Today companies have unprecedented data and visibility into their users’ behavior and the business results achieved by their customers. Totango monitors this data to generate insights on customer health and engagement. Using Totango, companies can pinpoint at-risk accounts that need attention; spot opportunities to increase user engagement and boost revenue; and then implement customer success best practices to scale up operations across a growing customer base.

Totango is used by some of the fastest-growing technology companies, including public companies like Zendesk and Autodesk; mid-stage companies like BigCommerce and Jobvite; and innovative startups like Optimizely and Mixpanel.

How Totango Drives Customer Success

Totango is all about analyzing customer data and translating it into actions for improving Customer Success.

With Totango, a Customer Success Manager can get instant feedback about which users require special attention, so they can better support them, increase adoption, and ultimately, grow  the lifetime value of a customer, all while reducing churn. For example, when using Totango a success manager gets notified when a user first starts using an advanced functionality, or if the number of active users in an account has declined to an all-time low. Totango can even send emails to users when certain conditions are met to fully automate the success process.

Totango’s Architecture Requirements

To accomplish this, we designed the Totango technology platform with a few key architecture requirements:

  • Real time: Totango needs to detect when a user does something important (such as trying out a new product feature or failing to complete an application action) and surface that data in near real time to the customer success staff so they can take action. Relying on long ETL processes with stale data is not sufficient in today’s fast paced environments.
  • Historical perspective: Totango often needs to reference a user’s full history in order to make meaning of their current situation. By comparing the activity level today with trends in recent weeks, months, or even years, Totango is able to determine if a user is doing well or falling behind. The technology must allow for efficient and expressive aggregation of time series data so that analysis can be easily made.
  • Flexible data-model:  Every cloud application provider is different.  User actions are different, contracts and subscriptions are different, not to mention, the customer lifecycle is different. Totango data-model needs to easily adapt to every cloud application with ease. This means we can’t rely on a rigid data schema and fully structured data
  • Scale, scale, scale: Totango is built for the world’s largest cloud applications. Millions of users and accounts need to be managed in a reliable and robust fashion.

Balancing these requirements requires a solid Data Architecture and use of the best NoSQL technologies the industry has to offer. We were fortunate to be one of the early adopters of Elasticsearch and have been using it for the past 4 years as a core technology in our stack. It has enabled us to address the needs above while supporting our growing customer base. Elasticsearch's flexibility has been key to us continuously innovating and delivering new functionality to the market.

In this post, we will highlight how Elasticsearch has helped us build the world’s finest data platform for Customer Success.

Elasticsearch in The Totango Platform Architecture

Within the Totango data architecture, Elasticsearch serves as the data-store infrastructure in our serving layer.

Totango Blog Image.png
Totango Elasticsearch Architecture

When a new event (such as a user logging in, or a new business transaction) is collected and processed by Totango, its results are indexed in realtime into the Elasticsearch index, making it available for use by the Totango users.

Additionally, each day Totango processes all collected information, applying statistical models and other data-science techniques. It then indexes a daily snapshot of insights within our historical archive, which also leverages Elasticsearch. Using this archive, we can present daily and weekly trends of metrics to see how things progress overtime.

In aggregate, Totango maintains multiple Elasticsearch indices for every one of our clients, each aggregating many millions of documents representing the state of a user or account in a snapshot in time.

All this happens transparently to our customers who receive a virtual, cloud-based solution. Elasticsearch, along with the rest of our infrastructure is abstracted and wrapped in the beautiful, action-oriented user-interface we developed at Totango.  The user receives alerts of important events, analyzes customer usage through segmentation and reports and even setup rules to send an email-campaign when users perform certain actions. This all happens without having to consider the complexities of collecting, normalizing, processing, and storing complex multidimensional data. 

Totango Blog Image_2.png
Totango in Action

Benefits of Elasticsearch 

We’ve come to rely on the following aspects of Elasticsearch to help us build a rock solid and flexible data layer architecture:

  • Schemaless data store: Elasticsearch ability to store and index documents without a predefined schema allows us to support different customers with ease. With the same fundamental architecture we can easily support customers with different applications and customer-lifecycles. This gives each customer total flexibility on how to model their Customer Success process without having to pre-define a data-schema that fits all
  • Aggregations: Totango relies on Elasticsearch aggregation to provide us real-time analytics capabilities. Need to know the average engagement of all accounts with a certain contract value? Or the health-score of accounts owned by a certain account-manager that renew this month? These are all a query away using our Elasticsearch cluster.
  • Horizontal Scaling:  In the way Totango uses Elasticsearch, we can scale endlessly by increasing the size of our Elasticsearch cluster. In this way, we economically grow our service as our customer base grows.
  • Flexibility: Elasticsearch rich filtering and aggregation capabilities allow us to constantly innovate on the ways we segment and present data to users. This allows us to constantly release new features to drive more value to our customers.
  • Faster Time-to-Market: The ease and speed by which we can develop using Elasticsearch has allowed us to develop and release features much faster than before. 

Future

While we make extensive use of Elasticsearch at Totango, we know there are still features and capability offered by Elastic that we have yet to fully utilize. Here are some of the items we look forward to include in one of our next projects:

  • Use the full ELK stack: While we make extensive use of Elasticsearch, we have yet to  start using Logstash and Kibana in earnest in our environment.
  • Usage of some of the latest features including aggregations and more advanced usage of parent-child document relationships.
  • Improve and streamline cluster operation using Marvel and other operational tools. 

To learn more about Totango visit our website or follow our technical writing on our engineering blog.