Gurunavi, Inc.: Analyzing Large Volumes of Log Data with Operational Efficiency and High-Speed Search Functionality

Gurunavi is a forerunner in Japan among online services providing information on restaurants and other topics, and it has administered a large-scale ICT environment to support those services since early on in its life. Obviously, these systems output a large volume of log data, and analyzing that data previously took up a lot of time and manpower.

Toshiaki Iwamoto is Deputy Section Chief of the Engineering Section in the Development Department at Gurunavi. He recalls the circumstances at the time as follows.

“Around 2016, Gurunavi already had thousands of servers. When there was a breakdown or maintenance was needed, we needed to log in directly to the affected server to check the logs, look up the log with the grep command, copy the log temporarily to a local environment using the scp command, and deal with the problem that way. This investigation process would take over an hour to complete, meaning we could not respond in real time and would have a lot of manual work to do, and this led to oversights and other careless mistakes. As our systems and logs grew bigger and bigger, we increasingly needed something to deal with this workload.”

It was then that Gurunavi came across two Elastic Stack products: Elasticsearch and Kibana.

Elastic Stack enables high-speed real-time search across multiple data sources without downloading the target data to a local environment, through the use of a wide range of provided product components. As the company’s services have multiplied and become more advanced, API calls between applications have become increasingly numerous as well. Given that Gurunavi needs to analyze a multitude of logs, Elastic Stack products were really the optimal solution for its needs. “At the time, there were absolutely no products on the market that could be called competitors, and we decided on Elastic because it was the only choice,” said Mr. Iwamoto of Gurunavi. Gurunavi fully implemented Elasticsearch and Kibana in 2016.

In late 2018, several years after implementation, Gurunavi began using AWS.

Gurunavi chose this timing to switch some of its Elastic Stack licenses over to the cloud, as they needed to be renewed at that same time, and the company began using Elastic Cloud’s Elasticsearch Service. The aim of this was to implement the collection and analysis of logs in an AWS environment, as well as to reduce the operational workload of the Elastic upgrade.

“Some users said that they wanted to view AWS Cloudwatch Logs using the same high-quality Kibana interface they were already accustomed to using as of 2018,” said Mr. Iwamoto. “We compared log data collection and other features with competitors’ products, and Elastic Cloud came out ahead in terms of cost, performance, and always having the latest version available. For implementation, we elected to use and Elasticsearch .”

The other priority for Gurunavi was reducing the operational workload of upgrades and similar tasks. It was essential to avoid having these tasks become too reliant on a single individual’s skills, since Mr. Iwamoto was the lone personnel member with the requisite technical expertise. “Version updates are frequent, but I was the only one actually able to take care of them, which meant there was a massive workload on me alone. I thought we could resolve these issues as well by adopting Elastic Cloud.”

(Toshiaki Iwamoto, Gurunavi)

In this way, Gurunavi was equipped with a comprehensive Elastic environment, both on-premise and via the cloud.

Gurunavi’s current system handles logs acquired from all applications and network devices, which amounts to 50 types of databases and a total data volume of 50 billion documents.

For data extraction, log data from on-premises high-capacity clusters is sent to Elasticsearch using Logstash, while Cloudwatch Logs data in the cloud environment (including containers) is sent from Functionbeat to Ingest, as stated above. [Amazon] S3 data is sent to Elasticsearch via Logstash. Because all logs can be monitored transparently and cross-sectionally, it is possible not only to troubleshoot this data but put it to more strategic uses.

“Gurunavi is a service provider, which means that we handle a wide range of languages. Moreover, the type of log data acquired depends on the nature of the application involved, and there are a lot of indexes involved,” said Mr. Iwamoto. “The developer side wanted to view everything cross-sectionally, including the cloud. Now that this comprehensive Elastic environment has been set up, that is actually possible.”

Mr. Iwamoto initially assessed the overall benefits of Elastic Stack based on two points.

“First, it used to take over an hour to copy log data and check it by hand, and it has been a tremendous boost to our efficiency that these tasks have been reduced to mere seconds. This accelerated the entire process from the appearance of a problem to looking at the conclusion of the investigation into it, and the precision of the process has improved dramatically. Using the provided API, log alerts are sent to Slack or Webhook, and this enabled us to respond more quickly. Additionally, Elastic Cloud can be upgraded non-disruptively without the major workload involved in switching over to another version, and the fact that the latest version is always available for use is another major benefit. In the past, when I was handling upgrades almost completely by myself alongside my other work, it would take around five to ten days — including investigations — to perform an upgrade. Now, upgrades take almost no time at all.” (Toshiaki Iwamoto, Gurunavi)

It is also impossible to overlook the system’s contribution to the company’s new inbound tourism strategy.

For example, it is possible to use Elastic Stack to extract geolocation data and visualize site visitors’ locations. “Amazon Cloud Front is a CDN service in AWS, and it is possible to use its logs to see site access data on a per-country basis,” said Mr. Iwamoto. “Why are we getting so many visits from this location? And why so few from here? It is possible to use this data to raise such questions, and to drill down into the data to determine, for example, that there is a problem with relevant content or a service provider. Based on this information, we can take steps to increase content delivery speeds, pick timing for launching a campaign, or implement other such decisions that make the information extremely beneficial to the future of our in-bound tourism strategy.”

Another element that contributed to this intuitive decision was the visualization of graphical data using Vega. A new feature of Kibana 6.2 made it possible to use Elasticsearch data with Vega and Vega-Lite to create rich visualizations. With these visualizations, users could get an overview of information pulled out of massive quantities of data and use it to come to intuitive conclusions. This feature is also useful to people who are not IT specialists, such as staff in marketing or management departments.

Summarizing the benefits of implementing Elastic, Mr. Iwamoto remarked that “with our current complex system environments and the associated massive amounts of log data, it is impossible to imagine a world where Elastic Stack does not exist.”

Having enabled high-speed log searches and reducing the workload of version upgrades using the cloud, Gurunavi has already begun considering the use of Elastic Stack for more strategic purposes.

“With the Elastic environment and Elastic Cloud in place, we have been freed from the constraints of being reliant on a single individual’s skills for operational matters, and the system’s uses have been dramatically increased as well,” said Mr. Iwamoto. “For example, a dashboard created by application-side personnel showed prospective customer conversion rates over time against KPI. By making maximum use of the log data not only for troubleshooting and other system management-related purposes but also business strategy-related purposes like this, we believe it is possible to generate even higher levels of value.”

Gurunavi continues to provide high value-added services to both customers and restaurants.

Elastic Stack plays an essential role in making maximum use of the massive quantities of log data, totaling 50 billion documents, that make such services possible.