Customers

Zero Latency: Building a better VR gaming experience with Elasticsearch

By
Zbigniew Nastalski

At Zero Latency, our purpose is to create the greatest free-roam, multiplayer, virtual reality experiences in the world. Wearing virtual reality goggles and a portable computer on their backs, players walk freely in a warehouse-scale space. It is impossible to describe what it truly feels like but the video should explain it better:

Unique challenges

In our games, players are isolated from the real world and are fully immersed in a virtual experience. They wear goggles and headphones that effectively make them lose awareness of physical walls and other people.

Inside a VR arena

Our players are constantly monitored by game masters who prevent dangerous behaviours. Also, we have developed technology that alerts players of the proximity of any obstacles or other players even if they are behind them and not directly visible. These measures work great in preventing collisions, but the human factor cannot be completely eliminated. Especially, in high adrenaline situations when players are cornered by zombies!

Avoid zombies (and walls)!

Our main challenge is designing experiences that allow for eight players in a limited physical space while ensuring safe distances between players and walls at all times. But another challenge is the sheer amount of backpack computers, servers, cameras and blasters required for the experience to work correctly. With 30 venues around the world there’s always a chance of a failure that needs to be detected and fixed quickly.

Player Analytics

We keep track of every movement a player makes, not just to help keep them safe, but also to help us improve gameplay. It's a lot of data to track, though, so we need a powerful analytics solution.

What’s great about the Elastic Stack is that we can use the same tool that we use for log aggregation to collect and visualize analytics data coming from our software. By following this blog post we configured Filebeat to read structured JSON logs and push them to Elasticsearch. Our games and other software applications write these JSON documents to a file — one line per document — that is picked up by Filebeat. We found it to be very simple yet very reliable way of collecting analytics data.

Player analytics ingest model

With structured analytics data in Elasticsearch we can start answering questions about how our players behave in the game space. Particularly interesting questions are:

  1. Is the physical space used efficiently?
  2. Are there any spots that are underutilized?
  3. How close players get to each other on average?
  4. How does this average proximity change as the game progresses?
  5. How often do we show the proximity alert?

To be able to answer those questions we collect position and orientation data every second for each player. We also collect some extra information about whether the proximity alert is shown, etc. The JSON documents look like this:

Player analytics data

With this data, let’s say, we want to visualize how the space is being used in our Engineerium game and where the proximity warnings are being shown. We have built a Vega-Lite scatter plot in Kibana showing top down view of the play space:

Player movement data

7% of collected player movement samples indicated that the proximity warning was shown to the player.

Engineerium is a slow-paced, mind-bending puzzle game where players walk on plates suspended in the air. The game forces players to stay on the plates that are mostly arranged in straight lines. From the data we collected we can see that out of about 4000 samples collected 7% percent of them indicate that players were shown the proximity warning. The warning is shown when the players get too close to each other or to the walls. With this data we can start making decisions about how to arrange the plates to improve game space utilization and reduce the time the proximity warning is shown.

Inside the game Engineerium

Engineerium

Logging

Elastic Stack has been great at giving us real-time view of our infrastructure. With centralized logging we no longer need to coordinate with the venue operators to turn the backpacks on so we can copy over the log files. Thanks to Elastic alerting features, we know about hardware or software problems before they impact our customers. Our software engineers and support staff love the ease of use and power of Kibana.

The architecture is very simple in our case:

Logging ingest model

Conclusion

What we love about Elastic Stack is its power and flexibility. It is perfect for log aggregation but it can also be leveraged for data analytics. With support for Vega we can build powerful visualizations that are specific to our use case. It is also easy to get started with, especially with Elastic Cloud and that balance of ease of use and power is what makes it such a great tool.

Zbigniew Nastalski is a Senior Software Engineer at Zero Latency.