Van life isn't always easy, unless you have a job that's distributed by design. Learn how Joe DeFever made work into an adventure by taking advantage of Elastic's distributed nature.

I’ve always appreciated the trust Elastic bestows on each of us to do our jobs in a way that suits our individual modes of productivity. While remote work is now ubiquitous because of the pandemic, Elastic <a href="https://www.elastic.co/blog/culture-elastic-distributed-by-design?baymax=web&elektra=culture-working-for-elastic-out-of-a-van">has been distributed by design</a> from the start. With this in the company's DNA, Elasticians have the advantage of a healthy, distributed culture that was carefully cultivated years before everyone had to shift to a remote format overnight.
You like working in sweatpants? No sweat.
You enjoy heavy metal throughout the workday? Rock on.
You want to learn a new programming language on the side? Groovy.
<a href="https://www.elastic.co/about/our-source-code?baymax=web&elektra=culture-working-for-elastic-out-of-a-van">Come as you are and do great work.</a>
Unlike previous jobs where attendance itself can be a voucher of effort, it's the quality of productive output at Elastic through which we prove our efforts. Elasticians’ dedication to solving hard problems brings us (and our extended community) together. In a distributed work environment, we find presence through sheer productivity, not productivity through sheer presence. It’s precisely because no one is there to look over my shoulder that I’m continuously eager to prove my level of productivity. Working remotely out of a van this past summer, this eagerness was doubly so.
<h2>Hitting the road</h2>Earlier this spring — amid toilet paper and sanitizer shortages — my girlfriend Maddie and I packed up her converted van with the belongings that would last us through the next four months. With hiking boots, climbing gear, and mountain bikes packed...repacked...ok now it fits, we set out.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt55dca3aec538f7f6/5fad568b6f82405d9a4abc7d/podcasts.jpg" data-sys-asset-uid="blt55dca3aec538f7f6" alt="Podcasts don't fail us now! " style="display: block; margin: auto;">
<figcaption>Podcasts don’t fail us now!</figcaption>We had decided on a westward voyage from our home base in Denver that would culminate in wanderings through Utah, Idaho, Washington, Oregon, California, and Wyoming. As this excursion also happened to be our way of moving out of Denver (we would be relocating upon our return to Colorado), our departure was full of personal unknowns during a time of global unknowns.
This video shows some of the highlights of our summer’s life on the wheels:
<iframe src="https://player.vimeo.com/video/467140527" allow="autoplay; fullscreen" allowfullscreen="" width="850" height="478" frameborder="0">
</iframe><figcaption>Video compilation of the summer's happenings during free time</figcaption>While this "highlight reel" paints a fun picture, the video really encapsulates the scraps of time we were able to string together between weekends and the occasional weeknight. In reality, we had to revolve everywhere we went and everything we did around our ability to commit to focused work during the week.
There were many questions we'd repeatedly ask at the arrival of each new location: How strong is our wifi signal here? Is it quiet enough? Will we harness enough solar power throughout the day to keep our computers/router charged? We'd determine our route based on our ability to put in quality work first. The adventures came only once we'd met that criteria.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blted842afc049ba20f/5fad56c972a3526f28dbb6d4/ideal_office.jpg" data-sys-asset-uid="blted842afc049ba20f" alt="The ideal office. " style="display: block; margin: auto;">
<figcaption>An ideal office location near Moab, UT: Strong signal, quiet, ample solar power.</figcaption>Bottom line: Despite what our video and whatever #vanlife social feeds out there may glorify, the reality of two full-time professionals living and working out of a submarine-style vessel is far less dazzling than the carefree image of soulful wanderlusts setting out into the sunset.
Fortunately, our internet connection was consistently strong throughout our time on the road — even during screen-sharing video meetings, bandwidth speeds didn’t falter. Maddie’s “office” was the van’s swivel-chair passenger seat, while mine was in the kitchenette/hallway/common area. Though, like myself, Maddie works in marketing, our jobs are very different. I help create marketing content for <a href="https://www.elastic.co/security?baymax=web&elektra=culture-working-for-elastic-out-of-a-van">Elastic Security</a>. Maddie is the one-woman marketing department for a much smaller (also fully distributed by design) cost engineering company. I have Zoom meetings steadily throughout the week. Maddie’s company communicates primarily through chat. Our shared office dynamic worked well, and it was pleasant to have a change of scenery outside the van’s sliding door each week.
None of this would have been possible if I wasn’t part of such an amazing team. By heading westward into the Pacific Time zone, my schedule now actually better aligned with my boss and the team members that I work closely with. While in Portland, I was even able to grab a socially distanced lunch with my longtime colleague and fellow Elastic Security Product Marketing Manager.
<h2>Traveling during a pandemic</h2>Exploring across different states, we saw how the pandemic had changed places, people, and attitudes. The spectrum of western Americans’ precaution towards the pandemic was wide-ranging — while some would cross the street to allow for more room when walking by, others didn’t wear masks in grocery stores. The mini culture shock of adjusting to local customs made crossing state lines seem like international endeavors.
We also noticed a lot of economic hardship on the road. Entire towns were closed. Homeless camps existed in places we hadn’t seen on our previous summer’s trip. We quickly came to bite our tongues when waxing nostalgic about the creature comforts absent in the van. We gained a fresh perspective on how fortunate we are.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt26b85577f216118f/5fad570624612d7400da4dd4/smokey_sky.jpg" data-sys-asset-uid="blt26b85577f216118f" alt="Smoky skies across Northern, CA. " style="display: block; margin: auto;">
<figcaption>View of the smoky skies across all of Northern California — from San Jose to southern Oregon.</figcaption>Then the fires hit. We were chased out of the Bay Area, Portland, Idaho, Salt Lake City — even a final stopover to say goodbye to friends in Denver had us on the run. As many experienced, this summer’s fires were fierce in the western states. Residents in every state we visited were choking from wildfires that were oftentimes strong enough to keep even neighboring states’ residents indoors. 2020 has seen a lot of superheroes, and firefighters are more than worthy of donning the cape.
<h2>Home is where you park it</h2>Waking up to increasingly chillier mornings as fall closed in, our life on the wheels was coming to a close. As we knew we’d be settling into our new home in Durango, CO, Maddie sold the van to another outdoor-loving couple who we trust are putting it to good use. It’s fun to think that our former home on wheels is still out there, creating more memories.
Van living may be parked for now, but our appreciation for everything we experienced on the road, both the fun and the challenging, still drives on. Regardless of where we as Elasticians may be logging in from — whether it’s a skyrise in Singapore or a rest stop in Wyoming — we’re connected to a global team composed of brilliant, compassionate people. I continue to be humbled to be part of this company and our mission, and am more than gracious to contribute in my own small way to the important work we’re doing.
Interested in joining Elastic? We’re hiring. Check out <a href="https://www.elastic.co/about/teams/?baymax=web&elektra=culture-working-for-elastic-out-of-a-van">our teams</a> and <a href="https://www.elastic.co/about/careers/?baymax=web&elektra=culture-working-for-elastic-out-of-a-van">find the right career for you</a>! Want to read more about life at Elastic? Read more on <a href="https://www.elastic.co/blog/category/culture?baymax=web&elektra=culture-working-for-elastic-out-of-a-van">our blog</a>!

working-out-of-van-blog-banner.jpg

working-out-of-van-blog-thumbnail.jpg

Working for Elastic (out of a van)

This Elastic{ON} Tour talk details how Hill Enterprise Data Center manages/distributes US Air Force maintenance data securely and at scale with Elasticsearch.

<html><head></head><body><h3>About Hill AFB
</h3>
Hill Enterprise Data Center (HEDC) provides hosting services for more than 100 information systems of the US Air Force logistics center at Hill Air Force Base (AFB), Utah. As Doug Babb, Chief Architect at Hill AFB, explains, the amount of data these logistic centers produce in their responsibility for the maintenance of aircraft and intercontinental ballistic missiles &#x201C;is just beyond belief.&#x201D;

The HEDC team collects, monitors, and analyzes logs from Hill AFB&#x2019;s numerous information systems &#x2014; as well as its overarching supporting infrastructure &#x2014; to provide National Institute of Standards and Technology (NIST)-compliant monitoring of their multi-tenant hosting platform in real time. They need to continuously innovate their own Platform as a Service (PaaS) to deliver logging and Department of Defense (DoD) compliance monitoring for the lifecycle of their hosted information systems. They accomplish all this with support from <a href="https://www.elastic.co/products/ece">Elastic Cloud Enterprise (ECE)</a>.

<h3>Log aggregation
</h3>
Log aggregation takes on a whole new meaning at HEDC. Through building Portable Operating Databases (PODs), their private cloud can be linked across more than a dozen separate sites, which they use to communicate with other PODs at bases around the US. This results in a lot of data transfer and aggregation. The team needed a way to correlate all their systems of record. ECE has all the backings of <a href="https://www.elastic.co/cloud/elasticsearch-service">Elastic Cloud</a>, but is self deployed to operate on any public or private infrastructure. This was the clear choice for HEDC in their need to connect geo-distanced PODs with remote access.

For HEDC, it all starts with an effective ingest process. Raw data streams in through established information system partitions &#x2014; each comprised of a collection of servers. <a href="https://www.elastic.co/products/beats/filebeat">Filebeat</a> ships the data from these individual information systems through a load balancer and then to <a href="https://www.elastic.co/products/logstash">Logstash</a>, where it receives node designation. Once Logstash has applied a translate filter through a <a href="https://www.elastic.co/guide/en/logstash/6.6/plugins-filters-memcached.html">memcache plugin</a>, the bundled and polished data is dropped into HEDC&#x2019;s corresponding Info Sys cluster within ECE. A catch-all HEDC infrastructure cluster collects raw logs to function as a backend that the team can use for systems of record in the event that they need to comprehensively correlate data together to capture a picture in time.

And with that, their data is ready for the analytical power of <a href="https://www.elastic.co/products/elasticsearch">Elasticsearch</a>. HEDC can then perform all their necessary analytics, metrics findings, and compliance upkeep &#x2014; displaying these reports through <a href="https://www.elastic.co/products/kibana">Kibana dashboards</a>. But these dashboards only serve their purpose if they can be readily and securely shared across other PODs and within the DoD. That&#x2019;s where feature parity and security across these multiple systems becomes a major priority.

<h3>Seamless role-based authentication
</h3>
Authorization through <a href="https://www.elastic.co/blog/elastic-app-search-announcing-role-based-access-control">role-based access control (RBAC)</a> is an essential function for HEDC to securely grant access &#x2014; from data to dashboards &#x2014; to the appropriate PODs and DoD users. An included security feature within Elastic&#x2019;s Gold and Platinum <a href="https://www.elastic.co/subscriptions">subscriptions</a> (ECE comes with Platinum features), RBAC revolves around 5 constructs:

<ul>
	<li>Secured Resource: the resource to which access is restricted (i.e., indices, documents, fields, or clusters)</li>
	<li>Privilege: a specified course of action a user can execute against a selected resource</li>
	<li>Permissions: a set of one or more permissions against a secured resource</li>
	<li>Role: a named set of permissions</li>
	<li>User: the authenticated alias</li>
</ul>
RBAC provides the team with a consistent, cloud-shared UI that allows them to seamlessly access and corroborate data across different privileges and locations. HEDC can customize roles to be very specific. They may provide certain levels of information system access (with necessary machine learning plugins) for an engineer, application administration access, or create an access role with specific functions for the information system security manager (ISSM) &#x2014; a DoD requirement.

<blockquote>&#x201C;[Role-based authentication] is a requirement of NIST &#x2026; rather than using some kind of index or tag to split out people&#x2019;s data and deliver it to them within an index, which we don&#x2019;t want to do.&#x201D; - Nate Benson, Cloud Hosting SME | Hill AFB
</blockquote>
This role-based authentication also extends into their PaaS infrastructure deployment. &#x201C;We can allow our security compliance engineers and all engineers [into] infrastructure-based roles,&#x201D; Cloud Hosting SME at Hill AFB Nate Benson explains. Now analysts in SecOps, operations, hyper_infra, etc. can all have different levels of access to HEDC&#x2019;s vast data store.

<h3>The Elastic benefit
</h3>
So why go through all this trouble? Why the hefty ingest routing and role designation? &#x201C;We&#x2019;re doing this because again and again, we see people in the DoD frustrated,&#x201D; Benson explains. &#x201C;They&#x2019;re frustrated with paper compliance &#x2014; the inefficacy of it. They&#x2019;re frustrated that they get all these silos and it doesn&#x2019;t mesh or talk &#x2026; there&#x2019;s no way to actually time sequence it well together.&#x201D;

ECE provides HEDC with a streamlined ingestion process, secure data transfer and permissions protocol, and a (compliance mandated) correlated sequence of record. Providing all of this cloud-based data transfer and access across PODs is a task that ECE is uniquely qualified to handle quickly, securely, and at scale.
Want to learn more about how ECE helped HEDC securely monitor and secure its geo-dispersed data center? <a href="https://www.elastic.co/elasticon/tour/2018/washington-dc/monitoring-and-securing-a-geo-dispersed-data-center-at-hill-afb">Check out their Elastic{ON} Tour Washington, D.C. presentation</a> from October 2018.
<a href="https://www.elastic.co/elasticon/tour/2018/washington-dc/monitoring-and-securing-a-geo-dispersed-data-center-at-hill-afb"><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt046ccbc3742d3c7c/5cb8425d44c599c63eef1a50/hillafb-elasticon-play.jpg" data-sys-asset-uid="blt046ccbc3742d3c7c" alt="Watch &quot;Monitoring and Securing a Geo-Dispersed Data Center at Hill AFB" style="display: block; margin: auto;"></a>
</body></html>

air-force-aircraft-airplane-2000by415.jpg

air-force-aircraft-airplane720by420.jpg

Monitoring and securing a geo-dispersed data center with Elasticsearch

Learn why NS1 -- a leading DNS provider -- chose Elasticsearch as its time series database for telemetry data management.

NS1 is a leading Domain Name System (DNS) host and web traffic management company that equips customers with resilient, redundant, and multi-purpose DNS solutions for various application delivery stacks. Providing industry-leading web redundancy and speed means NS1’s time series database (TSDB) needs to collect and analyze incredible amounts of telemetry data — nominal throughput spikes can reach 700k data points per second — in as close to real time as possible.
This telemetry data comes from a globally distributed infrastructure that spans across systems, providers, and third-party integrations that all dynamically scale and emit logs, time series data, packet captures, sflow, and more. As NS1 continued to grow, establishing an architecture that could support real-time telemetry became a major need — one that NS1 DevOps and Software Engineer Christian Saide describes as being “difficult to solve elegantly.”
The team’s initial solution was a 13-server OpenTSDB cluster for handling OS and application metrics, as well as a three-server Elasticsearch cluster for collecting log data and general unstructured telemetry. This dual system worked well enough … until NS1’s terabytes-per-day operations continued to scale.
<blockquote> “These systems started to fail. Our telemetry systems started requiring more time to manage than the infrastructure it was intended to monitor.” - Christian Saide, DevOps and Software Engineer, NS1
</blockquote>As the team deliberated options for restructuring their TSDB, they established some core requirements:
<ul>
	<li>A single system for metrics and unstructured telemetry</li>
	<li>Granular control over what data is stored and retained</li>
	<li>Easy scalability</li>
	<li>Large, diverse, responsive community for support</li>
	<li>Sufficient analysis capabilities</li>
</ul>NS1 decided to go all in with Elastic. Not only would their primary need for a <a href="https://www.elastic.co/blog/elasticsearch-as-a-time-series-data-store">single, scalable TSDB</a> be solved with Elasticsearch, but they could now smoothly transition to the Kibana UI to make sense of the mountains of diverse data being ingested by a variety of Elastic <a href="https://www.elastic.co/beats">Beats</a> — <a href="https://www.elastic.co/beats/metricbeat">Metricbeat</a>, <a href="https://www.elastic.co/beats/filebeat">Filebeat</a>, <a href="https://www.elastic.co/beats/packetbeat">Packetbeat</a>, and <a href="https://www.elastic.co/beats/heartbeat">Heartbeat</a>. Product features aside, it was in large part Elastic’s community that won over the team and influenced their decision.
“The community and all of the plugins all fit together and make a cohesive environment for you to utilize,” Saide explains. “There is a myriad of blog posts and different organizations and companies out there that are explaining how they use Elasticsearch and providing insight to others on how they might be able to utilize it … [<a href="https://discuss.elastic.co/">discuss.elastic.co</a>] has been an absolutely amazing tool for us.”
They also found value in data introspection and data structure support. Gone were the days of simple keyword and wildcard matching; they could now perform searches for everything from IP range and numeric lookups to actual full-text queries and tokenization. “These are incredibly powerful tools to allow your operations team to actually take a look at what you’re collecting and use that in interesting ways to solve problems or understand what’s happening in an infrastructure.”
<blockquote>“This has been an incredibly effective solution for us at NS1.” - Christian Saide, DevOps and Software Engineer, NS1
</blockquote>NS1’s biggest triumph by far has been their ability to manage a single operational telemetry cluster. “I cannot tell you how much time, money, and just cognitive brainspace that this has already saved us,” Saide says. “[W]e get to focus on one and really learn one cluster. And it has been an absolutely amazing opportunity for us to spend more time working on other systems that are more directly related to our company.”
Elastic’s clustering capabilities quickly became a hit at NS1. The automatic and manual multicast and unicast clustering options in Elasticsearch — and inherent fault tolerance therein — meant the team could readily scale up and down as needed without worrying about their cluster. They discovered they could take down nodes for major upgrades or maintenance with zero downtime.
Pioneering a major TSDB architecture overhaul doesn’t come without its learning curve, though. In the <a href="https://www.elastic.co/webinars/ns1-leveraging-elasticsearch-as-a-time-series-database">Leveraging Elasticsearch as a Time Series Database</a> webinar, Saide details not only the successes, but also the challenges that came with NS1’s rebuild and even dives into suggestions for adopting Elasticsearch as a primary cluster source.
<a href="https://www.elastic.co/webinars/ns1-leveraging-elasticsearch-as-a-time-series-database"><img src="https://images.contentstack.io/v3/assets/bltf7afce26b89a5b33/blt3abfb7923b7a49ba/5c934f67ce43ee193330c73a/download" data-sys-asset-uid="blt3abfb7923b7a49ba" alt="Webinar: Leveraging Elasticsearch as a time series database - lessons from a DevOps engineer" style="display: block; margin: auto;"></a>

antenna-cell-tower-2000by415.jpg

antenna-cell-tower-cellphone-masts-720by420.jpg

Elasticsearch as a time series database for telemetry data at NS1

This Elastic{ON} Tour recap explores how Cyber Ops in the Department of Defense used Elastic to scale security data ingest and conduct critical threat hunts.

<h3>About CERDEC/ARL</h3>The Communications-Electronics Research, Development and Engineering Center / Army Research Laboratory (CERDEC/ARL) performs network and system monitoring as one of 23 cyber defense entities within the U.S. Department of Defense (DoD). By leveraging real-time operational data through the application of new technologies and advanced analytics, CERDEC/ARL conducts defensive cyber operations and research against the most sophisticated and damaging cyber threats of today. Its scope of coverage spans across any and all services, personnel, and networks within the DoD — the world’s largest employer by number of employees.
CERDEC/ARL’s Chief of the Sustaining Base Network Assurance Branch, Curtis Arnold, had the unique challenge of overseeing a multifunctional department whose defensive cyber operations were reputed to be a couple years ahead of others in the DoD. Arnold’s team needed a scalable, resilient, reliable platform that could perform equally well in supporting massive amounts of data and enabling nimble threat hunting capabilities. <a href="https://www.elastic.co/products/ece">Elastic Cloud Enterprise</a> (ECE) proved to be the solution they needed.
As the DoD’s Cybersecurity Service Provider (CSSP), CERDEC/ARL leads the department in “solving the hard problems that nobody else wants to tackle.” As the DoD continued to refine its cloud infrastructure, the need for distributed search became apparent to Arnold and his team. Traditional network-based sensors were losing their visibility with increasing encryption, the ever-expanding application space was requiring unachievable growth in network data normalization, and defense in depth against back doors in the network was not being adequately covered.
<h3>Implementing ECE</h3>To address these concerns, Arnold and his team tackled a major revamp in infrastructure. One of the larger reconstructs was in streamlining their data ingest architecture. Data would now be streamed from multiple log sources — flow, API, and application logs — to a central location. From there it would be distributed to cloud storage and an ECE cluster.
Elastic Cloud Enterprise brought immediate results. Policy enforcement, <a href="https://www.elastic.co/products/stack/machine-learning">anomaly detection</a>, threat hunting, search visualization, and programmatic access functions were all drastically enhanced. In a cloud access monitoring use case, a team analyst was able to pull data on cloud usage and effectively monitor potentially compromising user access points within one hour of using Elastic.
ECE “[g]ives us a lot more information on our data,” Arnold explains. “We’re a big player within the big data platform … Elastic is one of those capabilities that we’ve been bringing to bear. We’ve provided training for all the defensive cyber operators in Elastic … so we’re really pushing hard on making sure [it’s] part of the toolkit for everybody.”
<h3>Cyber threat hunting</h3>In case CERDEC/ARL didn’t already have its work cut out for it, it also leads the DoD’s joint cyber threat program — an initiative focused on improving insider threat detection. This critical security exercise requires the ingestion of vast amounts of data from a variety of different sources to establish a regular “pattern of life” among users. When a user’s activity deviates from their normal day-to-day functions, CERDEC/ARL may have an insider threat on their hands. ECE provided the team with comprehensive data sourcing, while Elastic <a href="https://www.elastic.co/solutions/security-analytics">security analytics</a> enabled rapid response speeds in a setting where every second can mean sensitive documents are being infiltrated.
In addition to enhanced search and indexing capabilities, Arnold’s team was also empowered through <a href="https://demo.elastic.co">Kibana visualizations</a>. Showcasing data and incidents in a universal, visually digestible format helps with overall understanding, but it’s also been proven to greatly improve analysts’ retention rate for incident reports. “The fact that this type of search and visualization is so quick and easy to render allows our analysts to express and convey the state of security easily to those that are in the position to remediate,” Arnold writes in a use case on detecting policy violations.
<h3>Conclusion</h3>Whether restructuring the ingest architecture for the DoD’s vast stores of data or quickly drilling down on insider threats, ECE has proven to be a dynamic and effective tool in helping CERDEC/ARL take on challenges of all types. Learn how Arnold’s team is utilizing Elastic Cloud Enterprise to defend the DoD’s networks in this recap video from <a href="https://www.elastic.co/elasticon/tour/2018/washington-dc/countering-threats-with-the-elastic-stack-at-cerdec-arl">Elastic{ON} Tour Washington, D.C.</a>
<a href="https://www.elastic.co/elasticon/tour/2018/washington-dc/countering-threats-with-the-elastic-stack-at-cerdec-arl"><img src="https://images.contentstack.io/v3/assets/bltf7afce26b89a5b33/bltc1d450ede999ece9/5c900852d17715ce39242b0a/download" data-sys-asset-uid="bltc1d450ede999ece9" alt="Watch the Elasticon talk: Countering Threads with the Elastic Stack at CERDEC/ARL" style="display: block; margin: auto;"></a>

Articles by Joe DeFever

Working for Elastic (out of a van)

Monitoring and securing a geo-dispersed data center with Elasticsearch

Elasticsearch as a time series database for telemetry data at NS1

Countering cyber threats with Elastic Cloud Enterprise at CERDEC/ARL

プロダクト

Elasticについて

情報