Samtec shares how Elasticsearch and Kibana has helped them build out an internal business intelligence tool for their enterprise from sales to manufacturing.

<html><head></head><body>Let&#x2019;s get this out of the way: we&#x2019;re not the typical Elastic users. While most use Elasticsearch for logging, security, or search; we use Kibana and Elasticsearch for business intelligence across our enterprise from sales to manufacturing. Also, we have applications using Elasticsearch as a primary data store, and our &#x201C;production&#x201D; cluster is often running pre-releases to take advantage of the newest functionality. That being so, our origin with Elastic is likely the same as yours &#x2014; it all started with needing a place to dump a whole bunch of logs.
Samtec builds electrical interconnects found in products ranging from medical devices, servers to self-driving cars. Our group &#x2014; Smart Platform Group &#x2014; was born from an area of Samtec that builds high-speed copper and optical interconnects. Samtec&#x2019;s <a href="https://www.samtec.com/optics/optical-cable/mid-board/firefly">FireFly&#x2122;</a> family of interconnects are a great example. They have 192 Gbps of bandwidth packed onto a device the size of a dime.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt67efb54b1461a965/5c57e20edfaabbd40b9c2980/samtec_1.png" data-sys-asset-uid="blt67efb54b1461a965" alt="samtec_1.png" style="display: block; margin: auto;">
The process to manufacture Firefly&#x2122; requires hundreds of steps. Almost every step gets performed by equipment that produces logs containing volumes of data. For example, one step of the process is to place silicon die onto a printed circuit board. The placement gets measured in microns (1/1000th of a millimeter), and the machine that does the placement holds onto each die with a small amount of vacuum pressure. That pressure gets logged for every placement. Those logs look something like this:
<pre class="prettyprint">&quot;die_count_x&quot; =&gt; &quot;integer&quot;
&quot;die_count_y&quot; =&gt; &quot;integer&quot;
&quot;commanded_pick_force&quot; =&gt; &quot;float&quot;
&quot;actual_pick_force&quot; =&gt; &quot;float&quot;
&quot;commanded_place_force&quot; =&gt; &quot;float&quot;
&quot;actual_place_force&quot; =&gt; &quot;float
</pre>We needed to start with capturing the data that would be lost over time as the machines rotated their logs. Logstash, and by extension Elasticsearch, were determined to be the quickest and most cost-effective solution, requiring the least development and maintenance effort. So we wrote some Logstash configs and started dumping data into Elasticsearch. It may not have been pretty, but it worked and we more-or-less forgot about it. That was late 2015.
Three months went by, then six, then a year &#x2014; and the data was never touched. Finally, one fateful day a customer request came in with an early device failure &#x201C;in the field&#x201D;. We needed to look back to when that customer&#x2019;s order was running on a specific machine to see if everything looked normal in the production process.
Cue our first Kibana visualization, a simple line chart showing the force used to pick up that small silico die. We knew the build date of the device, so we filtered our chart by date and got something similar to the below.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blte01584a789210cd0/5c57e208713ebdec0ba06f93/samtec_2.jpg" data-sys-asset-uid="blte01584a789210cd0" alt="samtec_2.jpg" style="display: block; margin: auto;">
Each dip in the top line represents a period where the force seems to have dropped suspiciously low. Naturally, we had lots of questions, but the most pressing was: did the device in question get built during one of those periodic dips? The exact times each order and piece got ran was stored in our SQL database, and we needed to bring the two pieces of data together. Each machine had a vastly different data structure, and the logistics of moving them all into SQL seemed problematic. However, moving tabular SQL data into Elasticsearch was a <a href="https://www.elastic.co/guide/en/logstash/6.3/plugins-inputs-jdbc.html">well-documented</a> path with Logstash. So we wrote another Logstash config to bring in the SQL data and leveraged some Kibana-fu to overlay orders and serial numbers onto some time series charts.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6b93de6407214629/5c57e202773ad9030c5c2c24/samtec_3.jpg" data-sys-asset-uid="blt6b93de6407214629" alt="samtec_3.jpg" style="display: block; margin: auto;">
Given the above, it looked like there were a couple of transactions that occurred during a time of &#x201C;low pick force&#x201D;. We were going to have to review those in more depth.
What started as one or two Logstash configs was threatening to turn into hundreds as the number of data sources we wanted to pull into Elasticsearch grew. Rather than continue to spend developer time creating Logstash configs, a product idea was born focused on making Elasticsearch data imports more accessible. Moreover, we wanted to enable self-service Business Intelligence using the Elastic Stack.
<h3>Enter Conveyor</h3><a href="https://github.com/samtecspg/conveyor">Conveyor</a> is an open source plugin that we wrote with data analysts and business users in mind. We wanted a graphical, interactive way for loading data into Elasticsearch. We wanted it to be easily extended to support different data sources, and we wanted it to integrate well into the Elastic Stack.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc16166a246df82bb/5c57e1fc4c9f4feb0b4c589c/samtec_4.jpg" data-sys-asset-uid="bltc16166a246df82bb" alt="samtec_4.jpg" style="display: block; margin: auto;">
So now that any data is just a few buttons away from being in Elastic, what kind of possibilities are within an easy reach?
We&#x2019;ve replaced our Tableau Dashboards with Kibana ones. In this case a heads-up display for showing order status in a manufacturing center. Auto-refresh meant the display showed new orders near real-time.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltdac90b05a5ec2f3a/5c57e1ddc545afde0bc70880/samtec_5_6_2.png" data-sys-asset-uid="bltdac90b05a5ec2f3a" alt="samtec_5_6_2.png">
We&#x2019;ve used Conveyor to pull in bill of materials and inventory data. We put it all in to Graph to quickly trace suspect lots through manufacturing and to identify other affected lots. In this case, we were able to easily determine customer orders (green circles) that consumed a suspect lot (red circle) even though there was a sub-assembly process that occurred midstream (pink circles).<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt77f83cecdbaaecef/5c57e1f022d96be10bcca385/samtec_7.jpg" data-sys-asset-uid="blt77f83cecdbaaecef" alt="samtec_7.jpg" style="display: block; margin: auto;">
We&#x2019;ve also used Conveyor to pull in process control data and analyzed it for anomalies using Elastic machine learning. In this case, we analyze a metric that gauges the health of a test station &#x2014; and we can easily identify or be alerted when it isn&#x2019;t as expected.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt97afbe3a8e7b02f8/5c57e1ea22d96be10bcca37f/samtec_8.jpg" data-sys-asset-uid="blt97afbe3a8e7b02f8" alt="samtec_8.jpg" style="display: block; margin: auto;">
Finally, we use Conveyor and Kibana together to build powerful dashboards on all sorts of business metrics, like GitHub <a href="https://developer.github.com/v3/repos/statistics/">Repository Statistics</a>, for our open source chatbot platform called <a href="https://spg.ai/articulate">Articulate</a> to cheer-on the increasing downloads and stars!
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltd97e1c26b7c666dd/5c57e1e4516e21cf0b2a0fc7/samtec_9.jpg" data-sys-asset-uid="bltd97e1c26b7c666dd" alt="samtec_9.jpg" style="display: block; margin: auto;">
<h3>What&#x2019;s Next</h3>We&#x2019;ve now been using Elasticsearch as a primary data store for our projects for more than four years and though we don&#x2019;t have big data like some users of Elastic we have broad data. With the introduction of Conveyor, as well as new Kibana functionality like <a href="http://canvas.elastic.co/">Canvas</a> and <a href="https://www.elastic.co/guide/en/kibana/6.3/vega-graph.html">Vega</a> visualizations, we strongly believe that the Elastic Stack is the best open source business intelligence platform. To reinforce that we&#x2019;ll leave you with three thoughts:
<ul>
	<li>Joining disparate data sources in a single analytical tool is extremely powerful and compounds the value of your data.</li>
	<li>Dynamic mappings and re-index capabilities make Elasticsearch an excellent collect now, analyze later data store.</li>
	<li>Cross-index search enables powerful information gathering on business data. (Want all sales orders, shipments, and contacts for a customer? Just search for their name across multiple indices).</li>
</ul>We hope to be back to share more in-depth walkthroughs on using Conveyor with Vega and Canvas, as well as our experience with the new <a href="https://elastic.github.io/eui/#/">Elastic User Interface</a> (EUI) library. We&#x2019;re happy to share our experiences so keep an eye out for those here or on our site at <a href="https://blog.spg.ai">https://blog.spg.ai</a>. Also, be sure to check out <a href="https://github.com/samtecspg/conveyor">Conveyor</a>
<hr>Caleb Keller, Woo Ee, and Mike Lutz head a team of data lovers, tinkerers, and technology enthusiasts building open-source solutions for Samtec&apos;s Smart Platform Group 
</body></html>

samtec_banner3.png

samtec_thumb2.png

From Tableau to Elastic: How Samtec Streamlined Business Intelligence & Analytics

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Mike Lutz

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

El contenido de esta página no está disponible en el idioma seleccionado. Elastic está trabajando para garantizar que el contenido esté disponible en varios idiomas. Gracias por tu paciencia mientras trabajamos.

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/elastic-cloud-account-terms" target="_self">Condiciones de servicio</a>, y que acepta recibir las <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">comunicaciones seleccionadas</a> en los detalles de contacto que proporcionó anteriormente. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/serverless-preview-terms" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Al enviar esta información, aceptas los <a href="/es/agreements/global/cloud-services-monthly" rel="nofollow">Términos de servicio de Elastic</a>. Los datos personales se procesarán de conformidad con la <a href="/es/legal/privacy-statement">declaración de privacidad de Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/terms-of-use" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

Hosted by

All Solutions

Follow us on Youtube

Watch now (no PT)

Articles by

Share by Email

Filters

All (no PT translation)

Contact information

See when this webinar starts in my time zone

Ver cuándo comienza este webinar en mi zona horaria

Thank you for your interest!

Register to Attend

Small image for

Register to Watch

Continue reading

Featured webinar

What to explore next...

Reset all

Load more press releases

Sign In to Attend

Share

Upcoming webinar

Learn more

Search Integrations

Share on Reddit

Share on Twitter

Share on Facebook

Load more news

View more posts

On-demand webinar

Follow us on Facebook

Author

Press Release

You'll also receive an email with related content.

You'll also receive an email with related content

Además, te enviaremos contenidos relevantes

Print

Contact Info

Overview

Video for

Read less

Register now

Share this story

See All Posts

See all top stories

View next

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

¿No puedes asistir? Regístrate y te enviaremos la grabación. También recibirás un correo electrónico con contenido relacionado.

Follow us on LinkedIn

Highlights

Explore similar demos

Follow us on Twitter

Share on LinkedIn

Load More

See more insights

Speakers

Close

Thank you for registering. We will send you a confirmation email soon.

Gracias por registrarte. Te enviaremos un correo electrónico de confirmación pronto.

Author

Articles by Mike Lutz

From Tableau to Elastic: How Samtec Streamlined Business Intelligence & Analytics

Sign up for Elastic Cloud free trial