CADDi brings high-speed semantic search to manufacturing AI Data Platform with Elastic

Under the mission to "unleash the potential of manufacturing," CADDi, Inc. (referred to as "CADDi" below) offers products like the CADDi AI Data Platform for manufacturing companies to promote digital transformation that maximizes aggregated knowledge within manufacturing companies. However, during early stages of service launch, CADDi faced challenges in terms of the performance and stability of its search functionality — essentially the core of the system. As a solution, the company adopted Elasticsearch for its new search system, which successfully resulted in rapid search and reduction of operational overhead.

Established in 2017, CADDi specializes in transformation services dedicated to the manufacturing industry. Its first product, CADDi Manufacturing, was an end-to-end service for the procurement and sales operations of sheet metal components. As a manufacturer itself, the company oversaw supplier selection, quality inspection, and delivery internally.

Currently, CADDi is transitioning into a software company that provides services like its procurement application CADDi Quote, as well as the manufacturing Data Utilization Cloud CADDi Drawer, which aggregates data within client manufacturers for optimal use.

Engineering Manager Yuki Hashimoto joined CADDi in 2022 and currently works in the Product Development Division for CADDi Drawer as a manager, primarily overseeing search domains.

"While our original product CADDi Manufacturing performed well and expanded its business favorably, we identified a greater need within the manufacturing industry: that companies required business transformations on a much larger scale. By shifting our business and focusing resources into providing software solutions, we believe we can support transformations among a greater number of companies,” explains Hashimoto.

On the issues surrounding information management in the manufacturing industry — which formed the background for the development of CADDi Drawer — Hashimoto notes:

"Typically, there are two main operating systems in manufacturing. The first is Product Lifecycle Management (PLM), which manages information across each product's entire lifecycle. This includes information used in design and development, such as bills of materials, drawings, and attribute information. The other is Enterprise Resource Planning (ERP), which includes information regarding accounting, inventory, and order processing. While manufacturers had long been mindful of information management, integrating these details presented various issues.”

Most traditional search systems functioned as a term-matching system — for example, identifying where the drawings of a part matched a PDF file. This meant users couldn’t freely search historical data or filter search results according to various criteria. As a result, information from existing drawings was rarely reused, leading to inefficiencies such as having to create similar drawings from scratch.

"Traditionally, manufacturers used SoRs (Systems of Record), but our proprietary CADDi Drawer acts as an SoI (System of Insight) that aims to gather dormant information within companies and transform them into valuable assets,” explains Hashimoto.

As a tool to effectively utilize corporate information, CADDi Drawer was developed with two main features: a search function that allows users to search various types of data dispersed across the company in a single search, and a carefully designed user interface that enables users to fully control and leverage search capabilities.

"The CADDi Drawer prototype was implemented to support lexical search of drawings and allow users to search the entirety of its text by a keyword search. Furthermore, considering the time-consuming nature of checking search results one by one, we designed the interface to streamline the search process by displaying resulting drawings in a list format with thumbnails, achieving a quick view of search results," says Hashimoto.

However, the manufacturing industry encompasses a diverse range of businesses, and details like materials, units, and quantities require optimal search criteria for each industry. Not only that, but some companies need to search through hundreds of thousands of internal records, requiring a system capable of handling massive data searches. Furthermore, a key challenge remained in linking information managed across different departments to enable cross-departmental searches. Hashimoto and his team worked to resolve these obstacles one by one.

As development of CADDi Drawer progressed, the team adopted additional features beyond keyword searches, including semantic search that supported even ambiguous queries. To achieve this feature, CADDi hosted a dedicated engine for semantic search on the company's own servers, which was managed by the engineering team.

The team also identified the need for a relational database search system to query prestructured data, resulting in a total of three separate search systems in the back end of CADDi Drawer. "The three systems each had their strengths and weaknesses, making its management cumbersome. In particular, the semantic search engine was hosted on the company's own servers, imposing a significant operational burden on the infrastructure itself," reflects Hashimoto.

As Hashimoto worked tirelessly in search of a resolution, he came across Elasticsearch's new implementation of the vector search feature, which would allow for semantic search.

"I came across the news that Elasticsearch's search engine would support vector search in its upcoming version update. Not only that, but Elasticsearch would also support hybrid search that combined the capabilities of vector search with traditional keyword filtering. This meant that we may be able to realize our desired features on one centralized system."

Another advantage was Elasticsearch's simultaneous launch of the managed service Elastic Cloud.

"Adopting Elasticsearch would make data available in the optimal format, so we knew from preliminary testing that it would improve our search speed. By adopting Elasticsearch and utilizing its managed service, we could foresee a solution that would address the three challenges of integrating our search systems, outsourcing infrastructure management, and improving performance, all at once," says Hashimoto.

CADDi began transitioning from the former three independent search systems to Elastic Cloud, launching services in full scale in 2023. While the change wasn't directly visible to end users, Hashimoto explains that the user experience improved dramatically.

"The vast improvement in search speed is particularly apparent with regard to complicated search queries or hybrid search of different data sources. As long as the managed service provides search experiences as smooth as this, there's no need for us to conduct tuning of search engines ourselves."

This new rapid search speed made searching much easier for users, even when they needed to search multiple times, ultimately allowing them to retrieve desired results faster. Enhancing the search experience proved to be a crucial piece of the puzzle in achieving CADDi's vision to "transform information into assets."

Hashimoto also recognizes the value of the managed service's supportive role. "There was an instance where we fed the system too much data due to unexpected trouble, but we used the support ticket and Elastic resolved the issue immediately for a swift recovery. We have peace of mind knowing that we can rely on their support at any time."

Overall, Hashimoto says he is pleased with the largely improved search capabilities of CADDi Drawer, achieved through the implementation of Elasticsearch. With the added benefits of the managed service, the engineering team was relieved of the operational burden of the search system, allowing them to reallocate resources to focus on their primary mission of enhancing the customer experience.

“The knowledge we gained during our initial CADDi Manufacturing service phase was passed on to the AI Data Platform for manufacturing companies, which works directly with client manufacturers on-site every day. The feedback they learn on-site is shared with the development team for further improvement of efficiency,” says Hashimoto.

Hashimoto also values Elastic's proactive approach to information sharing, stating: "We initially considered a system governance structure that separated Elasticsearch's access rights between administrators and regular developers, but that function wasn't available at the time. However, once we approached Elastic with this idea, they informed us that the upcoming version of Elasticsearch would support this governance structure. This allowed us to coordinate internally in preparation for the update, ensuring a seamless implementation upon its release."

Looking to the future, Hashimoto expresses his anticipation of Elasticsearch's services in a serverless environment. "In CADDi's case, I am confident that serverless search would entail a significant improvement in cost performance," says Hashimoto.

CADDi believes that enhancing the value of the search experience can only be realized through combining high-quality data with a powerful search engine. In conclusion, Hashimoto remarks:

"Personally, I think we've only unlocked 20% of the search experience's true value. With the aim of untapping the potential of the remaining 80%, we look forward to our continued partnership with Elastic to enhance both our data and search capabilities."