App Search is available on Elastic Cloud as Enterprise Search, which combines App Search and Workplace Search into a single solution (read the Elastic Cloud announcement).
This document explains how to migrate your engines from App Search on Swiftype.com to Enterprise Search on Elastic Cloud.
When provisioning Enterprise Search for Elastic Cloud, you’ll need Enterprise Search (App Search) and Elasticsearch nodes.
Enterprise Search uses an Elasticsearch cluster to store everything, including your documents, logs, and analytics. You’ll spin up a number of Elasticsearch nodes along with your Enterprise Search instances on Elastic Cloud.
To determine how much storage you’ll need, you can start with a general rule of thumb: multiply the number of documents across all engines by 250kb.
Migrations with less than 100,000 documents should be handled well by the default deployment in Elastic Cloud. If you’re dealing with more than 100,000 documents, you may want to consider increasing your storage. Additionally, if you have analytics or log retention requirements, you’ll need to account for those when provisioning storage.
Provisioning an Enterprise Search deployment on Elastic Cloud requires manual resource management. Notably:
- Analytics and logging indexes must be managed manually or with some automation you will need to manage yourself (old analytics and logs indexes need to be removed according to your desired retention policy)
- Storage should be monitored closely during initial document ingestion
- CPU usage should be monitored closely once the new engine is receiving production traffic
For each engine you will:
- Migrate engine-level settings
- Add the new engine as an additional ingestion endpoint
- Back fill your documents
- Update the query and ingestion clients to point to the new engine
Documents should be imported to your new Enterprise Search on Elastic Cloud engines from the canonical data store, not from the existing App Search engine
- Analytics cannot be migrated. If you need access to historical analytics, you should export them using the Analytics API.
- The newly created engine will have new API tokens which will need to be updated in all clients.
Step 1. Create the engine to be migrated in App Search on Elastic Cloud.
Step 2. Migrate the Schema by using the Schema API to read settings from the Swiftype.com engine and write those same settings to the Elastic Cloud engine. Learn more about working with the Schema API.
Step 3. Backfill documents to the new engine in App Search on Elastic Cloud from the canonical data store.
Step 4. Migrate search settings, curations, and synonyms data by using the various settings APIs to read settings from the Swiftype.com engine and write those same settings to the Elastic Cloud engine.
Step 6. Update ingestion clients in production with additional API keys and endpoints to send writes to both engines: the original engine in App Search on Swiftype.com and the new engine in App Search on Elastic Cloud.
Step 7. Once there is parity between the two engines, change all client API keys and endpoints in production to send queries and writes to the new engine on Elastic Cloud.
Step 8. Export analytics from App Search on Swiftype.com as a backup. Learn more about the Analytics API.
Step 9. Validate that the original engine is no longer active by making sure there are no new events in the API logs.
You should verify that engines on Swiftype.com are not receiving queries and writes before decommissioning your Swiftype.com account.
Once you’ve migrated all your engines to Elastic Cloud, you may want to consider decommissioning your App Search account on Swiftype.com.
If your account is billed monthly, you can close your account from within the App Search dashboard. Visit your account settings to proceed.
If your account is billed annually, please contact your account representative. Visit Elastic support.