« The Searcher’s Guide to Workplace Search Custom sources API »

API reference

Connect to an unlimited number of private content sources using the Workplace Search APIs.

This overview will discuss:

Authentication

Each API call requires an access_token and a content_source_key.

The access_token is your private API key.

You’ll need it to authenticate each request. Keep it secret!

The content_source_key identifies which content source you’d like to reach with your request.

curl -X POST http://localhost:3002/api/ws/v1/sources/[CONTENT_SOURCE_KEY]/documents/bulk_create \
-H "Authorization: Bearer [ACCESS_TOKEN]" \
-H "Content-Type: application/json" \
-d '
  ...
'

Create a new Create a Custom API Source or view the details of an existing Custom API Source to find these values.

The access_token is shared amongst all Connectors. But the content_source_token is unique to each Connector.

Schema design

You can have up to 64 schema fields.

There are four field types: text, number, date, geolocation.

Data that matches a defined schema helps Workplace Search interpret and style the information. Objects can take any shape, but there are some parameters within which you need to work.

You can either:

Ingest a document to automatically create default fields of type text, then alter the type.
Build a schema in advance.

Read the Custom API sources guide for a direct walkthrough of the process.

The Five Keys of Schema Design

Always index new fields as the same type as existing documents.
- eg. An existing date field should not receive geolocation data.
Arrays are supported, but nested objects are not supported. You must flatten them.
Fields cannot be deleted once they have been created.
The reserved fields are: external_id, source, content_source_id, updated_at, last_updated, highlight, any, all, none, or, and, not, engine_id, _allow_permissions and _deny_permissions.
A field name can only contain lowercase letters, numbers, and underscores.

Field types

Workplace Search fields can be one of four different types:

Text fields

Text fields are at the heart of search.

They enable deeply analyzed full text search.

This is the default type for all new fields.

Any group of characters or text that you want to search over should be text.

eg. A description of an object, the name of a product, the content of a review.

Number fields

A finite double-precision floating point value: 3.14 or 42. Number fields enable fine grained sorting, filtering, faceting, and boosting.

eg. A price, a review score, the number of visitors, or a size.

Date fields

Dates must be in ISO 8601 format, eg: "2013-02-27T18:09:19Z" or "2013-02-27T17:09:19+01:00".

eg. A product release or publish date, birth date, an air date.

Geolocation fields

Geographic coordinates can leverage the location field. The location field allows filtering by distance from a specified point.

A location is specified using a JSON object containing the longitude and latitude, such as:"37.7894758, -122.3940638". The separating space after the comma may be omitted: "37.7894758,-122.3940638".

eg. A store where a product is located, location of a venue.

IDs

Each document within a content source must have a unique id. If you do not provide an id, a BSON id will be created for you. Two documents in two separate content sources may have the same id.

You can update existing documents by issuing a POST request to an existing id.

If the id does not exist, a new document is created. It is up to you to maintain the integrity of your id for each document within each Custom API Source.

We recommend that you avoid SHAs or any id derived from the content of a document.

Modification of original data will alter value, making it difficult to identify the document in the search index. This can lead to record duplication.

Document level permissions

There are two reserved fields _allow_permissions and _deny_permissions. Both are arrays which can hold arbitrary text values. The values match up with user permissions to provide granular document level access control.

Read more within the Document permissions for Custom Sources.