Introduction

​

Overview

The Bigdata Content API lets you manage and query private content that you or your organization uploads to Bigdata, making it available for the Search and Research Agent after enrichment and indexing. You can onboard content in two ways:

• Connectors API: For asynchronous, unsupervised, or automated workflows where content is not directly accessible by the user (e.g. email inbox, SharePoint). You configure a connector and the service ingests the content from that source.
• Direct upload (POST /documents): For clients that manage their own corpus and want to build the ingestion workflow manually. You request a pre-signed URL and document id; you upload each file via PUT to that URL, then use the id to poll for enrichment status. The workflow uploads your document, enriches it (extraction, structure and annotation of the content), and indexes it for availability in Search and Research Agent.

Both paths produce documents that you can list, tag, filter, and use with the Search and Research Agent Services. You can create and manage Connectors in the Developer Platform > Connectors.

---

​

Connectors

Connectors are for ingestion that is asynchronous, unsupervised, or part of a workflow where content is not directly in the user’s hands, for example, an email inbox or a SharePoint library.

• You create a connector with a type, label, and type-specific configuration. Supported types include Email Inbox and (coming soon) Microsoft SharePoint.
  • For Email, the API returns an inbox address; you forward messages from allowed senders to that address. Set up automatic forwarding rules in your email client to start building your corpus, Bigdata will ingest and process content automatically.
• Content arriving through the connector is turned into documents you can manage and use via Search and Research Agent Services.

Typical operations:

• Create connector: Register a new ingestion source. For email connectors, the response includes a connector ID and an inbox address; forward messages from allowed senders to that address to ingest content into Bigdata.
• List connectors: List all connectors you can access; use connector IDs to filter documents or to update/delete.
• Get connector: Fetch full details for one connector (config, share settings, timestamps).
• Update connector: Change label, description, sharing, or configuration of the connector.
• Delete connector: Soft-delete (archive) a connector by default—it stops ingesting new content but existing documents remain. Use ?force_delete=true to trigger a hard delete: all files related to the connector are deleted asynchronously, then the connector is removed from the database.

Once a connector is set up, documents are ingested according to its configuration.

---

​

Documents

Documents are the items Bigdata has uploaded, enriched, and indexed for search and analysis, whether they came from a connector (e.g. email, SharePoint) or from direct upload. Direct upload is intended for clients that manage their corpus and want to build the ingestion workflow from scratch: you choose what to send, when, and with what metadata (file name, tags, sharing). The main focus is enrichment: once you upload the file (PUT to the pre-signed URL), Bigdata enriches it (extraction, structure and annotation of the content), then indexes it for availability in Search and Research Agent. Use Enrich document to get a pre-signed URL and document id; PUT the file to that URL, then use the id with Get document metadata to poll for status until enrichment and indexing are complete. The following operations apply to all documents, whether they were ingested via a connector or direct upload:

• List documents: Paginated list of documents you can access. Supports filtering and sorting; each item includes a content ID for use with the other document endpoints.
• Get document metadata: Return metadata for one document by content ID (status, file name, connector, tags, timestamps). Use this to check enrichment status before downloading content.
• Get annotated document: Return a time-limited pre-signed URL. GET that URL to download private document content as structured JSON (metadata, title, body blocks, entities, sentences with sentiment). Use for search indexing, entity extraction, or structured display.
• Get original document: Return a time-limited pre-signed URL. GET that URL to download the private document’s original file in its native format (e.g. .eml, .pdf).
• Delete document: Remove a document from the platform. Deletes the original document, annotated version, and chunks from the vector database.

Documents can be tagged (e.g. by sender/recipient for emails). Use List tags to discover tag names and List documents with the tags parameter to filter by them.

---

​

Typical flow

Using connectors (async / unsupervised):

1. Create a connector (e.g. email) with type, label, and config; for email, use the returned inbox address to forward messages.
2. Optionally check that content has been enriched and indexed by listing your recent documents.
3. Use your content in the Search or Research Agent Service (e.g. filter by Emails in the playgrounds below).

Using direct upload (manual corpus):

1. Call Enrich document with metadata (file name, tags, sharing); use the returned URL to PUT the file (upload it).
2. Poll Get document metadata with the returned id to check status until enrichment and indexing are complete (completed). See Get document metadata in the API reference.
3. Use your content in the Search or Research Agent Service (e.g. filter by My Files in the playgrounds below).

---

​

Authentication and reference

All endpoints require an API key sent in the X-API-KEY header. See Authentication for details. Use the API reference (endpoints listed in this section) for request/response schemas, parameters, and examples. You can try the API from the Developer Platform playgrounds.