Skip to main content
Bring your proprietary knowledge into BigData (reports, research, internal docs, and more) and make it instantly searchable and actionable through Search and Research Agent services. Your content stays private and powers answers, summaries, and analysis.
Private & Secure: No LLM training on your data
Upload and manage content via the Content API. All endpoints require an API key in the X-API-KEY header. See Authentication for details. For full request/response schemas, see the Content API reference.
For async ingestion (e.g. email inbox, SharePoint), see Connectors in the Content API introduction.

​
Upload a file

Direct upload has two required steps; a third step is optional if you want to track processing status.
1

Request an upload URL

POST to /contents/v1/documents with JSON metadata. The response contains a single-use pre-signed url and the document id.
curl -X POST 'https://api.bigdata.com/contents/v1/documents' \
  -H 'X-API-KEY: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "file_name": "research_report.pdf",
    "published_ts": "2025-06-15T10:30:00Z",
    "tags": ["Research Team"],
    "share_with_org": true
  }'
Example response:
{
  "url": "https://s3.amazonaws.com/.../uploads/F22BC027BCE166BC89DD2A81358DA2F1?AWSAccessKeyId=...",
  "id": "F22BC027BCE166BC89DD2A81358DA2F1"
}
2

Upload the file to the pre-signed URL

PUT the file to the url from the previous response (use the correct Content-Type for your file). This completes the upload; Bigdata then starts processing the document.
curl -X PUT 'URL_FROM_RESPONSE' \
  --data-binary '@/path/to/research_report.pdf'
3

(Optional) Poll to track processing status

If you want to track when the document is ready, use the id from step 1 with Get document to poll status. While processing, you will see "status": "processing". When the document is ready for search and download, the status becomes "status": "completed". This step is not required—Bigdata processes the document in the background after the upload.
# Use the id from the POST response (e.g. F22BC027BCE166BC89DD2A81358DA2F1)
curl -X GET 'https://api.bigdata.com/contents/v1/documents/F22BC027BCE166BC89DD2A81358DA2F1' \
  -H 'X-API-KEY: YOUR_API_KEY'
Congratulations! Your file is indexed and available for the Research Agent and Search service. Try the how-to guide Upload and search your content to see it in action.

​
Tags

At upload time: Include a tags array in the POST body to apply tag names to the document (e.g. ["Research Team", "Q4 2025"]). These tags can be used to filter documents in the Search and Research Agent services. List tags: Use GET /contents/v1/tags to retrieve all tags visible to your organization, with document counts. Tags are returned with id, name, and file_count. Filter documents by tags: When calling List documents, use the tags query parameter (repeat for multiple tags). Documents matching any of the given tags are returned (OR logic). Use tags in Search and Research Agent: You can scope search and research to documents with specific tags. In the Search API, set query.filters.tag in the request body. In the Research Agent API, configure the search tool’s query_filters.content with the desired tag filter.

​
Working with your files

List documents: GET /contents/v1/documents returns a paginated list of documents you can access. Use query parameters for filtering (origin, tags, connector, from_date, file_name, etc.), sorting (sort_by, sort_order), and pagination (page, page_size). To list only documents uploaded via direct upload, use origin=file_upload. Each item includes id (content_id), file_name, status, connector_id, tags, request_origin, and timestamps. Example: list first page of directly uploaded documents, filter by tag 
curl -X GET 'https://api.bigdata.com/contents/v1/documents?origin=file_upload&page=1&page_size=10&tags=test' \
  -H 'X-API-KEY: YOUR_API_KEY'
Example response (excerpt) 
{
  "results": [
    {
      "id": "7FA511999C3984CB75005890B15A7096",
      "file_name": "research_report.pdf",
      "user_id": "user_id_001",
      "org_id": "org_id_001",
      "rp_collection_id": null,
      "raw_size": 654,
      "request_origin": "file_upload",
      "content_type": "application/pdf",
      "status": "completed",
      "connector_id": null,
      "error_code": null,
      "delete_started_ts": null,
      "file_metadata": null,
      "tags": [
        {
          "id": "019a48b4-e574-71d9-a8d9-6a5a86386847",
          "name": "Research Team"
        }
      ]
    }
  ]
}
Get document metadata: GET /contents/v1/documents/ returns metadata for one document by its content ID: status, file name, connector, tags, timestamps. Use this to check processing status before downloading content. Example: get metadata for one document 
curl -X GET 'https://api.bigdata.com/contents/v1/documents/E0618D0E9D3A960C1731A620EDE56B5C' \
  -H 'X-API-KEY: YOUR_API_KEY'
Download the original file: GET /contents/v1/documents//original returns a time-limited pre-signed URL. GET that URL to download the original file in its native format (e.g. .pdf, .docx). Example: get download URL for original file, then download 
# 1. Get the pre-signed URL
curl -X GET 'https://api.bigdata.com/contents/v1/documents/E0618D0E9D3A960C1731A620EDE56B5C/original' \
  -H 'X-API-KEY: YOUR_API_KEY'

# 2. Response contains a URL; GET it to download the file (e.g. save to disk)
# curl -o research_report.pdf 'PRE_SIGNED_URL_FROM_RESPONSE'
Download the annotated version: GET /contents/v1/documents//annotated returns a time-limited pre-signed URL. GET that URL to download the document as structured JSON (metadata, title, body blocks, entities, sentences with sentiment). Useful for search indexing, entity extraction, or structured display. Example: get download URL for annotated JSON 
curl -X GET 'https://api.bigdata.com/contents/v1/documents/E0618D0E9D3A960C1731A620EDE56B5C/annotated' \
  -H 'X-API-KEY: YOUR_API_KEY'
# Then GET the returned URL to retrieve the JSON file.

​
Deleting a document

To permanently remove a document from the platform, call DELETE /contents/v1/documents/. This deletes the original file, the annotated version, and the chunks from the vector database. Only documents in COMPLETED or FAILED status can be deleted. Attempting to delete a document that is still being processed will raise an exception. Use Get document to check status before deleting.
  • Success: 200 response with a null body.
  • Not found: 404 with body {"statusCode": 404, "message": "Document not found", "errorCode": "PRIVATE_CONTENT_NOT_FOUND", "requestId": "..."}.
Deleting a document is permanent and cannot be undone.

​
Next steps

  • Try the how-to guide — Upload and search your content to see your uploaded content in action.
  • Try the Batch file upload — Batch file upload script for uploading many files.
  • Try Playgrounds — Use the Search Service or Research Agent playgrounds below; select My Files to query your documents.

Search Service Playground

Search across your private content and other sources. In the playground, open the source selector and choose My Files to limit results to your uploaded documents.

Research Agent Playground

Run research over your private content and real-time data. In the playground, use the source selector and filter by My Files to ground answers in your documents.
Related documentation