Upload your own content

Bring your proprietary knowledge into BigData (reports, research, internal docs, and more) and make it instantly searchable and actionable through Search and Research Agent services. Your content stays private and powers answers, summaries, and analysis.

Private & Secure: No LLM training on your data

Upload and manage content via the Content API. All endpoints require an API key in the X-API-KEY header. See Authentication for details. For full request/response schemas, see the Content API reference.

For async ingestion (e.g. email inbox, SharePoint), see Connectors in the Content API introduction.

Upload a file

Direct upload has two required steps; a third step is optional if you want to track enrichment status. The workflow uploads your file and enriches it (extraction, structure and annotation of the content). By default it also indexes the document so it is available in Search and Research Agent. You can skip indexing or long-term storage with the optional no_index and no_store flags. See Processing options.

Request an upload URL

POST to /contents/v1/documents with JSON metadata. The response contains a single-use pre-signed url and the document id.

curl -X POST 'https://api.bigdata.com/contents/v1/documents' \
  -H 'X-API-KEY: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "file_name": "research_report.pdf",
    "published_ts": "2025-06-15T10:30:00Z",
    "tags": ["Research Team"],
    "share_with_org": true,
    "enrichments": ["reporting_entities", "translation"]
  }'

Include an optional enrichments array to configure the processing pipeline to apply extra enrichment steps to the document. See Enrichments below. Optionally set no_index or no_store if you want enrichment without Search indexing or without long-term retention. See Processing options.Example response:

{
  "url": "https://s3.amazonaws.com/.../uploads/F22BC027BCE166BC89DD2A81358DA2F1?AWSAccessKeyId=...",
  "id": "F22BC027BCE166BC89DD2A81358DA2F1"
}

Upload the file to the pre-signed URL

PUT the file to the url from the previous response. This uploads the document; Bigdata then enriches it (extraction, structure and annotation of the content). Unless you set no_index or no_store, it is also indexed for Search and Research Agent.

curl -X PUT 'URL_FROM_RESPONSE' \
  --data-binary '@/path/to/research_report.pdf'

(Optional) Poll to track enrichment status

If you want to track when the document is ready, use the id from step 1 with Get document metadata to poll status. While enriching, you will see "status": "processing". When enrichment is complete (and indexing, unless you skipped it), the status becomes "status": "completed". This step is not required; Bigdata processes the document in the background after you upload it.

# Use the id from the POST response (e.g. F22BC027BCE166BC89DD2A81358DA2F1)
curl -X GET 'https://api.bigdata.com/contents/v1/documents/F22BC027BCE166BC89DD2A81358DA2F1' \
  -H 'X-API-KEY: YOUR_API_KEY'

Congratulations! Your file has been uploaded and enriched. With the default settings it is also indexed and available for Search and Research Agent. Try the how-to guide Upload and search your content to see it in action.

Enrichments

Include an optional enrichments array in the upload POST body to configure the processing pipeline to apply additional enrichment steps to your document. Supported enrichments:

Enrichment	Description
`reporting_entities`	Identifies the document’s reporting company — the entity a transcript or filing is reported by — consistent with the `ReportingEntity` reporting detail used in Search, so you can reliably filter transcripts and filings by reporting company.
`translation`	Translates the document content so non-English material can be searched and analyzed alongside the rest of your corpus. Supports more than 70 languages.

You can request one or more enrichments in the same request:

{
  "file_name": "Q2_2025_equity_research_note.pdf",
  "published_ts": "2025-06-15T10:30:00Z",
  "tags": ["equity-research", "sector:semiconductors"],
  "share_with_org": false,
  "enrichments": ["reporting_entities", "translation"]
}

Enrichments can also be configured on a connector so they are applied to all content ingested through it. See Create connector.

Processing options

By default, an uploaded document is enriched, stored, and indexed so it can be retrieved later and used in Search and Research Agent. On POST /contents/v1/documents you can set these optional flags:

Flag	Effect
`no_index`	The document is enriched but not indexed. Original and annotated files stay stored and retrievable via the Content API, but the document does not appear in Search or Research Agent. Use this when you need the annotated output without adding the file to your searchable corpus.
`no_store`	The document is enriched but not indexed, and is only retrievable for 24 hours after enrichment completes. After that it is deleted. Use this for single-use processing when you do not want the file retained.

Both flags can be true in the same request. no_store already skips indexing, so no_index has no additional effect in that case.

{
  "file_name": "one_off_report.pdf",
  "published_ts": "2025-06-15T10:30:00Z",
  "share_with_org": false,
  "no_index": true
}

{
  "file_name": "ephemeral_memo.pdf",
  "published_ts": "2025-06-15T10:30:00Z",
  "share_with_org": false,
  "no_store": true
}

If you set no_store, download the annotated or original file while the document status is completed. After 24 hours the document is deleted and those endpoints return not found.

Working with your files

List documents: GET /contents/v1/documents returns a paginated list of documents you can access. Use query parameters for filtering (origin, tags, connector, from_date, file_name, etc.), sorting (sort_by, sort_order), and pagination (page, page_size). Example: list first page of directly uploaded documents, filter by tag

curl -X GET 'https://api.bigdata.com/contents/v1/documents?origin=file_upload&page=1&page_size=10&tags=test' \
  -H 'X-API-KEY: YOUR_API_KEY'

Example response (excerpt)

{
  "results": [
    {
      "id": "7FA511999C3984CB75005890B15A7096",
      "file_name": "research_report.pdf",
      "user_id": "user_id_001",
      "org_id": "org_id_001",
      "rp_collection_id": null,
      "rp_source_id": "E5A77C",
      "raw_size": 3832243,
      "pages": 50,
      "request_origin": "file_upload",
      "content_type": "application/pdf",
      "status": "completed",
      "shared_with_orgs": [
        "org_id_001"
      ],
      "created_at": "2026-08-10T14:21:40.333178Z",
      "updated_at": "2026-08-10T14:21:40.333181Z",
      "published_at": "2026-08-10T12:00:00Z",
      "connector_id": "019dbfd9-f1ea-7024-888d-e52af8c87b03",
      "error_code": null,
      "delete_started_ts": null,
      "file_metadata": null,
      "tags": [
        {
          "id": "019a48b4-e574-71d9-a8d9-6a5a86386847",
          "name": "Research Team"
        }
      ],
      "usage": {
        "private_data_tokens": 171079
      },
      "enrichments_requested": [
        "translation"
      ],
      "enrichments_applied": []
    }
  ]
}

Get document metadata: GET /contents/v1/documents/ returns metadata for one document by its content ID: status, file name, connector, tags, enrichments, usage, and timestamps. Use this to check processing status before downloading content. Example: get metadata for one document

curl -X GET 'https://api.bigdata.com/contents/v1/documents/E0618D0E9D3A960C1731A620EDE56B5C' \
  -H 'X-API-KEY: YOUR_API_KEY'

Private content only: The annotated and original download paths below apply only to documents you ingested via the Content API (this guide). They do not work for document id values returned by the Search API for RavenPack content; for those, use Fetch document (GET /v1/documents/{document_id}). Download the original file: GET /contents/v1/documents//original returns a time-limited pre-signed URL. GET that URL to download the original file in its native format (e.g. .pdf, .docx). Example: get download URL for original file, then download

# 1. Get the pre-signed URL
curl -X GET 'https://api.bigdata.com/contents/v1/documents/E0618D0E9D3A960C1731A620EDE56B5C/original' \
  -H 'X-API-KEY: YOUR_API_KEY'

# 2. Response contains a URL; GET it to download the file (e.g. save to disk)
# curl -o research_report.pdf 'PRE_SIGNED_URL_FROM_RESPONSE'

Download the annotated version: GET /contents/v1/documents//annotated returns a time-limited pre-signed URL. GET that URL to download the document as structured JSON (metadata, title, body blocks, entities, sentences with sentiment). Useful for search indexing, entity extraction, or structured display. Example: get download URL for annotated JSON

curl -X GET 'https://api.bigdata.com/contents/v1/documents/E0618D0E9D3A960C1731A620EDE56B5C/annotated' \
  -H 'X-API-KEY: YOUR_API_KEY'
# Then GET the returned URL to retrieve the JSON file.

Deleting a document

To permanently remove a document from the platform, call DELETE /contents/v1/documents/. This deletes the original file, the annotated version, and the chunks from the vector database. Documents uploaded with no_store are deleted automatically 24 hours after enrichment completes; you do not need to call DELETE for those unless you want them removed sooner. Only documents in COMPLETED or FAILED status can be deleted. Attempting to delete a document that is still being enriched will raise an exception. Use Get document metadata to check status before deleting.

Success: 200 response with a null body.
Not found: 404 with body {"statusCode": 404, "message": "Document not found", "errorCode": "PRIVATE_CONTENT_NOT_FOUND", "requestId": "..."}.

Deleting a document is permanent and cannot be undone.

Next steps

Try the how-to guide — Upload and search your content to see your uploaded content in action.
Try the Batch file upload — Batch file upload script for uploading many files.
Try Playgrounds — Use the Search Service or Research Agent playgrounds below; select My Files to query your documents.

Search Service Playground

Search across your private content and other sources. In the playground, open the source selector and choose My Files to limit results to your uploaded documents.

Research Agent Playground

Run research over your private content and real-time data. In the playground, use the source selector and filter by My Files to ground answers in your documents.

Related documentation

Content API introduction: Connectors vs direct upload, and document operations.
Search API filters: Use the tag and category filter to restrict queries to your uploaded files only. Or try it with the Search API reference playground
Research Agent search tool filters: Restrict context to your proprietary content with content filters and tags.

Getting Started

Research Service

Search Service

Monitors

Upload proprietary content

Knowledge Graph

Watchlist

Troubleshooting

Upload a file

Tags

Enrichments

Processing options

Working with your files

Deleting a document

Next steps

Search Service Playground

Research Agent Playground

​Upload a file

​Tags

​Enrichments

​Processing options

​Working with your files

​Deleting a document

​Next steps

Search Service Playground

Research Agent Playground

Upload a file

Tags

Enrichments

Processing options

Working with your files

Deleting a document

Next steps