Skip to main content

​
Overview

This guide demonstrates how to use the Bigdata Search Volume API endpoint (/v1/search/volume) to retrieve and visualize theme volume data over time. The script displays the evolution of documents, chunks, and sentiment for any given theme, helping you track narrative trends and coverage patterns.

GitHub Repository

Access the complete source code, requirements, and setup instructions.

​
Features

✅ Retrieves theme volume data from the Bigdata Search Volume API
✅ Visualizes three key metrics: number of documents, number of chunks, and sentiment
✅ Displays daily values as bars (documents and chunks) or lines (sentiment)
✅ Overlays weekly average trends for better pattern recognition
✅ Generates high-resolution PNG charts with theme-specific filenames
✅ Supports custom date ranges and themes via command-line arguments

​
Use Cases

The Search Volume endpoint is valuable for several scenarios:
  • Evaluate Coverage: See how many unique documents Bigdata has available for your particular query
  • Narrative/Thematic Screeners: Create a query and see how the defined narrative evolves over time
  • Query Strategy: Check coverage and plan how to structure your queries accordingly
Efficient Usage: The /v1/search/volume endpoint consumes exactly 1 API Query Unit regardless of the time period queried (up to 60 weeks maximum), making it a very efficient way to visualize narrative evolution over time.
Flexible Filtering: The Search Volume endpoint supports all the same filters as the Search endpoint, including entity, source, category, keyword, and sentiment filters. For more details, check out the Search Volume API specification.

​
Prerequisites

  • Python 3.9 or higher
  • A Bigdata.com API key

​
System Dependencies

Matplotlib requires some system libraries for chart generation. 
sudo apt-get install python3-tk

​
Setup

1

Clone the repository

git clone https://github.com/Bigdata-com/bigdata-docs-resources.git
cd bigdata-docs-resources/how_to_guides/api_theme_volume
2

Create a virtual environment (recommended)

python -m venv venv
Activate the virtual environment:
source venv/bin/activate
3

Install Python dependencies

pip install -r requirements.txt
4

Configure API Key

Copy the .env_template file to .env:
cp .env_template .env
Open the .env file and replace the placeholder value with your actual API key:
BIGDATA_API_KEY=your_bigdata_api_key_here
You can obtain your API key from the Bigdata Platform.
Never commit your .env file to version control. Add it to your .gitignore file.

​
Usage

​
Run the script

Run the script with required date range parameters: 
python theme_volume_evolution.py --start-date 2025-01-01 --end-date 2025-12-15

​
Command-Line Arguments

ArgumentShortDescriptionRequired
--start-date-sStart date in format YYYY-MM-DD or YYYY-MM-DDTHH:MM:SSZYes
--end-date-eEnd date in format YYYY-MM-DD or YYYY-MM-DDTHH:MM:SSZYes
--theme-tTheme to search for (default: “Tariffs impact”)No

​
Examples

Basic usage with default theme: 
python theme_volume_evolution.py --start-date 2025-01-01 --end-date 2025-12-15
Using short flags: 
python theme_volume_evolution.py -s 2025-01-01 -e 2025-12-15
With a custom theme: 
python theme_volume_evolution.py -s 2025-01-01 -e 2025-12-15 --theme "Quantum computing chip"
View help: 
python theme_volume_evolution.py --help

​
Output

The script generates a high-resolution (300 DPI) PNG chart with a timestamped filename: 
{theme}_volume_evolution_YYYYMMDD_HHMMSS.png
The chart includes three subplots:
  1. Unique Documents per Day: Daily document counts as bars with weekly average line
  2. Chunks per Day: Daily chunk counts as bars with weekly average line
  3. Sentiment per Day: Daily sentiment values as a line with weekly average line

​
Example Charts

Here are example charts generated for different themes:
Theme: "Tariffs impact"Tariffs Impact Volume Evolution