Comparative Evaluations

​

Why evaluate?

When financial institutions adopt agentic platforms for research automation, report quality is not optional. Accuracy, source provenance, analytical depth, and professional presentation must meet institutional standards across every report. To measure how well Bigdata MCP tools support this bar, we run structured comparative evaluations. Each evaluation takes the same prompt and generates two reports side by side: one grounded in Bigdata MCP tools (with access to SEC filings, earnings transcripts, analyst notes, and structured financial data) and one using general web search. Both reports are then scored across eight dimensions on a 1-10 scale. The goal is not to declare a winner in every category, but to surface where premium data access materially improves output quality and where it does not, giving teams a clear picture of the value MCP-grounded workflows deliver at scale.

---

​

Comparative Evaluation example

​

Figma Q4 2025 Earnings Digest

On February 21, 2026, we evaluated the following reports:

• Claude + Web Search: Generated using Claude and the general web search tool
• Claude + Bigdata MCP: Generated using Claude, Bigdata MCP tools and Bigdata skill workflow

Methodology: Each report is scored across 8 dimensions on a 1-10 scale (10 = best). Scores are based on factual accuracy verified against primary sources (Figma press release, SEC filings, CNBC, Yahoo Finance, Fortune, Motley Fool transcript), completeness, analytical depth, and professional presentation.

---

​

1. Factual Accuracy of Financial Data

Score, Web Search: 7.5 — Core revenue, customer, and guidance figures are accurate. The EPS consensus comparison is the most material issue: presenting a GAAP loss estimate of $(0.20) against adjusted EPS of $0.08 to claim a “$0.28 beat” conflates GAAP and non-GAAP measures, which is misleading in a professional digest. The FCF discrepancy is minor. Score, Bigdata MCP: 9.0 — Highly accurate across all key metrics. Minor nit: the “$289M net cash” balance sheet figure and some derived ratios (current ratio 2.58x) are not independently verified but appear reasonable.

---

​

2. Source Quality and Attribution

Score, Web Search: 7.5 — Solid sourcing from reputable business media. However, reliance on aggregator sites for analyst data (GuruFocus, Meyka, Stock Observer) introduces a layer of indirection. No SEC filing, 10-K, or full transcript directly cited. All links are publicly accessible, which is a practical advantage. Score, Bigdata MCP: 9.5 — Outstanding source quality. Direct access to SEC filings, full earnings transcripts, investor presentations, and individual analyst research notes is the gold standard for an earnings digest. The only drawback is that several links require Bigdata.com platform access.

---

​

3. Completeness and Coverage

Score, Web Search: 8.5 — Also very comprehensive, with some unique details: GAAP gross margin compression trend, deferred revenue, Weavy acquisition specifics, the 60% non-designer stat for Make, and the Barclays upgrade. Customer KPI table is less detailed (missing YoY for two major cohorts). Includes a proper disclaimer. Score, Bigdata MCP: 9.0 — Exceptionally comprehensive. The customer KPI table with full year-over-year comparisons is a standout. The analyst table with 7 firms, specific analysts, and old/new price targets is highly professional. Minor gap: no disclaimer.

---

​

4. Analytical Depth and Insight

Score, Web Search: 9.0 — Slightly stronger analytical depth overall, particularly on margin trends (GAAP gross margin erosion), FCF sequential deterioration, SBC quantification, and valuation context. The 60% non-designer adoption stat is a strong insight about TAM expansion. Score, Bigdata MCP: 8.5 — Strong analytical framework with good insight on the international revenue/MAU gap and guidance conservatism. Could use more on valuation and FCF trajectory.

---

​

5. Structure and Readability

Both reports follow nearly identical section structures, which is sensible for an earnings digest format. Score, Web Search: 9.0 — Clean, professional presentation from the first line. No artifacts or processing notes. Score, Bigdata MCP: 8.0 — Excellent structure, but the visible chain-of-thought processing notes at the very top of the file is unprofessional and should have been removed before final output.

---

​

6. Analyst Coverage Quality

Score, Web Search: 7.5 — Good coverage, and includes the Barclays upgrade which is unique. However, the consensus price target range of “$41-$48” appears inconsistent with the individual targets listed (most are $30-$44), which undermines credibility. Fewer named analysts. Score, Bigdata MCP: 9.0 — More detailed analyst table with named analysts and more firms. The consensus summary ($35 median, $30-$44 range) aligns well with the individual data points shown.

---

​

7. Timeliness and Recency

Both reports are dated February 21, 2026 (3 days post-earnings on Feb 18). Score, Web Search: 9.5 Score, Bigdata MCP: 9.5

---

​

8. Professional Polish and Trust Signals

Score, Web Search: 9.0 — Cleaner presentation with a proper disclaimer. The EPS consensus issue is the main trust concern. Score, Bigdata MCP: 8.0 — The lack of a disclaimer and the visible processing notes detract from what is otherwise a highly professional output.

---

​

Summary Scorecard

---

​

Key Takeaways

Web Search excels in analytical narrative, clean presentation, and several unique data points (GAAP gross margin trajectory, deferred revenue, Weavy acquisition details, SBC quantification, valuation multiples). It reads as a more polished final product. Its main weaknesses are the misleading EPS consensus comparison (GAAP vs. non-GAAP conflation), an inconsistent consensus price target range, and reliance on aggregator sites rather than primary sources for analyst data. Bigdata MCP excels in data precision, source quality, and analyst coverage depth. Its access to primary sources (SEC filings, full transcripts, investor slides, and individual broker research notes) gives it a clear edge in factual rigor and traceability. The analyst table with 7 firms and named analysts is particularly strong. Its main weaknesses are presentational: visible processing artifacts on line 1 and a missing disclaimer. (Comming soon) Bottom line: The Bigdata MCP report is the stronger choice for institutional or professional use where data accuracy and source provenance are paramount. The Web Search report is the stronger choice for a polished, publicly shareable summary with richer narrative context, but the EPS error would need to be corrected first.