Tickmark Bench
Menu
Get the sample

FinanceBench, FinQA, TAT-QA, FinRank, Finance Agent, and Tickmark Bench.

Five financial question answering benchmarks, and where Tickmark Bench differs. Figures for the other benchmarks come from their own papers and pages, linked in each section.

FinanceBench

FinanceBench is from Patronus AI. Its paper describes 10,231 questions, answers, and evidence triplets about 40 publicly traded US companies, drawn from 361 public filings released between 2015 and 2023: 10-Ks, 10-Qs, 8-Ks, and earnings reports. Each entry has the question, the answer, an evidence string, and a page number. An open sample of 150 cases is on GitHub, and Patronus AI says the full set is available to license. The authors reviewed 2,400 answers from the systems they tested by hand. Source: arXiv 2311.11944 and github.com/patronus-ai/financebench.

FinQA

FinQA was presented at EMNLP 2021. It has 8,281 question and answer pairs written by finance experts over the earnings reports of companies in the S&P 500, from 1999 to 2019, with a gold reasoning program for each answer. Source: ACL Anthology 2021.emnlp-main.300 and github.com/czyssrs/FinQA.

TAT-QA

TAT-QA was presented at ACL 2021. It has 16,552 questions over 2,757 hybrid contexts from 182 financial reports. Each context is a table with at least two paragraphs that describe it, and answers come with their derivation and scale where needed. The paper says the dataset is available for non-commercial use. Source: ACL Anthology 2021.acl-long.254.

FinRank

FinRank was posted to arXiv in August 2026. It has 1,185 question and answer records written by hand over the 10-K and 10-Q filings of 22 different companies in pharmaceuticals, oil and gas, and automotive, from filing years 2024 and 2025. Each record has a reference answer, gold supporting passages, and hard negatives chosen by hand from passages that are easy to confuse with the right one. FinRank measures passage retrieval, reranking, and hard-negative discrimination as separate tasks. The paper says the dataset and evaluation code are on GitHub. Source: arXiv 2608.07400 and github.com/datanxt/FinRank.

Finance Agent Benchmark

Finance Agent Benchmark is from Vals AI. The first paper describes 537 questions written by finance experts in nine task categories, focused on documents published in 2024 or later, and a harness that gives the agent tools such as Google search and EDGAR access. Source: arXiv 2508.00828.

Finance Agent v2 has 927 expert-reviewed questions: 27 public samples, 450 private validation questions available to license, and 450 test questions that stay private. The agent gets six tools, including EDGAR search, web search, a calculator, and price history. Each question has weighted checks, and answers are graded by a jury of three LLM judges. Source: vals.ai/benchmarks/fabv2 and github.com/vals-ai/finance-agent-v2.

Tickmark Bench Core

Core is the real_v1 verified batch: 149 tasks from 67 companies, with filing dates from 2026-04-07 to 2026-09-29. Every reference value is taken from, or computed from, the filed XBRL facts, and each task lists those facts with the accession, the concept, the period, and the unit. Core is scored entirely by grade.py. Numeric answers are scored within the stated tolerance, and claim checks on the verdict alone. Tasks ship in the Prime Intellect verifiers format. Core is $299 one-time, and a 20-task free sample is public.

Where they differ

  • Filing dates. FinanceBench covers filings from 2015 to 2023, and FinQA covers reports from 1999 to 2019. Core covers filings dated 2026-04-07 to 2026-09-29.
  • Size. FinanceBench, FinQA, TAT-QA, FinRank, and Finance Agent v2 are all larger in total. Core has 149 tasks.
  • Evidence. FinanceBench points to an evidence string and a page. FinQA and TAT-QA give the table and text around the answer. FinRank scores whether a system finds the gold passages. Core points to tagged XBRL facts in the filing.
  • Tools. Finance Agent gives the agent EDGAR search and web search and asks open research questions. Core gives the filing identifiers in the question and asks for one number or one verdict.
  • Scoring. Finance Agent v2 answers are graded by a jury of LLM judges. Core answers are checked in code against a tolerance or a yes or no verdict, with no hand review in the loop.
  • Access. FinQA, the FinanceBench sample, FinRank, and the Finance Agent v2 public set are on GitHub. Core is sold, and the license is on the Terms page.

Which to use

For a large free set of questions about older reports, FinQA, TAT-QA, and the FinanceBench sample are good places to start. For retrieval over filings, FinRank measures it directly. For open research with tools, Finance Agent is built for it. For tasks from 2026 filings with a code grader in the verifiers format, start with the free sample, then read how Core is graded.

Tickmark Bench is not affiliated with Patronus AI, Vals AI, or the authors of FinQA, TAT-QA, or FinRank.