A benchmark built from SEC filings.
Question answering tasks on 10-Q and 10-K filings dated Q2 2026 or later. Every Core reference traces to cells in the filing, and grade.py scores each answer in code.
Tickmark Bench is a set of graded questions built from the XBRL financial data that companies file with the SEC on EDGAR. It is meant for teams that test language models and agents on financial filings, and for teams that build RL environments and need tasks with a checkable answer.
What a task asks
Each task names the company, its CIK, the form, the accession number, and the filing date. It then asks for one answer in a fixed JSON shape, with the unit, the scale, and the rounding spelled out. This single_number task is from the free sample:
Momentus Inc. (CIK 0001781162) filed a 10-Q on 2026-08-11 under accession 0001628280-26-055624. What was its current ratio as of June 30, 2026? Express the answer as a pure ratio rounded half-up to 2 decimal places (unit "ratio", scale "ones"). Reply with JSON only, of the form {"value": "1.23", "unit": "ratio", "scale": "ones"} using the unit and scale requested above.
The reference answer is 13.70. It is computed from the current assets and the current liabilities that Momentus filed for June 30, 2026. The sample page shows one task of each type with its reference answer.
Four task families
single_number: one figure, often a computed ratio or margin, from one filing period.multi_period: a change across periods, or a figure from a segment.claim_check: a yes or no verdict on a stated claim.comparison: the same metric for more than one company.
How answers are graded
Core is scored entirely by grade.py. Numeric answers are scored within the stated tolerance, and claim checks on the verdict alone. The unit and the scale have to match the question. The official score is score = passed/149. The grading page walks through a pass and a fail.
What Core contains
Core is the real_v1 verified batch: 149 tasks from 67 companies, with filing dates from 2026-04-07 to 2026-09-29. The source filings are 10-Q and 10-K reports, including 10-K/A amendments. A task is kept only when every reference fact matches the filing's XBRL instance exactly: exact period dates, dimensions, and unit.
Core ships as one zip: tasks.jsonl, grade.py, README, datasheet, license and checksums. Tasks ship in the Prime Intellect verifiers format. Core is $299 one-time on the pricing page.
What is not published yet
Solve rates: not yet published. We publish only measured numbers, with model, attempts and date. The datasheet lists what is measured today and what is not.