Financial agent evaluation on SEC filings.

What the agent is asked

Every task in the sample names the company, its CIK, the form, the filing date, and the accession number, then asks for one number or one verdict in a fixed JSON shape. From the free sample:

Republic Services, Inc. (CIK 0001060391) filed a 10-Q on 2026-08-07 under accession 0001060391-26-000275. What was its operating margin for the quarter ended June 30, 2026? Express the answer as a percentage rounded half-up to 2 decimal places (unit "percent", scale "ones").

The reference is 20.34, from operating income of 901000000 USD over revenue of 4430000000 USD for the quarter.

What the agent has to get right

• Find the filing from its accession number, and the right facts inside it.
• Take the quarter, not the year-to-date figure, and the current period, not the prior-year column.
• Compute the ratio, the margin, or the change, and round the way the question says.
• Reply in the unit and scale the question names.

Some tasks need two filings. The Texas Pacific Land task in the sample takes the prior quarter's revenue from a separate 10-Q, accession 0001811074-26-000035, and asks for the change in millions of US dollars. The reference is 9.24.

What you bring, and what Core brings

Core is one zip: tasks.jsonl, grade.py, README, datasheet, license and checksums. The agent, its tools, and its access to EDGAR are yours. You run the agent on each question, collect its JSON replies, and score them with grade.py.

How the score works

Core is scored entirely by grade.py. Numeric answers are scored within the stated tolerance, and claim checks on the verdict alone. The official score is score = passed/149.

Other agent benchmarks

For open research questions with web and EDGAR tools, see the Finance Agent Benchmark from Vals AI. How it and others compare is on the comparison page.

Start

Run your agent on the free sample, then read how to evaluate a language model on SEC filings. Core is on the pricing page.