Technology

EdotEnv Wants to Train AI Agents on Real Market Data, Not Static Benchmarks

Martin HollowayPublished 4d ago5 min readBased on 3 sources
Reading level
EdotEnv Wants to Train AI Agents on Real Market Data, Not Static Benchmarks
source:edotenv.com

EdotEnv, a Y Combinator S26 startup founded by two former quantitative traders, is building reinforcement learning environments derived from real financial market data. The goal is to train and evaluate LLM-based agents on applied machine learning tasks. The company has open-sourced a sample task repository on GitHub and plans to sell continuously improving environments to AI labs, researchers, and enterprises that train their own agents. (edotenv.com; Hacker News)

The core premise is straightforward. Static synthetic benchmarks — the kind where an AI is tested on a fixed set of coding or math problems — saturate quickly once models get good enough, and they fail to capture the messiness of real-world decision-making. Markets, by contrast, never stop generating fresh data. They embed natural noise, present genuine trade-offs, and deliver immediate, verifiable rewards: either a strategy makes money or it does not. No LLM-based judge or human expert is needed to score the result. (edotenv.com; Hacker News)

In EdotEnv's environments, agents operate with professional-grade tools and can build their own tools in Bash. Tasks span building predictive features and models, designing portfolios, backtesting strategies against historical data, and adapting to shifting market regimes. Each task ships with a backtesting tool, an execution tool, and a final evaluation step. The reward function is designed to isolate the agent's feature-building skills while still benefiting from market properties. (Hacker News)

The founders, Rui and Michael, come from quantitative trading backgrounds. Their Y Combinator affiliation places EdotEnv in the S26 batch. (Hacker News)

EdotEnv's own findings from running state-of-the-art models through these environments are notable. The models struggle to iterate deeply on research ideas. Higher reasoning effort does not translate into improved performance. And when agents start losing money, they stop trading rather than adjusting their approach. (Hacker News)

These observations cut against the current grain of LLM agent research, where increased reasoning budgets and chain-of-thought scaffolding are generally expected to yield compounding gains on complex tasks. In a market-derived environment with adversarial noise and long-horizon feedback loops, that assumption does not appear to hold. The agents' tendency to disengage under losses rather than adapt is a specific failure mode that synthetic benchmarks would not surface, because synthetic environments rarely penalize inaction in a way that compounds.

The sample repository, available at github.com/MMcollab-dotcom/feature-engineering, lets researchers inspect the task structure, tooling, and reward design directly. (Hacker News)

The business model targets a real gap. AI labs training agents need environments that do not saturate. Most existing benchmarks, whether for coding, math, or general reasoning, eventually see frontier models approach or exceed human-level performance, at which point the signal fades. Markets offer a domain where the problem space regenerates. EdotEnv's plan to sell continuously improving environments to labs and enterprises positions it as an infrastructure provider for agent evaluation. (Hacker News)

There are inherent limitations to using financial markets as a training substrate worth noting. Market data is noisy, non-stationary, and subject to regime changes that can persist for extended periods. An agent trained to exploit a specific market regime may perform well in evaluation and poorly in deployment. This is a known problem in quantitative finance, and EdotEnv's reward design, which isolates feature-building skills from raw market exposure, is an attempt to manage it. Whether that isolation holds across diverse market conditions remains an empirical question.

A broader question is whether skills learned here transfer to other domains. EdotEnv frames its environments as teaching applied machine learning and long-horizon planning under adversarial noise — both general capabilities. A model that learns to build predictive features, backtest them, and adapt to regime shifts in a financial context may transfer those skills to other domains. It may also overfit to the specific structure of market data. The company's own observations that current models struggle to iterate deeply on research ideas suggest that transfer, if it occurs at all, is not happening yet at a level the founders find compelling.

The competitive landscape for RL environments is active. Several efforts, academic and commercial, are working on agentic benchmarks that move beyond static evaluation. EdotEnv's differentiation rests on using real market data rather than synthetic constructions, which provides natural ground-truth rewards and avoids the ceiling effects of hand-designed benchmarks. The trade-off is that real market data introduces confounders that synthetic environments can control more precisely.

For AI labs, the appeal of a non-saturating environment is direct. Training pipelines that depend on benchmark progress need fresh signal as models improve. A market-derived environment that regenerates its own difficulty could extend the useful life of an evaluation pipeline. For quantitative finance firms, the appeal is different: an agent that can build features, backtest, and adapt autonomously is a compressed version of a quant research workflow.

EdotEnv is early. The sample repository is public, the full environment set is not yet commercially available, and the founders' own results show current models falling short of the deep iteration the tasks demand. What is on offer is a clear methodology, a concrete open-source artifact, and a business thesis that addresses a genuine problem in agent evaluation.