Home › I tested it with AI

I tested it with AI · Experiment #1

I tested copy-trading the top Polymarket wallets (and Kalshi strategies) with AI. Here's why it loses money after fees.

"Just copy the whales" sounds smart. So I had AI pull 90 days of Polymarket data, find the top wallets and test copying them. If you'd known the winners in advance, $1,000 became $4,580. Using only what you'd actually have known at the time, it became $449.

Not financial advice. This is a backtest on past data. No real money was bet. Prediction markets and sports betting can lose you money fast, and they may not be legal where you live. If betting is causing you problems, call or text 1-800-GAMBLER (U.S.).

The short version

• The top wallets really did win: the top 100 made $91M in 90 days.
• But last month's winners mostly weren't next month's winners. Copying them lost 4.5% per bet after fees.
• On Kalshi, buying the favorite in sports games lost money in all 64 versions I tested.
• When Kalshi and Polymarket priced the same game, they basically agreed, so there was no easy gap to grab.

What I tested

I asked AI to write scripts that pull public Polymarket data, rank wallets by profit over the last 90 days, and look at what the winners bet on. Then I had it simulate copying them. The rule for the fair test was simple: you only get information you'd have had on the day.

Every Polymarket number on this page uses the same setup:

• Who to copy: the top 20 wallets by profit over the previous 30 days, re-ranked as time goes on.
• Delay: you enter 5 minutes after they trade, because you have to notice the trade first.
• Bet size: $20 per bet from a $1,000 bankroll.
• Costs: 1¢ of slippage per trade, plus Polymarket's taker fees where they apply.

The "hindsight" version uses the same settings but picks the final 90-day top 20 in advance, which nobody could actually do.

The top wallets really did win

The top 100 wallets made $91M in 90 days. Sports was their biggest category at $43M, about a 12% return on the money they put in. So the obvious idea is to copy them.

The catch is what the biggest winners looked like. The top 5 made $22.4M combined, mostly from a few huge bets. The #1 wallet made $7M on just 18 positions. And 32 of the top 100 were brand-new wallets. You can't follow someone who didn't exist last month.

What happened when I copied them

• 3,035 bets were copyable at a 5-minute delay.
• 48.4% of them won.
• After fees, they averaged −4.5% per bet.
• The $1,000 bankroll ended at $449, down 55%.
• Adding a take-profit/stop-loss made it worse: −6.8% per bet. Selling early means paying the spread twice and giving up some wins.

The hindsight version, with the same settings, ended at $4,580. That gap is the whole story: what looks like a system in hindsight is a slow leak in real life.

Then I tried Kalshi

The idea: buy the favorite in sports games when it's priced between 40¢ and 70¢, 2 hours before the start.

• 3,809 bets. The favorites won 55.9% of the time and still lost 5.7% per bet.
• They were already slightly overpriced: −2.7% before any fees. Kalshi's fee adds about 3% more.
• I tested 64 versions (different timing, price ranges and exits). Every one lost money. Take-profit/stop-loss exits did worse because you pay the fee and spread twice.

Last idea: bet when the two sites disagree

If Kalshi and Polymarket price the same game differently, maybe you can grab the gap. Across about 11,000 matched pairs, the median gap was 0¢. A gap of 5¢ or more showed up 25 times, mostly on dead markets nobody was trading. The two sites basically agree.

Why it loses

• Winners don't repeat. Ranking wallets on last month's profit didn't pick next month's winners.
• Big wins are lumpy. The top earners made a few giant bets. That's not a pattern you can follow.
• Costs add up. A cent of slippage and a fee on every bet turns a near-coin-flip into a steady loss.
• Prices are already fair-ish. Favorites win more often, but you pay for that in the price.

Limitations (read these)

• It's a backtest of one 90-day window. A different period could look different.
• The fills are simulated. Real trades can get worse prices, especially on thin markets.
• Bankroll results swing a lot with luck, so the per-bet numbers are more trustworthy than the final dollar amounts.
• I tested a handful of simple rules, not every possible strategy.
• The AI wrote the scripts, and I checked the headline numbers against the raw results. AI code can still have bugs, so treat this as a careful experiment, not proof.

How to run a test like this with AI

You don't need to be a programmer. The trick is to make the AI be honest about hindsight. These prompts work in most AI chat assistants that can write and run code.

1. Get the data and write down your rules first.

   I want to backtest a simple idea: [DESCRIBE THE IDEA, e.g. copying the top wallets on a prediction market]. Help me find a public data source, then write the exact rules before we look at any results: what triggers a bet, the bet size, the delay before I could act, and all fees and slippage.

2. Make it avoid hindsight.

   Write the backtest so every decision uses only data available at that moment. No future information. Then add a separate 'hindsight' version that cheats on purpose, so I can see how big the difference is.

3. Count every cost.

   Include realistic costs on every trade: [FEES], [SLIPPAGE per trade], and the spread when selling early. Show results with and without costs.

4. Ask for honest numbers.

   Report: number of bets, win rate, average return per bet with a 95% confidence interval, final bankroll and worst drawdown. Then tell me plainly whether this would have made money, and list the biggest limitations of this test.

5. Stress-test it.

   Re-run with a longer delay, higher costs and a different time period. Does the result hold up? What's the most likely reason it would fail in real life?

Tip: if a result looks amazing, ask the AI "Where could hindsight or a bug be sneaking in?" before you believe it.

Bottom line

Copying winners sounds like a shortcut, but in this test it was a slow way to lose money once you account for delay and fees. The more useful skill is the method: write the rules first, ban hindsight, and count every cost. That works for testing almost any "can't-lose" idea someone pitches you.

Data window: 90 days ending Oct 2026. Backtest only, no real money bet. Not financial advice.