Methodology

Every number on this site, explained plainly — then exactly.

You don't need a finance or math background to read this. Each section starts in plain language, then shows the exact formula and data underneath for anyone who wants to check the work or reproduce it.

Tests corrected across
6,092
Published here
1,112

Three sources, all free and public — nothing paid, nothing proprietary, so anyone can check this against the originals.

SEC EDGAR

The U.S. government's own database of official company filings — the same source regulators use. Every “8-K” form (a company's legally required notice of material news) comes from here.

Technical detail + validation

Pulled via data.sec.gov's public submissions API, per company, using each company's SEC-assigned CIK number. We keep the accession number, form type, the exact filing acceptance timestamp, item codes, and period of report. SEC policy requires a descriptive User-Agent header and a rate limit — we cap requests at roughly 8/second. Validation: spot-checked individual filings against EDGAR's own full-text search during development; confirmed the acceptance timestamp field is UTC (not already Eastern time, which would silently corrupt every date calculated from it).

Wikipedia

Used only to get the current list of company names in the S&P 100, S&P 400, and S&P 600 stock market indexes — i.e., which companies count as 'large', 'mid-size', or 'small' for this study.

Technical detail + validation

The spec originally called for iShares' official holdings files, but those are blocked behind a bot-detection page even for a plain, honest script request — confirmed by direct testing. Wikipedia's constituent tables (selected by their stable id="constituents" table, not position on the page) were live-verified as a working substitute. Trade-off: Wikipedia is community-maintained and can lag the official index by a few days around a membership change — noted, not hidden.

Yahoo Finance

Daily stock prices — open, high, low, close, and trading volume — adjusted for stock splits and dividends so a 10-for-1 split doesn't look like a 90% crash.

Technical detail + validation

Yahoo's chart endpoint returns unadjusted prices plus a separate adjusted-close figure; we reconstruct the full adjusted open/high/low by applying that same day's adjustment ratio to all four prices, not a separately computed cumulative factor. Validation: tested against a real 10-for-1 stock split in the sample — the day-over-day price move stayed in a normal range with no artificial jump, confirming the math holds up across a real split.

The companies weren't hand-picked. They were chosen by a fixed, repeatable rule, applied the same way to every candidate — so the selection can't be (consciously or not) tilted toward a result.

  1. 01
    Start with the S&P 100, S&P 400, and S&P 600
    These are three standard stock market groupings by company size — roughly the 100 largest US public companies, 400 mid-sized ones, and 600 smaller ones. That's 1,104 candidate companies to start.
  2. 02
    Common stock only
    Drop anything that isn't a plain, ordinary share — special share classes, preferred stock, and similar.
  3. 03
    Priced at $5 or more
    Very cheap ('penny') stocks behave differently and are easier to manipulate — excluded for reliability.
  4. 04
    Actually trades enough to matter
    Needs at least 500,000 shares changing hands per day on average over 3 months, so the price data reflects real trading, not a handful of trades.
  5. 05
    Ticker isn't an ordinary English word
    Tickers spelled like ordinary words (ALL, NOW, SO...) are excluded — a rule to avoid confusing a company's stock symbol with the plain word in later text-based work. A real, if blunt, side effect: this drops some very well-known companies purely because of their ticker spelling, not anything about the company.
  6. 06
    Filed something with the SEC in the last year
    Confirms the company is an active, currently-reporting business, not a dead shell.
  7. 07
    No single industry dominates a group
    No sector is allowed to be more than 40% of the large/mid/small group, so one hot industry (say, tech) can't quietly take over the sample.

After all of that: 660 companies made the final cut (74 large, 273 mid-size, 313 small), out of 1,104 candidates considered. Every single company that was dropped, and exactly which rule dropped it, is logged — 444 exclusion records in total, nothing silently discarded. Breakdown of why companies didn't make it in:

  • Doesn't trade enough shares per day251
  • Ticker symbol is an ordinary English word137
  • Not enough price history available (often a recent IPO/spinoff)45
  • Priced under $57
  • Not a plain common-stock ticker4

A company can file news at any time — 6am, during trading hours, or 11pm. The stock market is only open 9:30am–4pm Eastern on weekdays, so we need a consistent rule for which day's price move actually reflects that news.

  1. 01
    Filed before 9:30am on a trading day
    That whole day counts — the news was out before the market even opened, so the entire day's move (the opening jump and the trading session) is fair game.
  2. 02
    Filed between 9:30am and 4pm on a trading day
    Same day, but only from that moment forward really reflects the news — the price before the filing was set without it.
  3. 03
    Filed after 4pm, or on a weekend/holiday
    Counts as the next trading day — nobody could react until the market reopens.

A worked example: a filing accepted at 5:30pm Eastern on a Friday counts as the following Monday (the next trading day). One accepted at 2:00pm on a Friday counts as that same Friday. This exact rule is checked by an automated test before every run — including tricky edge cases like a filing landing on a market holiday.

Why this matters more than it sounds
SEC filing timestamps are recorded in UTC, not Eastern time — confirmed directly against live data. Treating a UTC timestamp as if it were already Eastern time would shift every single event by up to several hours, which can flip a “before the open” event into an “after the close, next-day” event. Converting timezones correctly, and using an actual market-holiday calendar (not just “skip weekends”), is the single easiest thing to get quietly wrong in a pipeline like this.

Stocks move every day, mostly just because the whole market moves. If the S&P 500 is up 1% and your stock is up 1.2%, that's not really news-driven — that's just the stock doing roughly what the market did. “Abnormal return” is the part of a stock's move that's left over after subtracting out what you'd expect from the market alone.

To know “what you'd expect,” we look at how that specific stock usually behaves relative to the market — some stocks (like a big tech name) typically swing more than the market, others swing less. We measure this relationship using the 60+ trading days before the news (roughly 3 months to a year earlier, not including anything close to the event itself, so the news can't contaminate the baseline).

The exact formula
Estimation: r_stock = α + β × r_market + ε
              (fit by ordinary least squares, using ~250 trading days
               from 270 to 21 trading days before the event)

Abnormal return on day t:
  AR_t = r_stock,t − (α + β × r_market,t)

α (alpha) and β (beta) are estimated per stock, per event, from that stock's own trading history — not looked up from an outside source. β roughly captures “how much this stock typically amplifies or dampens a market move”; α captures a small constant drift.

We don't just look at one moment — we track the reaction over four different stretches of time, because a move that happens right away can behave very differently from one that builds over a week, or fades afterward.

What we call itTechnical labelWhat it captures
Same dayCAR[0]Just the day the news counts as landing on.
Next day tooCAR[0,+1]That day plus the following trading day.
First weekCAR[0,+5]That day through roughly a week later.
2+ weeks laterCAR[+2,+10]Deliberately skips the first two days — checks whether an initial move sticks or reverses afterward.

“CAR” stands for Cumulative Abnormal Return — the abnormal returns from section 4, simply added up across each window's days.

The pipeline also measures two windows that end before the filing was public — the week before it (CAR[-5,-1]) and the week before that (CAR[-10,-6]) — to ask how much of the move had already happened by the time the news was filed. Those are a different question from the four above, and they are kept out of every results view and out of the count in section 8 on purpose: a “before” number sitting next to reaction numbers reads as the opposite of what it means. A plain-language walkthrough of what they show is still to be written.

A stock's daily move actually happens in two separate pieces: the jump between yesterday's closing price and today's opening price (the “gap” — this is when overnight news, pre-market trading, and anyone able to react before the bell shows up), and the move during the trading day itself (the “session” — 9:30am to 4pm). We calculate an abnormal return for each piece separately, using the same α and β from section 4.

The exact formula
gap return     = ln(open_t / close_{t-1})
session return = ln(close_t / open_t)

(these two always add up exactly to the day's total return)

Note: the abnormal (news-adjusted) gap and session pieces don't sum back to exactly the day's total abnormal return — that's an accepted, documented side effect of estimating each piece's own α/β contribution separately, not an error.

Any number you calculate from real-world data has some randomness in it. If you averaged the returns of 20 random, newsless days, you wouldn't get exactly zero — you'd get some small number just from noise. So for every result, we ask: is this bigger than noise alone would produce? We use four different checks, because each is good at catching a different kind of “fake signal.”

Does it move in a consistent direction? (the BMP test)

Named after the researchers who developed it (Boehmer, Musumeci, Poulsen, 1991). It checks whether the AVERAGE direction of the reaction — up or down — is consistent enough across many events that it's unlikely to be chance.

The exact mechanics

Each abnormal return is first standardized by how volatile that specific stock/event normally is (so one wild stock doesn't dominate the average), then we check whether the average of those standardized values is far enough from zero, given how many events we have and how much they vary.

Same question, a different way (the Corrado rank test)

A second, independent check on direction that doesn't assume returns follow a neat bell-curve shape — it just ranks each day's return from smallest to largest (within that stock's own history) and checks whether event days rank unusually high or low. Real-world stock returns often have more extreme outliers than a perfect bell curve predicts, so having a rank-based check that doesn't rely on that assumption is a useful second opinion.

Correcting for events that clump together (Kolari-Pynnönen adjustment)

If more than 1 in 5 events in a group happened on the exact same calendar date (which happens a lot with earnings — many companies report in the same week), the BMP test above can be fooled into overconfidence, because those same-day events tend to move together (the whole market had a day, not just that stock). This adjustment detects that clumping and makes the test appropriately more skeptical when it's present.

Does it move by a lot, regardless of direction? (the magnitude test)

This is the one behind the 'how big the swing is on earnings day' number on the homepage. Instead of asking 'does it go up or down on average,' it asks 'is the SIZE of the move — up or down — bigger than you'd expect from an ordinary, newsless day?' This is important because good news and bad news often cancel out in the DIRECTION test above, even when every individual event moved the stock a lot.

The exact mechanics

Technically: compares the average ABSOLUTE VALUE of the standardized abnormal returns against what pure statistical noise would produce (a known baseline, about 0.80 for a normal bell-curve distribution). If the real average is meaningfully higher than that baseline, the events are genuinely more volatile than an ordinary day — not just differently directed.

Here's a trap: if you test 100 totally random, meaningless patterns, on average about 5 of them will look “statistically significant” just by chance (that's literally what a 5%-chance threshold means). The run behind this site checked 6,092 different combinations of company size × news type × time window — so without correcting for that, we'd expect a good number of fake “findings” to show up looking real. The more that gets tested, the higher the bar climbs for everyone — adding the same-filing combination checks genuinely made the direction test stricter for every other result too, which is the correction working as intended, not a bug.

Tested is not the same as published. 4,980 of the cells in that count are same-filing combinations — earnings filed together with financial exhibits, and so on. They were tested and they raised the bar for every result you can see here, but they are not published on this site and there is no view that lists them. What you can browse is the 1,112 post-filing cells keyed to a single news type. So the number above is deliberately larger than anything you can count on the page: it describes the correction that was applied, not the size of the table.

That count covers cells keyed by the SEC's own item codes, whether or not they are published here. The pipeline also splits three of those codes into finer machine-classified sub-types (earnings tone and guidance direction inside 2.02; topic inside the 7.01 and 8.01 catch-alls). Those sub-type cells are tested in their own separate correction families, in separate output files — deliberately not pooled into the number above, because mixing them would change the bar for every item-code result for a reason that has nothing to do with new evidence about item codes. They are not yet displayed here; until they are, treat this page as describing the item-code findings only.

We use a standard statistical correction (Benjamini-Hochberg, 1995) that adjusts the bar for “significant” upward based on how many things were tested — the more tests, the higher the bar each individual result has to clear. Only results that survive this stricter, adjusted bar get a star on this dashboard. The direction test and the magnitude test are corrected separately, since they're answering different questions — and so are the before-the-filing windows from section 5, which is why they aren't in the count above. Keeping those groups apart matters in both directions: it stops a new measurement from quietly raising the bar on results that were already published, and it stops two numbers that were never compared against the same yardstick from being read side by side.

Every value that appears anywhere on this site, in one place:

n
How many news events are behind this number.
Count of usable events in this bucket × item-code × window combination, after excluding any with insufficient price history around them.
Same day / CAR[0] etc.
The average price move, as a percentage, over that time window.
mean_car — the simple average of each event's cumulative abnormal return (section 4 + 5) across all events in the group. Can be positive or negative.
“How big the swing is”
The average SIZE of the move, ignoring direction.
mean_abs_car — same as above, but each event's move is made positive before averaging (so an equal number of +5% and -5% moves averages to 5%, not 0%).
“before open” / “during day”
How much of the average move happened before the market opened vs. during trading.
mean_car_gap / mean_car_session — the section 6 decomposition, averaged the same way as the main number.
Star rating (*, **, ***)
How confident we are this is a real pattern, not noise.
Set when fdr_significant (direction) or magnitude_fdr_significant (size) is true, after the correction in section 8. *** = strongest confidence (adjusted p < 0.01), ** = p < 0.05, * = p < 0.10. No star at all (even with a big-looking number) means it didn't clear the bar.
“too few”
Fewer than 10 events — not enough data to say anything about the group.
suppressed — the count is still shown honestly, but every number for that cell is left blank (“—”) rather than printed: no average, no before-open/during-day split, no test result, and never a star. At these sample sizes an “average” can be a single company on a single day, which is not an aggregate at all. A blank never means zero.

Before trusting any of this on real data, the underlying calculation code had to pass two hard checks. If either one fails, the whole pipeline refuses to publish results — it doesn't matter how good the rest looks.

  1. 01
    Plant a fake, known effect and see if we find it
    We generated completely made-up, computer-simulated stock data with a hidden, precisely known effect built in on purpose (+1% on the fake event day, hundreds of fake events). Then we ran our own calculation on it blind and checked: did it recover that planted +1%, and correctly flag it as statistically real? It did — and a second version with NO planted effect correctly found nothing. This is like testing a scale with a weight of known mass before trusting what it says about anything else.
  2. 02
    Check a well-known real-world effect actually shows up
    It's extremely well established in finance research that earnings announcements move stock prices a lot (see the homepage note on this). If that hadn't shown up clearly in our real data, it would mean our code was broken — not that the finance theory was wrong. It did show up, strongly (see the homepage numbers).

The second exam. Passing those two checks means the calculation works. It does not mean any particular pattern we found is real. If you search a large dataset for patterns, you will always find some — and many of them are coincidences that evaporate the moment you look at different data.

So every pattern gets a second exam, on data it was never allowed to see. We split the filings, look for patterns in one part only, write down what we found as a fixed prediction, and then test that written-down list against the untouched part. We do this twice over, splitting two different ways: once by time, so a pattern found in earlier filings has to hold up in later ones, and once by company, so a pattern found in one set of companies has to hold up in a completely separate set with no overlap.

A finding is only reported as validated if it survived both. Plenty did not — and those are kept in the record and reported as refuted rather than quietly dropped, because a pattern that failed its second exam is exactly the thing you would most want to know about if you were deciding whether to trust the rest.

Gate status on this build: ✓ code tests passed · ✓ enough companies/price data · ✓ known effect confirmed

  1. 01
    It's not a trading signal
    This is descriptive research on the past, not a prediction of what any specific stock will do next. Nothing here should be used to make a trading decision.
  2. 02
    Survivorship bias
    The company list is today's S&P 100/400/600 membership, not who was actually in those groups 4 years ago. Companies that got removed (bought out, delisted, went bankrupt) during the window are entirely missing from this sample — which tends to make the sample look a bit better/safer than reality, since the failures aren't represented.
  3. 03
    One 4-year window, one point in time
    This isn't a decades-long study. A different 4-year period could show somewhat different numbers.
  4. 04
    Daily prices only
    No minute-by-minute data — the 'before open / during day' split is as precise as it gets here. A move described as 'during the day' could really be concentrated in the first five minutes after the open.
  5. 05
    No outlier capping
    A couple of extreme individual moves (real ones, not errors — checked) are included as-is in the averages. They're a small fraction of the data, but worth knowing they're not filtered out.
  6. 06
    Same-filing combinations are tested but not published here; cross-time sequences aren't tested at all
    Two separate limits, easy to confuse. First: combinations of news types that appear TOGETHER ON ONE FILING (e.g. earnings + financial exhibits) ARE tested — they count toward the correction in section 8 — but they are not published on this site, so you cannot browse them. Second, and different: sequences ACROSS separate filings over time (e.g. an executive departure followed by a weak earnings report 30 days later) are not tested at all. That would need a different kind of analysis — joining a company's own event history against itself — and most specific sequences would likely still be too rare even at a much larger sample, since you're multiplying two already-uncommon events together.

Full technical limitations list, including every build decision and its reasoning, is in the repo's LIMITATIONS.md.