1. Where the data comes from
Three sources, all free and public — nothing paid, nothing proprietary, so anyone can check this against the originals.
The U.S. government's own database of official company filings — the same source regulators use. Every “8-K” form (a company's legally required notice of material news) comes from here.
Technical detail + validation
Pulled via data.sec.gov's public submissions API, per company, using each company's SEC-assigned CIK number. We keep the accession number, form type, the exact filing acceptance timestamp, item codes, and period of report. SEC policy requires a descriptive User-Agent header and a rate limit — we cap requests at roughly 8/second. Validation: spot-checked individual filings against EDGAR's own full-text search during development; confirmed the acceptance timestamp field is UTC (not already Eastern time, which would silently corrupt every date calculated from it).
Used only to get the current list of company names in the S&P 100, S&P 400, and S&P 600 stock market indexes — i.e., which companies count as 'large', 'mid-size', or 'small' for this study.
Technical detail + validation
The spec originally called for iShares' official holdings files, but those are blocked behind a bot-detection page even for a plain, honest script request — confirmed by direct testing. Wikipedia's constituent tables (selected by their stable id="constituents" table, not position on the page) were live-verified as a working substitute. Trade-off: Wikipedia is community-maintained and can lag the official index by a few days around a membership change — noted, not hidden.
Daily stock prices — open, high, low, close, and trading volume — adjusted for stock splits and dividends so a 10-for-1 split doesn't look like a 90% crash.
Technical detail + validation
Yahoo's chart endpoint returns unadjusted prices plus a separate adjusted-close figure; we reconstruct the full adjusted open/high/low by applying that same day's adjustment ratio to all four prices, not a separately computed cumulative factor. Validation: tested against a real 10-for-1 stock split in the sample — the day-over-day price move stayed in a normal range with no artificial jump, confirming the math holds up across a real split.
2. How we picked which companies to study
The companies weren't hand-picked. They were chosen by a fixed, repeatable rule, applied the same way to every candidate — so the selection can't be (consciously or not) tilted toward a result.
- 01Start with the S&P 100, S&P 400, and S&P 600These are three standard stock market groupings by company size — roughly the 100 largest US public companies, 400 mid-sized ones, and 600 smaller ones. That's 1,104 candidate companies to start.
- 02Common stock onlyDrop anything that isn't a plain, ordinary share — special share classes, preferred stock, and similar.
- 03Priced at $5 or moreVery cheap ('penny') stocks behave differently and are easier to manipulate — excluded for reliability.
- 04Actually trades enough to matterNeeds at least 500,000 shares changing hands per day on average over 3 months, so the price data reflects real trading, not a handful of trades.
- 05Ticker isn't an ordinary English wordTickers spelled like ordinary words (ALL, NOW, SO...) are excluded — a rule to avoid confusing a company's stock symbol with the plain word in later text-based work. A real, if blunt, side effect: this drops some very well-known companies purely because of their ticker spelling, not anything about the company.
- 06Filed something with the SEC in the last yearConfirms the company is an active, currently-reporting business, not a dead shell.
- 07No single industry dominates a groupNo sector is allowed to be more than 40% of the large/mid/small group, so one hot industry (say, tech) can't quietly take over the sample.
After all of that: 660 companies made the final cut (74 large, 273 mid-size, 313 small), out of 1,104 candidates considered. Every single company that was dropped, and exactly which rule dropped it, is logged — 444 exclusion records in total, nothing silently discarded. Breakdown of why companies didn't make it in:
- Doesn't trade enough shares per day251
- Ticker symbol is an ordinary English word137
- Not enough price history available (often a recent IPO/spinoff)45
- Priced under $57
- Not a plain common-stock ticker4
3. How we decide which trading day “counts”
A company can file news at any time — 6am, during trading hours, or 11pm. The stock market is only open 9:30am–4pm Eastern on weekdays, so we need a consistent rule for which day's price move actually reflects that news.
- 01Filed before 9:30am on a trading dayThat whole day counts — the news was out before the market even opened, so the entire day's move (the opening jump and the trading session) is fair game.
- 02Filed between 9:30am and 4pm on a trading daySame day, but only from that moment forward really reflects the news — the price before the filing was set without it.
- 03Filed after 4pm, or on a weekend/holidayCounts as the next trading day — nobody could react until the market reopens.
A worked example: a filing accepted at 5:30pm Eastern on a Friday counts as the following Monday (the next trading day). One accepted at 2:00pm on a Friday counts as that same Friday. This exact rule is checked by an automated test before every run — including tricky edge cases like a filing landing on a market holiday.
Why this matters more than it sounds
4. What “abnormal” means (the core idea)
Stocks move every day, mostly just because the whole market moves. If the S&P 500 is up 1% and your stock is up 1.2%, that's not really news-driven — that's just the stock doing roughly what the market did. “Abnormal return” is the part of a stock's move that's left over after subtracting out what you'd expect from the market alone.
To know “what you'd expect,” we look at how that specific stock usually behaves relative to the market — some stocks (like a big tech name) typically swing more than the market, others swing less. We measure this relationship using the 60+ trading days before the news (roughly 3 months to a year earlier, not including anything close to the event itself, so the news can't contaminate the baseline).
The exact formula
Estimation: r_stock = α + β × r_market + ε
(fit by ordinary least squares, using ~250 trading days
from 270 to 21 trading days before the event)
Abnormal return on day t:
AR_t = r_stock,t − (α + β × r_market,t)α (alpha) and β (beta) are estimated per stock, per event, from that stock's own trading history — not looked up from an outside source. β roughly captures “how much this stock typically amplifies or dampens a market move”; α captures a small constant drift.
5. The four reaction windows
We don't just look at one moment — we track the reaction over four different stretches of time, because a move that happens right away can behave very differently from one that builds over a week, or fades afterward.
| What we call it | Technical label | What it captures |
|---|---|---|
| Same day | CAR[0] | Just the day the news counts as landing on. |
| Next day too | CAR[0,+1] | That day plus the following trading day. |
| First week | CAR[0,+5] | That day through roughly a week later. |
| 2+ weeks later | CAR[+2,+10] | Deliberately skips the first two days — checks whether an initial move sticks or reverses afterward. |
“CAR” stands for Cumulative Abnormal Return — the abnormal returns from section 4, simply added up across each window's days.
The pipeline also measures two windows that end before the filing was public — the week before it (CAR[-5,-1]) and the week before that (CAR[-10,-6]) — to ask how much of the move had already happened by the time the news was filed. Those are a different question from the four above, and they are kept out of every results view and out of the count in section 8 on purpose: a “before” number sitting next to reaction numbers reads as the opposite of what it means. A plain-language walkthrough of what they show is still to be written.
6. Before-the-open vs. during-the-day
A stock's daily move actually happens in two separate pieces: the jump between yesterday's closing price and today's opening price (the “gap” — this is when overnight news, pre-market trading, and anyone able to react before the bell shows up), and the move during the trading day itself (the “session” — 9:30am to 4pm). We calculate an abnormal return for each piece separately, using the same α and β from section 4.
The exact formula
gap return = ln(open_t / close_{t-1})
session return = ln(close_t / open_t)
(these two always add up exactly to the day's total return)Note: the abnormal (news-adjusted) gap and session pieces don't sum back to exactly the day's total abnormal return — that's an accepted, documented side effect of estimating each piece's own α/β contribution separately, not an error.
7. Is the pattern real, or just noise?
Any number you calculate from real-world data has some randomness in it. If you averaged the returns of 20 random, newsless days, you wouldn't get exactly zero — you'd get some small number just from noise. So for every result, we ask: is this bigger than noise alone would produce? We use four different checks, because each is good at catching a different kind of “fake signal.”
Named after the researchers who developed it (Boehmer, Musumeci, Poulsen, 1991). It checks whether the AVERAGE direction of the reaction — up or down — is consistent enough across many events that it's unlikely to be chance.
The exact mechanics
Each abnormal return is first standardized by how volatile that specific stock/event normally is (so one wild stock doesn't dominate the average), then we check whether the average of those standardized values is far enough from zero, given how many events we have and how much they vary.
A second, independent check on direction that doesn't assume returns follow a neat bell-curve shape — it just ranks each day's return from smallest to largest (within that stock's own history) and checks whether event days rank unusually high or low. Real-world stock returns often have more extreme outliers than a perfect bell curve predicts, so having a rank-based check that doesn't rely on that assumption is a useful second opinion.
If more than 1 in 5 events in a group happened on the exact same calendar date (which happens a lot with earnings — many companies report in the same week), the BMP test above can be fooled into overconfidence, because those same-day events tend to move together (the whole market had a day, not just that stock). This adjustment detects that clumping and makes the test appropriately more skeptical when it's present.
This is the one behind the 'how big the swing is on earnings day' number on the homepage. Instead of asking 'does it go up or down on average,' it asks 'is the SIZE of the move — up or down — bigger than you'd expect from an ordinary, newsless day?' This is important because good news and bad news often cancel out in the DIRECTION test above, even when every individual event moved the stock a lot.
The exact mechanics
Technically: compares the average ABSOLUTE VALUE of the standardized abnormal returns against what pure statistical noise would produce (a known baseline, about 0.80 for a normal bell-curve distribution). If the real average is meaningfully higher than that baseline, the events are genuinely more volatile than an ordinary day — not just differently directed.
8. Correcting for testing hundreds of things at once
Here's a trap: if you test 100 totally random, meaningless patterns, on average about 5 of them will look “statistically significant” just by chance (that's literally what a 5%-chance threshold means). The run behind this site checked 6,092 different combinations of company size × news type × time window — so without correcting for that, we'd expect a good number of fake “findings” to show up looking real. The more that gets tested, the higher the bar climbs for everyone — adding the same-filing combination checks genuinely made the direction test stricter for every other result too, which is the correction working as intended, not a bug.
Tested is not the same as published. 4,980 of the cells in that count are same-filing combinations — earnings filed together with financial exhibits, and so on. They were tested and they raised the bar for every result you can see here, but they are not published on this site and there is no view that lists them. What you can browse is the 1,112 post-filing cells keyed to a single news type. So the number above is deliberately larger than anything you can count on the page: it describes the correction that was applied, not the size of the table.
That count covers cells keyed by the SEC's own item codes, whether or not they are published here. The pipeline also splits three of those codes into finer machine-classified sub-types (earnings tone and guidance direction inside 2.02; topic inside the 7.01 and 8.01 catch-alls). Those sub-type cells are tested in their own separate correction families, in separate output files — deliberately not pooled into the number above, because mixing them would change the bar for every item-code result for a reason that has nothing to do with new evidence about item codes. They are not yet displayed here; until they are, treat this page as describing the item-code findings only.
We use a standard statistical correction (Benjamini-Hochberg, 1995) that adjusts the bar for “significant” upward based on how many things were tested — the more tests, the higher the bar each individual result has to clear. Only results that survive this stricter, adjusted bar get a star on this dashboard. The direction test and the magnitude test are corrected separately, since they're answering different questions — and so are the before-the-filing windows from section 5, which is why they aren't in the count above. Keeping those groups apart matters in both directions: it stops a new measurement from quietly raising the bar on results that were already published, and it stops two numbers that were never compared against the same yardstick from being read side by side.
9. Every computed number, defined
Every value that appears anywhere on this site, in one place:
- n
- How many news events are behind this number.
- Count of usable events in this bucket × item-code × window combination, after excluding any with insufficient price history around them.
- Same day / CAR[0] etc.
- The average price move, as a percentage, over that time window.
mean_car— the simple average of each event's cumulative abnormal return (section 4 + 5) across all events in the group. Can be positive or negative.- “How big the swing is”
- The average SIZE of the move, ignoring direction.
mean_abs_car— same as above, but each event's move is made positive before averaging (so an equal number of +5% and -5% moves averages to 5%, not 0%).- “before open” / “during day”
- How much of the average move happened before the market opened vs. during trading.
mean_car_gap/mean_car_session— the section 6 decomposition, averaged the same way as the main number.- Star rating (*, **, ***)
- How confident we are this is a real pattern, not noise.
- Set when
fdr_significant(direction) ormagnitude_fdr_significant(size) is true, after the correction in section 8. *** = strongest confidence (adjusted p < 0.01), ** = p < 0.05, * = p < 0.10. No star at all (even with a big-looking number) means it didn't clear the bar. - “too few”
- Fewer than 10 events — not enough data to say anything about the group.
suppressed— the count is still shown honestly, but every number for that cell is left blank (“—”) rather than printed: no average, no before-open/during-day split, no test result, and never a star. At these sample sizes an “average” can be a single company on a single day, which is not an aggregate at all. A blank never means zero.
10. How we checked our own work
Before trusting any of this on real data, the underlying calculation code had to pass two hard checks. If either one fails, the whole pipeline refuses to publish results — it doesn't matter how good the rest looks.
- 01Plant a fake, known effect and see if we find itWe generated completely made-up, computer-simulated stock data with a hidden, precisely known effect built in on purpose (+1% on the fake event day, hundreds of fake events). Then we ran our own calculation on it blind and checked: did it recover that planted +1%, and correctly flag it as statistically real? It did — and a second version with NO planted effect correctly found nothing. This is like testing a scale with a weight of known mass before trusting what it says about anything else.
- 02Check a well-known real-world effect actually shows upIt's extremely well established in finance research that earnings announcements move stock prices a lot (see the homepage note on this). If that hadn't shown up clearly in our real data, it would mean our code was broken — not that the finance theory was wrong. It did show up, strongly (see the homepage numbers).
The second exam. Passing those two checks means the calculation works. It does not mean any particular pattern we found is real. If you search a large dataset for patterns, you will always find some — and many of them are coincidences that evaporate the moment you look at different data.
So every pattern gets a second exam, on data it was never allowed to see. We split the filings, look for patterns in one part only, write down what we found as a fixed prediction, and then test that written-down list against the untouched part. We do this twice over, splitting two different ways: once by time, so a pattern found in earlier filings has to hold up in later ones, and once by company, so a pattern found in one set of companies has to hold up in a completely separate set with no overlap.
A finding is only reported as validated if it survived both. Plenty did not — and those are kept in the record and reported as refuted rather than quietly dropped, because a pattern that failed its second exam is exactly the thing you would most want to know about if you were deciding whether to trust the rest.
Gate status on this build: ✓ code tests passed · ✓ enough companies/price data · ✓ known effect confirmed
11. What this can't tell you
- 01It's not a trading signalThis is descriptive research on the past, not a prediction of what any specific stock will do next. Nothing here should be used to make a trading decision.
- 02Survivorship biasThe company list is today's S&P 100/400/600 membership, not who was actually in those groups 4 years ago. Companies that got removed (bought out, delisted, went bankrupt) during the window are entirely missing from this sample — which tends to make the sample look a bit better/safer than reality, since the failures aren't represented.
- 03One 4-year window, one point in timeThis isn't a decades-long study. A different 4-year period could show somewhat different numbers.
- 04Daily prices onlyNo minute-by-minute data — the 'before open / during day' split is as precise as it gets here. A move described as 'during the day' could really be concentrated in the first five minutes after the open.
- 05No outlier cappingA couple of extreme individual moves (real ones, not errors — checked) are included as-is in the averages. They're a small fraction of the data, but worth knowing they're not filtered out.
- 06Same-filing combinations are tested but not published here; cross-time sequences aren't tested at allTwo separate limits, easy to confuse. First: combinations of news types that appear TOGETHER ON ONE FILING (e.g. earnings + financial exhibits) ARE tested — they count toward the correction in section 8 — but they are not published on this site, so you cannot browse them. Second, and different: sequences ACROSS separate filings over time (e.g. an executive departure followed by a weak earnings report 30 days later) are not tested at all. That would need a different kind of analysis — joining a company's own event history against itself — and most specific sequences would likely still be too rare even at a much larger sample, since you're multiplying two already-uncommon events together.
Full technical limitations list, including every build decision and its reasoning, is in the repo's LIMITATIONS.md.