Auto Analyst: What an Instant Statistical Report Actually Does
After reading this you will know what each number in the report means, how the tool decides which statistical test to run, and how to avoid the two mistakes that turn a true pattern into a false story: chasing tiny effects and getting fooled by Simpson's paradox.
What it is, with one hook
Auto Analyst reads a spreadsheet the way a careful statistician would on the first pass. It guesses what each column holds, checks the data for the usual damage, describes every distribution, then tests every pair of columns and ranks the real relationships. All of it runs in your browser, so the file never leaves your device.
Here is the hook. Load the Berkeley 1973 admissions sample, which holds the six largest departments. In those departments 45% of the men who applied were admitted, against 30% of the women. That fifteen-point gap looks like bias against women. But split by department and it reverses in four of the six and nearly vanishes in the other two. Women applied more often to departments with low admission rates for everyone. The pooled number told one story; the within-department numbers told the opposite. Auto Analyst flags exactly this, and the rest of this article explains how.
When to use it, and when not
Use it for a first look at a table you do not know well, from a few dozen rows to a few hundred thousand and up to a few dozen columns. A table of 200,000 rows and 12 columns takes about five seconds; with 70 columns there are 2,400 pairs to test and it takes a few seconds more. It is good at telling you where to look, which columns are broken, and which pairs move together. It is a triage tool.
Do not treat its output as a finished analysis. The relationships it reports are associations, not causes. The key-driver ranking is predictive, not causal. If you need a specific estimate with a confidence interval you control, or a model you will deploy, use the report to pick variables and then fit that model yourself.
Every reported relationship is still a hypothesis. The tool controls the rate of false alarms across all tests, but it cannot know that your "revenue" column is actually two currencies stacked together, or that a survey question changed wording halfway through. Read the data-check section before you trust any finding.
The test each pair gets, and why
There is no single correlation that works for every pair of columns. The type of each column decides the test. Auto Analyst picks one of three families.
- Two numbers
- Spearman's rank correlation \rho. It ranks the values first, so a single huge outlier cannot drag it the way it drags a plain Pearson correlation.
- A number across groups
- Mann-Whitney for two groups, Kruskal-Wallis for three or more. Both ask whether one group tends to sit higher in the ranking than another.
- Two categories
- A chi-square test of independence, or Fisher's exact test when a 2x2 table has small counts.
Because the tool may run hundreds of tests, raw p-values are misleading. Test 100 unrelated columns at the usual 0.05 threshold and about 5 will look "significant" by chance alone. The Benjamini-Hochberg procedure adjusts all the p-values together so that, among the relationships it reports, no more than one in twenty is expected to be a false alarm. The computed statistics agree with SciPy to about thirteen significant digits.
One strength scale for every test
A p-value answers "is this real?" It does not answer "is this big?" With 50,000 rows a correlation of \rho = 0.02 is highly significant and completely uninteresting. So Auto Analyst ranks findings by effect size, on a single scale that works across all three test families.
Each test produces a number between 0 and 1: Spearman's \rho for two numbers, the rank-biserial r or \sqrt{\eta^2} for a number across groups, and Cramer's V for two categories. The same cut points apply to all of them.
| Value | Label | What it feels like |
|---|---|---|
| < 0.1 | negligible | not worth a sentence |
| 0.1 to 0.3 | weak | visible in a chart, noisy |
| 0.3 to 0.5 | moderate | clear trend |
| ≥ 0.5 | strong | one column largely tracks the other |
A worked example with the penguins
Load the Palmer penguins sample: 344 rows, measurements of three species at Palmer Station. Two of its numeric columns, flipper length and body mass, move together tightly.
Flipper length against body mass
- Both columns are numbers, so the test is Spearman's \rho.
- Rank all 342 non-missing flipper lengths from 1 to 342, then rank the body masses the same way.
- Correlate the two rank columns. The result is about \rho \approx 0.84.
- 0.84 is above 0.5, so the report labels it strong and ranks it near the top.
- The p-value is far below any threshold, and survives the Benjamini-Hochberg adjustment, so it is reported, not filtered out.
In plain English: penguins with longer flippers almost always weigh more. The chart below shows the raw cloud of points, which is what the report draws beside the sentence.
Reading the key drivers and the time view
Pick a column and Auto Analyst grows a random forest in your browser to predict it from the rest. Two numbers come out of that.
The out-of-bag score measures accuracy on rows each tree never saw during training, which keeps it honest. For the penguins' body mass the forest scores an out-of-bag R-squared of about 0.87: the other columns together explain about 87 percent of the variation in mass on rows the trees never saw. Species ranks first, then flipper length, then sex.
The importance of each column is how much that out-of-bag score drops when you shuffle that one column. If shuffling flipper length cuts the score more than shuffling island does, flipper length is the stronger predictor. The partial-dependence curve then shows the average prediction as that one column slides across its range, holding the rest fixed by averaging.
Importance is predictive, not causal. Flipper length predicts body mass well, but lengthening a penguin's flipper would not fatten it. Both track the bird's overall size, which differs by species and sex. The forest ranks what helps you guess, not what you could change.
Simpson's paradox: the warning that matters most
A pattern in the whole table can reverse inside every subgroup. That is Simpson's paradox, and the Berkeley admissions data is the classic case.
Why the Berkeley gap flips
- Pooled over the six departments: men admitted
45%, women30%. A gap of 15 points, apparently against women. - Department A admitted
62%of men and82%of women. Here women did better. - Most departments show a small female advantage or near parity once you split.
- Women applied far more often to departments C to F, which admitted between 6 and 37 percent of applicants of either sex, while half of the men applied to A and B, which admitted over 60 percent. That mix dragged the female pooled average down.
- Auto Analyst compares the pooled association with a Mantel-Haenszel odds ratio pooled within departments. The two disagree, so it raises a paradox warning.
The scan runs on every grouping column with 2 to 8 groups. It reports both patterns that reverse and patterns that merely vanish once you control for the group.
The data checks and the mistakes they catch
Before any test runs, the tool audits the table. Each check maps to a mistake that silently corrupts results.
- Placeholder codes. A column of ages with a few
999values, or-1for "unknown", will inflate every average. These are treated as missing. One999among forty ages near 30 lifts the mean from about 30 to about 54. - Patterned missingness. If income is missing for 40 percent of one region but 5 percent of another, dropping those rows biases the comparison. The report flags columns that go missing together and groups that go missing more.
- Spelling variants.
USA,U.S.A.andusasplit one category into three and weaken every test on it. The tool merges labels that differ only in case, spacing or punctuation; true synonyms such asUnited Statesstay separate, so glance at each column's top values. - Outliers. Values beyond three interquartile ranges from the quartiles are marked, not deleted, so you can judge them.
- Ambiguous dates.
03/04could be 3 April or 4 March. When any date in the column settles the order, such as25/03, the whole column follows it; when none does, the tool uses the order set under Reading options (day first unless your browser is set to US English) and says so in the data checks. - Duplicate and repeated columns. Exact duplicate rows, duplicated IDs, and a column that simply restates another all get flagged.
On files past 20,000 rows the pairwise tests run on a random sample of 20,000 and the forest on 6,000. At 20,000 rows a correlation is pinned to about \pm 0.01, so the sampling costs you almost nothing.
Related tools on this site
Auto Analyst is the reading step. These handle the steps around it. Clean or reshape first with the CSV Viewer & Editor, convert spreadsheets with the Excel to CSV Converter, or query with the SQL Playground. To summarize counts yourself, use the Pivot Table Maker. To draw a specific chart from a finding, open the Data Visualizer. If you need to share a file, mask the sensitive columns first with the Data Anonymizer. To probe suspicious numbers, try the Fabricated Number Detector.
Frequently asked questions
Does my data get uploaded anywhere?
No. Every step, the type detection, the tests, the random forest and the paradox scan, runs in your browser. The file never leaves your device.
Why are findings ranked by strength and not by p-value?
Because with enough rows almost everything becomes statistically significant. A p-value of 0.0001 on a correlation of 0.03 is real and useless. Ranking by effect size puts the relationships worth looking at on top.
Why Spearman instead of the usual Pearson correlation?
Spearman ranks values before correlating, so one extreme outlier or a curved-but-steady relationship does not break it. It answers "do these move together in the same order?", which is the honest first question for messy data.
Can I trust the key drivers as causes?
No. They are the columns that best predict your target, measured on rows the trees never saw. Prediction is not cause. Use them to decide what to study, then design an experiment or a causal model.
What does the false-discovery control actually promise?
Among the relationships the report shows you, no more than one in twenty is expected to be a false alarm, on average. It is a guarantee about the whole batch, not about any single finding.