Research notes
Independent research
After the conversation, its factual claims were checked against independent sources. These notes are not part of what Patrick or Ada said. A verdict describes the evidence for the claim, not the value of the idea.
R-R01Supportedcentral claim · high confidence
Claim by Ada Clara (conversation #4, #5, #9):
“Ada Clara said Patrick's explanation (T003) describes the equivalence of gravitational and inertial mass. In Newtonian terms, gravity pulls harder on a heavier object, but its extra mass also gives it more inertia, so the acceleration comes out the same (a = (m_g/m_i)·g). She corrected one point: inertia is not an opposing force. It is the property that sets how much acceleration a given force produces. She also said the two masses are experimentally equivalent, and that their equality is an experimental discovery, not something Newton's laws guarantee on their own.”
Ada Clara's account holds up. She did not name any experiments or give a precision figure. Independent research supplies them. The best test so far, the final MICROSCOPE satellite result (Physical Review Letters 129, 121102, 2022), measured the Eötvös parameter for a titanium–platinum pair as η = [−1.5 ± 2.3 (stat) ± 1.5 (syst)] × 10⁻¹⁵. That is consistent with zero, meaning no violation of the weak equivalence principle at roughly the 10⁻¹⁵ level. The best ground-based torsion-balance result, from the Eöt-Wash group (Schlamminger et al., 2008), was η(Earth; Be–Ti) = (0.3 ± 1.8) × 10⁻¹³. MICROSCOPE improved on that bound by nearly two orders of magnitude. Earlier tests reached about 10⁻⁸ (Eötvös, 1889) and about 10⁻¹² (Braginskii and Panov). In short, within current precision, materials of different composition fall with the same acceleration.
Detail and sources (2)
Who said what. The free-fall mechanism (more gravitational pull, matched by more inertia) was first put forward by Patrick in T003. Ada endorsed it, gave the Newtonian formula, and added the correction about inertia (T004, T005). She described the equivalence of the two masses as experimental (T005, T009). The experiments named here (Eötvös-type torsion balances, Eöt-Wash, MICROSCOPE) and their precision bounds come from independent research, not from the conversation.
What the evidence shows. Strictly speaking, these experiments do not prove that the two kinds of mass are identical. They put upper limits on any difference in how different materials fall, as measured by the Eötvös parameter η. 'Equivalent within about 10⁻¹⁵' is the accurate wording; 'exactly equal' goes beyond what the evidence shows. Ada's phrase 'the ratio is one' (T005) should be read that way. The ratio is one by convention once units are chosen, and experiments show it is the same for all materials within these limits.
Open questions. Theories that go beyond general relativity predict tiny violations, so searches continue. Planned follow-ups such as a proposed MICROSCOPE successor aim for about 10⁻¹⁷. I found no newer result as of late 2026 that supersedes MICROSCOPE's bound, but I did not search exhaustively for 2025–2026 preprints.
One caveat for publication. In T002 Ada gave 9.8 m/s² as the universal acceleration. That figure is the near-Earth value of g. It is not part of the equivalence claim itself.
- MICROSCOPE mission: final results of the test of the Equivalence Principle (arXiv 2209.15487; PRL 129, 121102) · arXiv / Physical Review Letters (APS), 2022 · peer-reviewed · supports · not retrieved (unverified)
Search-result abstract text: no weak equivalence principle violation was found. η(Ti,Pt) = [−1.5 ± 2.3(stat) ± 1.5(syst)] × 10⁻¹⁵, described as the tightest bound to date. Earlier MICROSCOPE results had reached 2 × 10⁻¹⁴. - Test of the Equivalence Principle Using a Rotating Torsion Balance · Physical Review Letters (APS) / arXiv 0712.0607, 2008 · peer-reviewed · supports · not retrieved (unverified)
Search-result abstract text: Eötvös parameter η(Earth, Be–Ti) = (0.3 ± 1.8) × 10⁻¹³, measured with a continuously rotating torsion balance.
Relevant to: Why a pea and a bowling ball fall together
R-R02Supportedsupporting claim · high confidence
Claim by Ada Clara (conversation #5, #9):
“Newton's laws do not themselves guarantee the equality of gravitational and inertial mass. It is an empirical fact that Einstein raised to a principle (the equivalence principle).”
Accurate, and standard textbook history. In Newtonian mechanics, the mass in the law of gravitation (gravitational mass) and the mass in F = ma (inertial mass) are separate quantities. Nothing in the theory requires them to be proportional. Newton checked their proportionality experimentally, using pendulums made of different materials. In 1907 Einstein took the equivalence of a uniform gravitational field and an accelerated reference frame as a starting assumption, and this became a founding principle of general relativity.
Detail and sources (1)
Some precision on wording. Newton did treat the equality as an experimental result, but he then built it into his theory: he inferred that the two masses are the same thing. So 'not guaranteed by the laws' means Newtonian theory would still be consistent if the two masses differed. It does not mean Newton left the question open. Einstein's principle in its 1907 form concerns the equivalence of gravity and acceleration. It includes the equality of the two masses, but it is a broader statement. Ada says in T005 that general relativity reinterprets free fall as motion along paths in curved spacetime. That is the standard reading, though general relativity also takes the equivalence as an assumption rather than deriving it from something deeper. The open question Ada points to, why the two masses are equal at all, is therefore moved into the theory's foundations rather than answered by it.
- Equivalence principle · Wikipedia, 2026 · reference · supports
Says Newton compared pendulums of different materials, found them identical, and inferred that gravitational and inertial mass are the same thing. Says that in 1907 Einstein assumed 'the complete physical equivalence of a gravitational field and a corresponding acceleration of the reference system.'
Relevant to: Why a pea and a bowling ball fall together
R-R03Partly supportedsupporting claim · low confidence
Claim by Patrick (conversation #6, #8, #9):
“Patrick's criticism is that schools show the falling-objects demonstration (such as the pea and the bowling ball) but don't teach why it happens, i.e. the cancellation between gravitational and inertial mass: "they teach the what, not the why" (T006, T010). Ada Clara's considered diagnosis is narrower. She said schools often teach the observation before the mechanism and sometimes never get around to explaining the mechanism (T009). She added that schools aren't hiding it but are "sprinting past the best part" (T008).”
The evidence is mixed. I found no systematic study of curricula on this exact point. Many standard teaching resources do present the Newtonian mechanism. The Physics Classroom derives a = mg/m = g and notes that the mass cancels. Some textbooks state the equivalence of gravitational and inertial mass explicitly (Crowell). Others, such as OpenStax University Physics §5.4, state Galileo's result and derive w = mg without spelling out the cancellation or distinguishing the two kinds of mass. Physics education research also shows that the belief 'heavier objects fall faster' often survives instruction. In one cited sample, 52% of fourth-year university physics students held it. So the mechanism is often taught on paper, but it often does not stick. Practice varies by country, grade and teacher. Ada Clara's qualified version (the observation often comes before the mechanism, and the mechanism is sometimes skipped) is consistent with this evidence. The broader reading, that schools teach the demonstration without the why, is not established as a general fact: the resources above are counterexamples.
Detail and sources (3)
The evidence admits several readings. (a) Resources commonly include the force/mass cancellation, especially once Newton's second law has been taught. Free fall, however, is often introduced earlier in kinematics as a fact (a = g), before forces are covered. That ordering may be what Patrick means by 'what before why'. (b) The explicit distinction between gravitational and inertial mass, and the question of why the two are equal, is covered less often at secondary level. It tends to appear in conceptual or more advanced texts. (c) Students who still believe heavier objects fall faster after instruction suggest the explanation is sometimes ineffective or skipped. But the research also links this belief to everyday experience with air drag and to confusing weight with mass, not only to a missing mechanism.
Attribution: Patrick and Ada Clara did not state the same claim, and their frequency claims differ. Patrick's framing is general and comes from his own schooling ('Why is this stuff that they don't teach us in school?', T006). In T010 he restated Ada's point as 'They teach the what, not the why'. In her short voice reply (T008), Ada said schools 'love to show what happens first and never get around to why', but she qualified this in the same turn ('not that schools are hiding it, they're just sprinting past the best part'). Her fuller answer (T009) is the hedged version: schools 'often' teach the observation first and 'sometimes' never explain the mechanism. In T009 she also acknowledged a legitimate educational reason for starting with experiments, and noted that the equality of the two masses was discovered experimentally.
Gaps: I found no cross-national curriculum survey. The evidence here is a handful of textbook and teaching-resource examples plus misconception studies, which is not a representative sample. One arXiv paper (1607.02594) says mass-dependent acceleration has 'rarely been addressed' in pre-relativistic mechanics. That statement concerns a technical frame-dependence issue, not school curricula, so it should not be cited for this claim.
- University Physics Volume 1, 13.2 Gravitation Near Earth's Surface · OpenStax (Rice University), 2016 · textbook · qualifies
Explicitly derives that 'the mass m of the object cancels' and that 'this explains why all masses free fall with the same acceleration'. Does not discuss inertial versus gravitational mass or the equivalence principle in that section. - Dropping Balls (formative assessment probe) · National Science Teaching Association, n.d. · institutional · context · not retrieved (unverified)
Search-result summary: the probe checks whether students think mass affects how fast objects fall. It cites research that students, including university students, think heavier objects fall faster. - Observations on teaching first-year physics · arXiv, 2005 · other · context · not retrieved (unverified)
Search-result summary: an instructor reports that students' 'all objects fall at the same rate' answer is rehearsed, and that the misconception reappears in novel situations.
Relevant to: Schools teach the what and skip the why
R-R04Supportedsupporting claim · high confidence
Claim by Ada Clara (conversation #12, #14):
“Patrick proposed teaching the 'why' first: the teacher gives the mechanism, the class predicts what should happen if it is true, and then an experiment tests the prediction. He presented this as teaching the scientific method of forming a hypothesis and validating it by experiment. Ada Clara called the sequence 'essentially the hypothetico-deductive method' (propose an explanation, derive a testable prediction, confront it with evidence). She added that an experiment supports a hypothesis rather than proving it.”
Ada Clara's description holds up broadly. In Patrick's sequence the teacher gives an explanation, students predict what a demonstration will show, and the demonstration then tests the prediction. That follows hypothetico-deductive reasoning. Ada did not compare it to any named teaching model. The comparisons below come from this research. The sequence overlaps with established practice. Predict–Observe–Explain (POE) comes from White and Gunstone (1992). Lawson's hypothetico-deductive learning cycles frame reasoning as 'if… then… therefore' and test students' predictions. The closest supporting evidence is Crouch, Fagen, Callan and Mazur (2004). In that physics study, students who only watched a classroom demonstration understood it no better than students who never saw it. Students who predicted the outcome first understood it significantly better. The resemblance is not exact, though. In POE, students predict from their own prior ideas so that misconceptions come to the surface. In Patrick's version, the teacher supplies the mechanism first. Several established inquiry approaches also deliberately put exploration before the explanation, the reverse of 'why first'. These include the classic learning cycle (exploration, then concept invention, then application) and productive failure. So the prediction-and-test part of Patrick's idea is well studied and has good support. Giving the 'why' before any exploration is a design choice the literature has not settled.
Detail and sources (3)
From the conversation: in T012 Patrick described the sequence and linked it to forming a hypothesis and validating it by experiment. In T013–T014 Ada Clara called it hypothesis → prediction → experiment and 'essentially the hypothetico-deductive method'. She also cautioned that experiments support hypotheses rather than prove them. She suggested the 'why' be treated as 'a suspect', with rival explanations in play. Everything below is research analysis, not something either party said.
What matches: POE was designed to uncover students' predictions and their reasons, and to prompt discussion and further investigation. Lawson describes science as generating and testing alternative explanations, and recommends making if-then-therefore reasoning explicit to students. That is close to Ada Clara's 'suspect' framing. Crouch et al. (2004) is direct evidence that a prediction step turns a demonstration from entertainment into learning.
What differs: (1) In POE the prediction comes from the student. In Patrick's version the teacher hands over the mechanism and students deduce what follows from it. That is closer to testing a single hypothesis supplied by the teacher than to drawing out misconceptions. (2) The learning cycle that NARST describes runs exploration first (often a lab), then derives the concept from the data. Productive-failure research (Sinha & Kapur 2021) favours problem-solving before instruction for older learners. Both orderings are the opposite of 'why first'. (3) Explicit-instruction research would partly back giving an explanation early, which shows the ordering question is genuinely debated.
Unknowns: this research found no study comparing Patrick's exact order (teacher explanation, then student prediction, then demonstration) against POE or against exploration-first sequences, whether on free fall or similar topics. The Koyunlu Ünlü POE meta-analysis and POE systematic reviews appeared in search results but could not be opened, so no effect sizes are reported for POE.
- Predict, Observe, Explain (POE) · NZCER Assessment Resource Banks, n.d. · institutional · context · not retrieved (unverified)
Search snippet: the strategy comes from White and Gunstone (1992), Probing Understanding. Students predict an outcome, observe it, and explain any mismatch. - Systematic Literature Review: Predict-Observe-Explain (POE) Model in Science Learning at Junior High School Level · Jurnal Penelitian Pendidikan IPA (JPPIPA), 2026 · peer-reviewed · supports
Opened. Reviews 13 articles (2020–2025). Reports that POE supports scientific literacy and critical thinking and is effective for science learning outcomes. - Science Teaching and the Development of Thinking; Hypothetico-deductive reasoning skill and concept acquisition · Wadsworth (1995); Journal of Research in Science Teaching (1991), 1991/1995 · peer-reviewed · supports · not retrieved (unverified)
Search results: Lawson treats science as hypothetico-deductive (generating and testing alternative explanations) and describes a 'hypothetical-deductive' type of learning cycle for science teaching.
Relevant to: The lesson as an experiment: why, then predict, then test
R-R05Partly supportedcentral claim · medium confidence
Claim by Patrick (conversation #18, #19, #20):
“Patrick proposes an extension of his why-predict-test teaching method. He says the method is "not perfect" but is repeatable, and that doing it "again and again" will "drill into their head the scientific method". As the extension, teachers would purposely give students an incorrect "why" together with what he calls an "incorrect proof". His follow-up suggests that this most likely means an experiment that fails to produce the outcome the wrong "why" predicted, though the phrase itself is ambiguous. Students would then ask "what does that mean?" and conclude, in his words, that "we prove that the why was not correct by creating an experiment that failed to prove the outcome it predicted."”
Related research supports parts of the idea, but no study we found tests Patrick's exact proposal. In T018, Patrick proposed teaching through repetition and deliberately wrong explanations. Teachers would give students an incorrect "why" with an "incorrect proof", and students would work out that the "why" was wrong. The phrase "incorrect proof" is ambiguous. Read with his follow-up sentence, it most likely means an experiment that fails to produce the outcome the wrong "why" predicted. This note uses that reading but treats it as an interpretation, not as something Patrick stated outright. Ada Clara added two things in T019–T020. First, a rationale: the exercise trains students to be comfortable being wrong. Second, a diagnostic step: the class looks into why the prediction failed instead of being told. The research below comes from this pipeline, not from either speaker. The closest research is on refutation texts, which state a common misconception and then explicitly correct it. Two meta-analyses find they reliably beat standard explanatory texts: Schroeder & Kucera (about 44 comparisons, g ≈ 0.41) and a pre-registered meta-analysis by Danielson et al. (2024/25, 294 effect sizes). Studying incorrect worked examples alongside correct ones (Booth et al. 2013 and related work) can help, especially learners with low prior knowledge, though a 2025 review calls the evidence on moderators and long-term effects contradictory. Productive failure is a meta-analytic finding that problem-solving before instruction improves conceptual learning. In productive failure, though, learners struggle and generate their own flawed solutions; teachers do not present wrong explanations. The effect also reversed for younger children (grades 2–5) in the reported moderator analysis. On Ada Clara's comfort-with-error rationale: error-management training in adults improves transfer performance (Keith & Frese 2008, d ≈ 0.44). There is little direct evidence that repeated failure drills in school make students more comfortable being wrong or better at scientific reasoning. Patrick's proposal, on its likely reading, is plausible and fits these findings, but it has not been tested as such.
Detail and sources (7)
Who said what. In T018, Patrick says his method is not perfect but is repeatable, and that repeating it drills in the scientific method. He then goes "one step further": teachers would "purposefully give incorrect why with incorrect proof", ask "hey, this didn't work, what does that mean?", and conclude that "we prove that the why was not correct by creating an experiment that failed to prove the outcome it predicted." On its own, "incorrect proof" could mean several things, such as a flawed experiment or an experiment whose result contradicts the prediction. His follow-up sentence makes the second reading the likely one: an experiment that fails to produce the outcome the wrong "why" predicted. This note relies on that reading, but it is an interpretation, and Patrick did not spell out how the "incorrect proof" would be set up. Ada Clara then added several points Patrick did not make. In T019–T020 she framed the exercise as training comfort with being wrong. In T020 she had the class work out what went wrong: an incorrect hypothesis, a badly derived prediction, a poorly designed experiment, or an uncontrolled variable. In T019–T020 she also suggested sometimes using an incorrect explanation that still produces a correct prediction. The research terms below (refutation texts, erroneous examples, productive failure, error-management training) were chosen by this pipeline for comparison. Neither speaker used them.
How the research fits, on the likely reading of Patrick's proposal: (1) Refutation texts are the closest match to "present a wrong explanation, then show why it fails". But they are usually texts read on paper, not experiments students run, and they target misconceptions students already hold, not new ones planted by the teacher. They also differ in another way: a refutation text states the correction explicitly, while in Patrick's proposal (as likely read) the failed experiment and the class's conclusion that the "why" was wrong do that work. Danielson et al. note that earlier positive findings had met some scepticism. Their pre-registered meta-analysis nonetheless found a consistent, significant advantage, and no moderator changed it. (2) Erroneous or incorrect examples: Booth et al. (2013), known here only through secondary citations, and McLaren's group report benefits, especially when incorrect examples are contrasted with correct ones. A 2025 Educational Psychology Review synthesis says findings on prior knowledge, cognitive load and long-term effects are contradictory. (3) Productive failure (Sinha & Kapur 2021, Review of Educational Research, 53 studies, 166 comparisons): problem-solving before instruction came out ahead, with g ≈ 0.37–0.58 when studies closely followed productive-failure design principles. These figures come from search-result summaries, because the full paper could not be opened. The effect reportedly reversed for grades 2–5 and for domain-general skills, which is a caution for using the approach with young children. The mechanism also differs from Patrick's: learners generate the errors themselves. (4) Error-management training (Keith & Frese 2008, 24 studies, adult training settings) bears on Ada Clara's comfort-with-error rationale. It supports the idea that encouraging errors improves transfer, but not specifically in school science. Ada Clara's twist, a wrong explanation that still gives a correct prediction, matches the philosophy-of-science point that a confirmed prediction does not prove a theory. No classroom research testing it was found. A risk to keep in mind: presenting a plausible wrong explanation without a clear refutation could plant a misconception. Refutation research works partly because the correction is explicit. On the likely reading of Patrick's proposal, the refutation comes from the failed experiment and the class drawing the conclusion. He did not say whether the correct explanation would also be given. Unverified: no study we found measures whether such drills, repeated across schooling, change adults' habits of mind, as Ada Clara suggested in T019–T020.
- Refutation Text Facilitates Learning: a Meta-Analysis of Between-Subjects Experiments (Schroeder & Kucera) · Educational Psychology Review / PMC, 2022 · peer-reviewed · supports
Opened. 44 comparisons, n = 3,869, g = 0.41, robust across contexts. Trim-and-fill estimate g = 0.28. Mostly science and math, mostly one-off short texts. Only two studies tested retention beyond a month. - Those who fail productively are all the wiser · Phys.org (ETH Zurich release), 2021 · journalism · supports
Opened. Sinha & Kapur meta-analysis: 53 studies, 166 comparisons favouring problem solving before instruction. Weaker for primary students. Requires prior knowledge and instruction after the failure. Humanities and general skills excluded. - When Problem Solving Followed by Instruction Works: Evidence for Productive Failure (abstract record) · Review of Educational Research (via X-MOL), 2021 · peer-reviewed · qualifies · not retrieved (unverified)
Search snippet only (page failed to load): g = 0.37–0.58 when Productive Failure principles were followed closely. Effects reversed for grades 2–5 and domain-general skills. - The Effectiveness of Incorrect Examples and Comparison When Learning About Decimal Magnitude (Durkin dissertation) · Vanderbilt University, 2012 · primary · qualifies · not retrieved (unverified)
Search snippet: 342 fourth- and fifth-graders. Adding incorrect examples did not clearly outperform studying correct examples only. - Effectiveness of error management training: A meta-analysis (Keith & Frese) · Journal of Applied Psychology, 2008 · peer-reviewed · supports · not retrieved (unverified)
Search snippet: 24 studies, N = 2,183, d = 0.44. Larger effects on transfer (d = 0.56) and adaptive transfer (d = 0.80). Explicit encouragement of errors was an effective element. Adult training context. - Searching for the backfire effect: Measurement and design considerations (Swire-Thompson et al.) · Journal of Applied Research in Memory and Cognition / PMC, 2020 · peer-reviewed · context · not retrieved (unverified)
Search snippet: recent studies often fail to find backfire effects, including familiarity backfire from repeating a myth when correcting it. - Why Minimal Guidance During Instruction Does Not Work · Educational Psychologist, 2006 · peer-reviewed · qualifies · not retrieved (unverified)
Search snippet: argues that minimally guided instruction is less effective for novices than guided instruction.
Relevant to: Teaching with wrong explanations on purpose
R-R06Partly supportedcentral claim · low confidence
Claim by Patrick (conversation #15, #17):
“Schooling that emphasizes what to think produces a more gullible public, more vulnerable to misinformation.”
Patrick's causal claim has not been tested directly. The claim, put with his own hedge ('somehow it seems like school are stuck trying to teach what to think'), is that this kind of schooling creates a 'very gullible public' (T015). The nearby evidence fits the idea but does not establish it. People who score higher on analytical reasoning tell fake headlines from real ones better (Pennycook & Rand). More educated people believe conspiracy theories less, partly through analytic thinking (van Prooijen). Both findings are correlational. Teaching people to resist misinformation is possible, but results depend heavily on the method. A short video course in critical thinking did not improve misinformation detection in a large randomised trial. A Stanford curriculum teaching students to check sources by leaving the page and looking them up elsewhere ('lateral reading') roughly doubled students' scores, yet they still earned only about half the possible points. The Stanford researchers specifically criticise a school habit, reading a text closely and trusting it, as poorly suited to the internet. That partly echoes Patrick's concern, but it is a narrower and different mechanism.
Detail and sources (4)
No study found compares schools that emphasise 'what to think' with schools that emphasise 'how to think' and then measures how gullible their graduates are as adults. The supporting evidence is indirect. (1) Reasoning ability and discernment go together (Pennycook & Rand 2019, online samples), but others, such as Kahan, argue that stronger reasoners can be more polarised, so the 'lazy thinking' explanation remains debated. (2) More education predicts less belief in conspiracy theories, through several mediators: analytic thinking, belief in simple solutions, and feelings of powerlessness. This cuts somewhat against the idea that current schooling breeds gullibility, since more of it is associated with less. (3) Critical-thinking skills correlate with spotting misinformation (r ≈ 0.40), but a brief intervention failed to improve them, and the authors cite reviews suggesting students do not become much better critical thinkers over their education. That could be read as consistent with Patrick's view that schools fail to teach 'how to think', but it is not evidence that teaching 'what to think' causes gullibility. (4) Lateral reading succeeds because it teaches judging sources, not just reasoning in general. Ada's caution in T017, that rejecting every expert can be just as gullible as believing every expert, matches this literature: effective approaches teach when to trust, not blanket scepticism. Ada also pushed back on the premise that schools teach only 'what to think', and Patrick did not resolve that point. Patrick's empirical premise about what schools actually emphasise was not assessed here.
- Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning (Pennycook & Rand) · Cognition, 2019 · peer-reviewed · supports · not retrieved (unverified)
Search snippet: N = 3,446 MTurk workers. Higher Cognitive Reflection Test scores were associated with better telling fake from real headlines, even when the fake stories fit participants' politics. Correlational. - Why Education Predicts Decreased Belief in Conspiracy Theories (van Prooijen) · Applied Cognitive Psychology / PMC, 2017 · peer-reviewed · qualifies · not retrieved (unverified)
Search snippet: higher education predicts lower conspiracy belief. Mediated by analytic thinking, belief in simple solutions and powerlessness. Correlational. - Critical thinking and misinformation: randomized video intervention (Frontiers in Education) · Frontiers in Education, 2025 · peer-reviewed · qualifies
Opened. Critical-thinking scores correlated with identifying misinformation (R = 0.43). The short video intervention did not improve critical thinking or misinformation detection. Cites reviews suggesting students do not become much better critical thinkers over their education. - It doesn't take long to learn how to spot misinformation online, Stanford study finds · Stanford Graduate School of Education, 2022 · institutional · qualifies
Opened. Randomised study of about 500 high-school students. Six lessons on lateral reading roughly doubled scores, but students earned only half the possible points. Criticises schools for teaching close start-to-finish reading, which fails online.
Relevant to: How to think versus what to think
R-R07Supportedcentral claim · medium confidence
Claim by Ada Clara (conversation #22, #23):
“The evidence on inquiry-based, problem-based learning and scientific argumentation is encouraging but depends heavily on implementation and guidance.”
Broadly supported, with an important qualification about guidance. Meta-analyses find that inquiry and discovery approaches work when students get substantial guidance and do worse when they don't. Alfieri et al. (2011) found unassisted discovery underperformed explicit instruction (d ≈ −0.38), while guided ('enhanced') discovery outperformed other methods (d ≈ 0.30). Furtak et al. (2012) found a mean effect of about 0.50 for inquiry-based science teaching, with larger effects for teacher-guided forms. A 2024–25 overview of 326 systematic reviews concluded that inquiry-based instruction 'probably improves learning' and that guidance makes it more effective. The same overview found the evidence for problem-based learning uncertain and the evidence specific to argumentation thin. PISA 2015 analyses by the OECD found that, across many systems, more frequent enquiry-based science teaching went with lower science scores, mostly in disorderly classrooms. Ada's word 'encouraging in many contexts' fits inquiry with guidance. It is generous for problem-based learning and for unguided approaches.
Detail and sources (6)
The publication should present this as a real debate. Kirschner, Sweller & Clark (2006) argued that minimally guided approaches, a category in which they placed discovery, problem-based, experiential and inquiry learning, are less effective than direct guidance for novices. They based this on cognitive load and how novices differ from experts. Hmelo-Silver, Duncan & Chinn (2007) and Schmidt et al. (2007) replied that problem-based and inquiry learning, done properly, are heavily scaffolded and are not 'minimal guidance'. Much of the disagreement therefore turns on definitions: what counts as 'inquiry', and how much guidance it includes. The PISA findings are correlational and come from student surveys. The OECD reports that the negative association is much weaker in disciplined classes, and that enquiry-based teaching is more common in disadvantaged schools, which makes it harder to read the association as cause and effect. The 2024–25 overview by Oxman et al. rated the certainty of evidence as moderate at best and never high. It noted that few reviews measured critical thinking directly, and found argumentation outcomes in only 5 of 326 reviews. Overall, 'depends heavily on implementation and guidance' is well supported. 'Encouraging' is fair for guided inquiry. It does not cover problem-based learning as a whole (results uncertain against conventional teaching) or argumentation (too little evidence). This bears on Patrick's proposal: his repeated hypothesis–prediction–experiment–revision drills look like guided inquiry, and the evidence suggests they would need strong teacher scaffolding to work. Note that this is research context, not a position Patrick took.
- Does Discovery-Based Instruction Enhance Learning? (Alfieri, Brooks, Aldrich & Tenenbaum) · Journal of Educational Psychology 103(1), 2011 · peer-reviewed · supports · not retrieved (unverified)
Search results summarise two meta-analyses covering 164 studies. Unassisted discovery underperformed explicit instruction (d = −0.38); enhanced or guided discovery outperformed other instruction (d = 0.30). The PDF opened but its text could not be extracted. - Why Minimal Guidance During Instruction Does Not Work (Kirschner, Sweller & Clark) · Educational Psychologist 41(2) / Harvard T.H. Chan case resources, 2006 · peer-reviewed · qualifies
Abstract page: argues that minimally guided approaches (discovery, problem-based, experiential, inquiry) are less effective than guided instruction. - Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006) · Educational Psychologist 42(2), 2007 · peer-reviewed · context · not retrieved (unverified)
Search-result abstract: argues that problem-based and inquiry learning use scaffolding extensively, and that Kirschner et al. wrongly grouped them with unguided discovery. - Experimental and Quasi-Experimental Studies of Inquiry-Based Science Teaching: A Meta-Analysis (Furtak, Seidel, Iverson & Briggs) · Review of Educational Research, 2012 · peer-reviewed · supports · not retrieved (unverified)
Search results: 37 studies from 1996–2006, mean effect size about 0.50. The paper separates the cognitive features of an activity from the degree of guidance; secondary summaries report larger effects for teacher-guided inquiry. - The effects of teaching strategies on learning to think critically in primary and secondary schools: an overview of systematic reviews (Oxman et al.) · F1000Research / PMC, 2025 · peer-reviewed · qualifies
Overview of 326 reviews. Inquiry-based instruction probably improves learning, and guidance probably makes it more effective. Problem-based learning is uncertain compared with teacher-led instruction. Argumentation evidence is thin (5 reviews). Certainty is very low to moderate, never high. - The science of teaching science: An exploration of science teaching practices in PISA 2015 (Mostafa, Echazarra & Guillou) · OECD Education Working Papers, 2018 · institutional · qualifies · not retrieved (unverified)
Search results: enquiry-based teaching is negatively associated with science performance, but the association is greatly weakened in disciplined classes; teacher-directed instruction is associated with better scores. The page returned 403 when opened.
Relevant to: Are facts still worth teaching?
R-R08Supportedsupporting claim · high confidence
Claim by Ada Clara (conversation #23, #25, #26, #38):
“Critical evaluation requires sufficient background knowledge, so facts remain essential to reasoning.”
Cognitive science strongly supports the claim that critical evaluation depends on domain knowledge. Daniel Willingham's widely cited synthesis argues that thinking processes are bound up with the content being thought about. On this view, students cannot weigh perspectives or spot implausible claims in a field they know little about, and generic critical-thinking programmes have had limited success. Reading research points the same way. The often-cited 'baseball study' (Recht and Leslie 1988) found that background knowledge predicted comprehension more than general reading ability did, although that study is small and has been replicated only thinly. The cognitive-load literature adds that guidance becomes less necessary only as learners' prior knowledge grows. One qualification matters: knowledge is necessary but not sufficient. Abrami et al.'s 2015 meta-analysis found that explicit critical-thinking instruction, both generic and subject-embedded, has a small-to-moderate positive effect (mean effect size of about 0.30). Reasoning skills can therefore be taught, but they work on top of knowledge, not in place of it. This supports Ada's position that facts remain essential. The idea of facts as 'the raw material of thinking' is Ada's (T023). She read Patrick as arguing for a reversal of priorities rather than for dropping facts (T025, T026). Patrick never explicitly endorsed that reading, and he had called teaching facts 'pretty much useless' (T024). If Ada's reading is right, the finding fits Patrick's priorities. The conversation does not show that it is right.
Detail and sources (4)
Where Patrick stands: in T021 Patrick called facts 'the easiest possible thing to acquire these days', and in T024 he called teaching facts 'pretty much useless', though 'not as useless' as agreeing with any answer. Ada pushed back (T023, T025, T026, T038). She argued that facts are scaffolding, and she suggested that what Patrick was 'really arguing' was a matter of priority: teach reasoning continuously and bring in facts as tools. That is her interpretation. Patrick did not confirm it. The research favours Ada on the narrow point that background knowledge is a precondition for evaluating claims. One possible reading of Patrick's point is that facts are easy to look up, not that they are unnecessary. That reading runs into its own problem in cognitive science: looking something up does not give a person the knowledge they need to judge it, which is a core part of Willingham's argument. Patrick did not concede the point in the conversation, and this finding does not settle what he meant.
Caveats:
(a) Willingham's argument is a synthesis of research and partly a position piece, not a single decisive experiment.
(b) Critics call the baseball study small (64 students) and only lightly replicated. Some also say it does not show that content instruction beats skills instruction.
(c) Abrami et al. show that generic critical-thinking instruction has measurable effects. 'Mixed' approaches (generic principles plus subject content) scored highest at +0.38, but the differences between approaches were not statistically significant. So the strong claim that critical thinking is purely domain-specific is itself debated.
(d) Ada's formulation ('facts as raw material, reasoning as organizing principle') is a reasonable synthesis but not a tested instructional model.
Source access: the Willingham and baseball-study material comes from search results, because the Willingham PDFs were blocked or unreadable.
- Why Content Knowledge Is Crucial to Effective Critical Thinking · KQED MindShift, 2019 · journalism · supports
Quotes Willingham: 'scientists are united in their belief that content knowledge is crucial to effective critical thinking'. Without background knowledge you 'will not comprehend the author's claims in the first place'. Familiarity eases the load on working memory. - Critical Thinking: Why Is It So Hard to Teach? (Willingham) · American Educator / Reading Rockets, 2007 · other · supports · not retrieved (unverified)
Search results: thinking processes are intertwined with domain knowledge, and critical thinking is not a skill that transfers freely. The page returned 403 when opened. - Strategies for Teaching Students to Think Critically: A Meta-Analysis (Abrami et al.) · Review of Educational Research 85(2), 2015 · peer-reviewed · qualifies · not retrieved (unverified)
Search results: weighted mean effect size of 0.30 across 341 effects. Both generic and content-specific critical-thinking instruction can be effective, which qualifies the strong claim that critical thinking cannot be taught apart from content. - Why Minimal Guidance During Instruction Does Not Work (Kirschner, Sweller & Clark) · Educational Psychologist 41(2), 2006 · peer-reviewed · context
Its guidance argument rests on cognitive architecture and on how novices differ from experts. Search excerpts note the central role of long-term memory. The opened page showed only the abstract.
Relevant to: Are facts still worth teaching?
R-R09Supportedsupporting claim · medium confidence
Claim by Ada Clara (conversation #22, #23):
“Inquiry-based teaching is harder to standardize and assess, and needs more instructional time and teacher training.”
Research on why inquiry teaching is hard to implement consistently reports the barriers Ada named. Fitzgerald, Danaia & McKinnon (2019) interviewed 34 early-adopter teachers. They reported severe time pressure, professional development of little value, a lack of clear models and definitions of inquiry, and a lack of quality resources. Other studies report that pressure from national assessments and exam-based curricula dampens teachers' enthusiasm for inquiry. That the assessments rewarded most easily are fact recall rather than the evaluation of reasoning is a widely held view. Most of this evidence, however, comes from teacher perceptions and qualitative studies, not causal research.
Detail and sources (3)
Caveats. (1) Most of the evidence is self-reported by teachers in particular settings (for example, an Australian astronomy programme), so it may not generalise. (2) At least one dissertation suggests that teachers' own intentions, rather than the barriers they perceive, may be what really limits inquiry. That is a minority view. (3) That reasoning is 'harder to assess' is plausible, and fits with the exam pressure documented in the literature, but this search found no direct measurement study comparing how reliable or costly the two kinds of assessment are. (4) Ada later conceded (T025–T026) that implementation difficulty is not a measure of educational value. Whether inquiry's extra demands are worth its benefits is a separate question that the conversation left open, and the research has not settled it either.
- Barriers inhibiting inquiry-based science teaching and potential solutions: Perceptions of positively inclined early adopters (Fitzgerald, Danaia & McKinnon) · Research in Science Education 49(2) / Charles Sturt University, 2019 · peer-reviewed · supports
Interviews with 34 teachers identified four barriers: severe time constraints, professional development of little value, a lack of good models and definitions of inquiry, and a lack of quality resources. - Inquiry-based science teaching in England and Spain (Nottingham repository) · University of Nottingham, unknown · peer-reviewed · supports · not retrieved (unverified)
Search snippet: teachers were frustrated by lack of time, resources and classroom-management difficulties, and their enthusiasm was tempered by national assessment and curriculum expectations. - Inquiry-based learning: Perceptions, use, and barriers for 6-12 science teachers · University of Houston–Clear Lake, 2017 · other · qualifies · not retrieved (unverified)
Search snippet: suggests teacher intent, rather than perceived barriers, may be the stronger limit on inquiry-based learning.
Relevant to: Are facts still worth teaching?
R-R10Supportedsupporting claim · high confidence
Claim by Ada Clara (conversation #27, #28, #29):
“Ada Clara built on Patrick's assembly-line analogy (T027). She said that mass schooling took on industrial characteristics: students grouped in age batches, standardized stages, and periodic inspections. She added that this model solved a real problem of access and scale. She also warned that this historical influence should not be mistaken for proof that schools were deliberately designed as factories.”
Ada Clara's version of the claim fits what historians of education say. This includes her caveat in T029 that we "shouldn't confuse the historical influence of industrialization with proof that schools were deliberately designed as factories." Mass schooling did adopt age-graded classrooms, standardized curricula, examinations and record-keeping. In the early 20th century it also borrowed efficiency ideas from scientific management. It greatly widened access, a point Ada raised in T028 and T029. But historians widely call the popular story of schools deliberately built as factories ahistorical. That story is often told as the US copying a Prussian system to produce docile workers. The phrase "factory model school" itself only dates from about 1970–72. Ada's caveat is therefore warranted. The access-and-scale point and the caution about deliberate design were Ada's additions. Patrick did not take up either one in the conversation. His own statement in T027 was that "we've ended up doing industrialized training" and "we basically decided that we were going to train kids exactly like we manufacture goods on an assembly line." Read as a description or analogy, it holds up. Read as an account of deliberate historical intent, it goes further than the evidence.
Detail and sources (3)
Evidence of industrial influence:
- Ellwood Cubberley wrote in 1916 that "our schools are, in a sense, factories."
- Taylorist efficiency studies reached school administration in the 1910s.
- Age-graded "cells and bells" classrooms became standard.
- The older monitorial (Lancaster/Bell) schools looked even more like factories.
Evidence against deliberate design:
- Free charity schools and compulsory-attendance laws existed before Horace Mann's Prussian-inspired reforms.
- Control of US schools stayed decentralized, unlike Prussia's.
- Prussia was not highly industrialized when its system took shape.
- Factories in Mann's time did not resemble later assembly lines.
- David Tyack notes that 19th-century reformers used "machine" language without today's negative overtones.
- Carl Kaestle argues that schools and factories shared an efficiency ethos; neither simply copied the other.
- Sherman Dorn notes that "the industrial era" is not a single coherent period.
The competing view: some economists, such as Galor and Moav, argue that industrialization created demand for schooling a literate, compliant workforce. Whether that counts as "factory design" depends on how the term is defined.
The assembly line itself (around 1913) came after mass schooling began, so the "assembly line" image is at best a later analogy.
Attribution: the analogy is Patrick's (T027). The list of industrial features, the access-and-scale benefit and the caution about deliberate design are Ada Clara's (T028, T029).
Scope: most of the historical sources here are US-focused, and the history varies by country. In T036 Patrick separately criticized today's education systems worldwide ("universally, with varying degree") as failing to fit a world changed by AI. That is a claim about present-day performance, not about how mass schooling originated. It should not be read as a worldwide historical claim, and this note's research does not test it.
- Factory model school · Wikipedia, 2026 · reference · supports
Summarizes historians: Callahan documents efficiency-era leaders calling children 'raw goods'; Cubberley used the factory metaphor heavily but warned against it; Tyack says 'factory' language lacked today's negative meaning; Dorn and Watters dispute the deliberate factory/Prussian design story. - The Invented History of 'The Factory Model of Education' · Audrey Watters, 2015 · other · qualifies · not retrieved (unverified)
Per search summary: no 1840s usage of 'factory model of education'; the phrase rises after 1970; Mann criticized Prussian authoritarianism. Not opened directly. - Myths about the history of schools · Arizona State University, Future of Learning, unknown · institutional · qualifies · not retrieved (unverified)
Search result relating to Dorn and Gamson's '23 Myths About the History of American Schools', which treats the factory model as a myth. Not opened.
Relevant to: Schooling as an assembly line
R-R11Partly supportedperipheral claim · medium confidence
Claim by Ada Clara (conversation #22, #23):
“Historically, information was scarce and memorization had great practical value; today answers are cheap and checking them is costly.”
This is broadly reasonable as a sweeping generalization, but it needs a caveat. Before printing, and for most people long after it, access to texts, teachers and libraries really was limited and costly. Memory and laborious copying were central to keeping knowledge. But the historian Ann Blair shows that educated Europeans after printing complained of having too much to read. They relied on notes, excerpts and reference books rather than memory alone. So 'scarcity' depends on the era and the social group, and 'information overload' is not new. The point about the present, that answers are cheap while checking them takes effort (made sharper by AI), is a fair description, but this check found no source measuring it.
Detail and sources (2)
Blair's 'Too Much to Know' (2010) documents information overload after the printing press. Scholars managed it with note-taking and compiled reference works, a form of externalized memory that undercuts a simple story of scarcity followed by abundance. For ordinary, mostly non-literate people, access was indeed scarce, and that is the strongest reading of Ada Clara's claim. The claim that verification is now 'expensive' is a judgement call and was not checked quantitatively here. The claim belongs to Ada Clara and only frames the argument; Patrick's point concerns what education rewards.
- Ye old information overload · Harvard Gazette, 2010 · journalism · qualifies
Blair argues information overload predates digital media: after printing, scholars faced too much to read and relied on note-taking and published reference compilations; recording information by hand was laborious. - Too Much to Know: Managing Scholarly Information before the Modern Age · Princeton Alumni Weekly, 2010 · other · qualifies · not retrieved (unverified)
Search result describing the book as examining the 16th–17th century European response to the flood of printed books. Not opened.
Relevant to: Are facts still worth teaching?
R-R12Partly supportedsupporting claim · medium confidence
Claim by Patrick (conversation #33, #34, #35):
“Training in the "more complex perspective" (scientific reasoning) is currently given mainly to scholars, so the ability to innovate and create new knowledge is concentrated among them. If it were taught to everyone, new knowledge would come not just from frontier science but from every field.”
The evidence partly fits Patrick's claim. It also cuts against the premise that knowledge creation is concentrated among scholars, if that premise is read strongly. Research on user innovation, led by Eric von Hippel at MIT, finds that a lot of new knowledge and many new products come from people outside formal research institutions. In a representative UK survey, 6.1% of adults (about 2.9 million people) had created or modified a consumer product in the past three years. Their combined spending on this was more than 1.4 times UK firms' total R&D on consumer products. In some fields users lead: about 80% of 111 scientific instruments studied were invented and prototyped by users, and roughly half of banking services were first developed by users. So knowledge creation is less concentrated among scholars than Patrick's wording in T033 suggests. The idea that practitioners such as farmers, mechanics and nurses already create practical knowledge was a qualification that Ada Clara added (T034–T035), not part of Patrick's claim. Patrick did not take it up directly; in T036 he moved on to school reform. The evidence matches Ada's qualification. Still, formal, systematic and published knowledge creation does remain concentrated in academia and industry R&D. Most user innovations stay local: only about 17% of consumer innovations spread to other people, and they are rarely patented.
Detail and sources (2)
Patrick's point in T033 is mostly about training. He says the "more complex perspective" is taught mainly to scholars, which concentrates the ability to innovate in that group. Teaching it to everyone would spread new knowledge to every field, and he is "certain that every field could benefit from new knowledge." (The transcript says "getting thought" and "if we thought that"; given the context, these are speech-recognition errors for "taught.") Read this way, the user-innovation research fits his argument in one important respect. It shows that people outside research institutions already innovate, which suggests there is untapped capacity. It also shows that technical education (UK) and level of education (US) predict who innovates, which fits the idea that training matters. It does not test his prediction that universal training would produce new knowledge in every field. The observation that practitioners already create knowledge came from Ada Clara (T034–T035). She offered it as a qualification and also noted a practical limit: testing hypotheses can require equipment, expertise and safeguards. Patrick did not address either point, so this note does not treat them as agreed. The figures come mainly from management and economics studies of product innovation, which do not directly measure "new knowledge" in the scientific sense. Research on practice-based knowledge in agriculture (farmer experimentation) and nursing was not examined. Some figures, such as the 54% lead-user share, come from secondary summaries and were not checked against the original papers.
- Comparing Business and Household Sector Innovation in Consumer Products: Findings from a Representative Study in the United Kingdom · Management Science (via RePEc), 2012 · peer-reviewed · qualifies
Abstract: 6.1% of 1,173 UK adults (about 2.9 million people) innovated in consumer products in 3 years; consumer development spending more than 1.4 times UK firms' consumer-product R&D; little IP protection; 17% diffuse. - The Age of the Consumer-Innovator · MIT Sloan Management Review, 2011 · peer-reviewed · supports · not retrieved (unverified)
Search result: multi-country surveys (UK, US, Japan) showing millions of consumers develop products. Not opened.
Relevant to: Everyone as a knowledge creator
R-R13Partly supportedsupporting claim · low confidence
Claim by Ada Clara (conversation #37, #38):
“Ada Clara put this two ways. In her shorter reply (T037) she said it firmly: "AI compresses the shelf life of know-how," adding that AI "doesn't erase foundations." In her longer reply (T038) she hedged: "The useful lifespan of specific technical knowledge may be shrinking." That reply also linked this to AI speeding up how fast professional knowledge and required skills change. Both replies set this against the lasting value of foundational knowledge and of knowing how to learn.”
Good evidence shows that the skills jobs ask for change quickly, especially in technology-heavy fields. The strongest peer-reviewed example is Deming and Noray (2020). Using job-vacancy data, they found that STEM jobs change fast enough that older workers' skills lose value, and that the early pay advantage of applied STEM degrees falls by more than half within a decade. Large employer and platform forecasts also name AI as a main driver of future skill change. The World Economic Forum's 2025 report says 39% of core skills will change by 2030. LinkedIn's 2025 report says 70% of the skills used in most jobs will change by 2030. However, the stronger claim that skill lifespans are shrinking over time, and that AI is already one cause, rests mostly on forecasts and opinion surveys rather than measured long-term trends. The popular 'skill half-life' figures (for example, 'five years') have little independent validation. Ada's two replies differed in strength. Her longer reply (T038) hedged in line with the evidence: the lifespan of specific technical knowledge "may be shrinking." Her shorter reply (T037) was firmer: "AI compresses the shelf life of know-how." Read as a claim about what has already happened, that goes further than the evidence supports.
Detail and sources (3)
What supports it: (1) Deming and Noray measure real changes in occupations' skill requirements between 2007 and 2019 and link them to flatter lifetime pay curves. That shows real-world obsolescence of technology-specific skills, but it covers the period before generative AI and does not show acceleration over time. (2) The WEF Future of Jobs Report 2025 (a survey of more than 1,000 employers) and LinkedIn's Work Change Report (based on job-posting and profile skills) both expect large skill change by 2030 and name AI as a main driver. Both are forecasts, and both come from organisations with an interest in the 'reskilling' story. Notably, the WEF's expected change actually fell from 44% in 2023 to 39% in 2025, which does not fit a simple 'always accelerating' story. Caveats: (a) The 'half-life of knowledge' idea comes from Fritz Machlup (1962). Even its supporters admit there is no guarantee that knowledge declines exponentially, and it is hard to define what counts as 'knowledge'. Commonly cited examples (an engineering degree's half-life falling from 35 years around 1930 to about 10 years in 1960; psychology specialties ranging from 3.3 to 19 years in a 2016 Delphi poll) are old, specific to one field, or based on expert opinion. (b) Search results report that the widely repeated 'five-year skill half-life' traces to popular books and a 2017 WEF opinion piece, and that a 2018 review found it was never validated. I could not open that review myself. (c) Meta-analyses of 'skill decay' measure forgetting when a skill is not used, not loss of market value, so they do not bear on this claim. Still unknown: whether AI has already measurably shortened how long skills stay useful, as opposed to being expected to. No long-run, rigorous measure of this exists yet. On how Ada worded it: the hedged version (T038, "may be shrinking", about "specific technical knowledge") fits the evidence. The firmer version (T037, "AI compresses the shelf life of know-how") reads as a statement of current fact, and the evidence does not yet establish that. In both replies, Ada's accompanying point that foundational knowledge (maths, language, science, history) keeps its value fits the economics literature, which separates technology-specific skills from general ones.
- The Future of Jobs Report 2025 – 3. Skills outlook · World Economic Forum, 2025 · institutional · qualifies · not retrieved (unverified)
Search excerpts: employers expect 39% of workers' core skills to change by 2030, down from 44% in 2023; the 2016 edition said 35%. AI and big data are the fastest-growing skills. This is an employer survey of expectations, not a measurement. The page returned 403 when opened. - Why Half Your Skills Expire Every Few Years · Arpit Bhayani (personal blog), 2024 · other · qualifies
Traces the half-life idea to a 1966 estimate by Thomas Jones that assumed a ten-year half-life for engineering knowledge. Later figures are given as estimates and ranges, without direct measurement. Admits some researchers argue knowledge moves into new contexts rather than decaying. - Skills Half-Life and Continuous Reskilling · Rework, 2025 · other · context · not retrieved (unverified)
Seen in search results only. Reportedly traces the '5-year skills half-life' claim to the 2011 book A New Culture of Learning and a 2017 WEF opinion piece, which shows the figure spread through popular writing rather than research.
Relevant to: A failing system in the age of AI
R-R14Partly supportedcentral claim · medium confidence
Claim by Patrick (conversation #36):
“Patrick holds that the schooling system is "drastically bad" and "not designed for the world we are in today". He says this applies universally, not just locally, though "with varying degree". Against the backdrop of the fast-paced change that AI has introduced, he says education systems throughout the world are at "different degree[s] of failure", but failing everywhere.”
The 'partially-supported' label here reflects only one thing: international data show broad, serious weaknesses in basic literacy and numeracy, which is relevant context for Patrick's concern. It does not mean his universal claim has been corroborated. That claim, that systems everywhere are failing to varying degrees in a world changed by AI, is still unresolved, because the available evidence does not test it directly. PISA 2025 (released September 2026) recorded the lowest OECD averages ever in reading and mathematics: reading fell about 28 points and maths about 22 points between 2015 and 2025, and one in five 15-year-olds was a low performer in all three subjects. The OECD's 2023 Survey of Adult Skills found adult literacy and numeracy declining or stagnating in most participating countries. In low- and middle-income countries, an estimated 70% of 10-year-olds could not read and understand a simple text after the pandemic, up from 57% before it. These figures measure basic skills. They do not measure whether systems are designed for a world changed by AI, or whether they fail to prepare students for one. The declines also mostly began before generative AI, so they say nothing about how AI-driven change relates to school performance. Results vary by country: some systems improved, and several stay high-performing on these tests. Patrick himself framed the failure as happening "to varying degrees". In the conversation, Ada separated two claims (T038). She found the claim that education systems are increasingly misaligned with today's world "quite persuasive". She said she would not accept that every system is failing without first defining what failure means. Patrick did not respond to that distinction, because the conversation ends there.
Detail and sources (3)
Context on basic skills, which bears on Patrick's concern without testing his claim: (1) PISA declines are broad and long-running, starting before COVID and before generative AI. PISA 2022 had already recorded an OECD drop of almost 15 points in maths, described as unprecedented, with declines in 41 of 72 comparable systems. PISA 2025 then showed further falls to record lows. (2) The OECD adult survey shows falling performance among the weakest 10% in most countries, and widening inequality. (3) The global 'learning poverty' figures point to a basic-skills crisis in poorer countries. (4) Results are not uniform. Eleven systems improved in science between 2015 and 2025, for example Türkiye, Lithuania, the Slovak Republic and the UAE. Finland and Denmark improved significantly in adult literacy. Eight countries, including Finland and Singapore, improved in adult numeracy. Systems such as Singapore, Japan, Korea and Estonia stay high-performing. These figures show variation in basic skills. They do not settle whether any system is succeeding or failing at preparing students for a world transformed by AI. (5) PISA 2025 adds a correlational finding that bears on Patrick's broader argument. Students who did not use AI to draft writing assignments outperformed those who did. Among frequent AI users, those who had been taught to judge AI-generated information did better. Heavy use of devices or AI was linked to lower performance. This fits the idea that teaching students to evaluate information matters, but it does not show causation.
Why the core claim stays unresolved: (a) No international test directly measures readiness for a world transformed by AI. The PISA 2025 'Learning in the Digital World' results are not due until May 2027; only a preview has been released. (b) The causes of the declines are disputed. The OECD points to the pandemic, digital distraction and other factors. School design is not the only possible explanation, and the declines began before AI's recent rapid progress. The data therefore cannot show how school failure relates to AI-driven change. Patrick offered that change as the setting for his diagnosis, and the data neither confirm nor refute it. (c) The criterion matters. In T038, Ada proposed a conditional yardstick. If educational success means producing adults who can independently acquire knowledge, evaluate evidence, solve unfamiliar problems and adapt when their existing understanding becomes obsolete, then traditional examinations and graduation rates tell us "surprisingly little" about whether education is succeeding. She argued that, judged against that objective, systems should be assessed on how well they develop independent reasoning, not only on how efficiently they transfer information. This yardstick was Ada's proposal. Patrick did not respond to it, because the conversation ends there. Under this yardstick, basic-skills test scores are limited indicators, so they can neither confirm nor refute the universal claim. In short, the data document widespread underperformance in basic skills, which is relevant context. They do not establish whether education systems everywhere are failing to meet the needs of a world transformed by AI, and they do not rule that out either.
- PISA 2022 Results (Volume I): The State of Learning and Equity in Education · OECD, 2023 · institutional · supports · not retrieved (unverified)
Search excerpts: OECD mean mathematics fell almost 15 points and reading about 10 points between 2018 and 2022, the largest changes on record. The decline is only partly due to the pandemic, and reading and science were already falling. Australia, Japan, Korea, Singapore and Switzerland kept or raised high performance, and 31 systems at least held their mathematics level. The page returned 403 when opened. - Do Adults Have the Skills They Need to Thrive in a Changing World? Survey of Adult Skills 2023 · OECD, 2024 · institutional · supports · not retrieved (unverified)
Search excerpts: literacy declined slightly on average over the past decade. The share of low-skilled adults rose significantly in 11 of 27 countries and fell only in Denmark and Finland. 18% of adults lack basic proficiency in every domain. The page returned 403 when opened. - 70% of 10-Year-Olds now in Learning Poverty, Unable to Read and Understand a Simple Text · World Bank, 2022 · institutional · supports · not retrieved (unverified)
Search excerpts: estimates that 70% of 10-year-olds in low- and middle-income countries cannot understand a simple text, up from a pre-pandemic baseline. Sources disagree on the baseline (53% vs 57%). Seen in search results only.
Relevant to: A failing system in the age of AI