Opening
Host
Welcome back. Today we are on instrumental variables. A school offers a child a place through a lottery. The family celebrates, then chooses another school. Meanwhile, a losing applicant eventually finds a place. Has the lottery become useless for learning whether the school helps?
Economist
It still gives us something valuable: a random offer. An offer and attendance are different events. Keeping that distinction lets us learn from an experiment that people do not follow perfectly. We will also visit a police experiment and a study of family size. Each asks how a random influence can reveal a treatment effect.
Host
What if people ignore their assigned treatment? I want an answer that respects their choices. We cannot simply pretend every school offer became attendance. And dropping the people who changed their minds sounds suspicious.
Economist
By the end of this episode, you should be able to explain why comparing treatment received can restore selection bias. You should also be able to interpret an instrumental variables estimate, including whose effect it measures. Finally, you should know how to check the assumptions and use the method responsibly.
Host
Our mind map follows The Question, The Investigation, and The Resolution. We will keep those names as spoken signposts. The police case comes before the school calculation, then family size takes us beyond a formal lottery.
Economist
You need no diagram or algebra to follow. Listen for a repeating sequence: a random influence changes treatment, and treatment changes an outcome. We will measure each link, ask what could break it, and avoid claiming more than the evidence supports.
The Question
Host
The Question begins with a practical problem. People are free to respond to an assignment. A patient might decline medicine, a lottery winner might choose another school, or an officer might change a planned response. Why is that more than an administrative detail?
Economist
Assignment describes the opportunity or instruction someone receives. Received treatment describes what actually happens. If those differ, an experiment still identifies the effect of assignment. But that effect need not equal the effect of receiving treatment. An offer that nobody accepts cannot tell us what attending the school would do.
Host
My prediction is that we can rescue the experiment by keeping only people who followed instructions. They received exactly what researchers intended. Would that leave a cleaner comparison?
Economist
That prediction fails because following instructions can be a choice related to outcomes. A family may accept a school offer because it expects unusually large benefits. An officer may reject advice and make an arrest because the incident seems dangerous. Removing those decisions from the data removes the protection supplied by random assignment.
Host
So the naive comparison sorts people by what they received. It might compare students who enrolled with students who stayed away. What makes the resulting difference selection bias?
Economist
Selection bias means the groups would have differed even under the same treatment. More motivated families might support learning regardless of school. More dangerous incidents might produce repeated violence regardless of police action. A difference between received-treatment groups mixes those preexisting differences with any actual treatment effect.
Host
Could we fix that by controlling for everything we observe? We have already discussed regression as a way to compare people with similar recorded characteristics. Why introduce another method?
Economist
Regression can help when the important confounding differences are observed and handled appropriately. But motivation, expectations, and incident severity can remain partly unmeasured. Instrumental variables uses a different route. It isolates a source of treatment variation that does not share those hidden differences, if its assumptions hold.
Host
Let us build the vocabulary before calculating. What is the instrument, and how is it different from treatment or outcome? I would like to keep the school example attached to each term.
Economist
The instrument is the outside influence we use to move treatment. In a school lottery, it is receiving an offer. Treatment is attending the school. The outcome is the later test score. Instrumental variables, shortened to IV, uses offer-driven attendance changes rather than all differences in attendance.
Host
And the first stage sounds like an early step in that process. Does it describe the school improving scores, or the offer changing school attendance?
Economist
The first stage measures how much the instrument changes treatment. Here, it compares attendance rates by offer status. The reduced form instead measures how much the instrument changes the outcome. Here, it compares later scores by offer status. Those comparisons both preserve the lottery; neither sorts students by their enrollment choices.
Host
Then we will divide an outcome change by a treatment change. What should I call the answer? And why does the map introduce different kinds of people?
Economist
The answer can be a local average treatment effect, shortened to LATE. Local means it describes people whose treatment changes because of this instrument. We call them compliers. Always-takers receive treatment either way, never-takers receive it neither way, and defiers respond in the opposite direction. We will unpack those possibilities carefully.
Host
Let me recap The Question. An assignment can be random even when treatment received is not. Comparing actual recipients can bring hidden differences back. We need to keep the random influence and learn how strongly it changes treatment.
Economist
Exactly. Remember the first distinction before remembering any abbreviation: what someone was offered, what they received, and what happened afterward. The first stage and reduced form connect those events. Their causal meaning depends on assumptions we are about to investigate.
The Investigation
Host
The Investigation starts with non-compliance in Minneapolis. This is a serious setting: police responses to domestic violence. What decisions did the experiment study, and what did it leave outside its scope?
Economist
Participating officers encountered eligible suspected misdemeanor domestic assaults. The experiment excluded severe or life-threatening injury cases. Both the suspect and victim had to be present. A randomly prepared report form instructed the officer to arrest, advise, or separate. Separation meant ordering the suspect away temporarily. Advice could include mediation.
Host
So an arrest instruction was not the same thing as an arrest. I predict officers were more willing to override the gentler instructions when they thought someone remained at risk. Is that the pattern?
Economist
That prediction matches the reported pattern. About 99 percent of arrest assignments led to arrest. The matching response occurred in about 78 percent of advice assignments and 73 percent of separation assignments. Officers sometimes arrested despite another instruction, including when a suspect became aggressive. That made actual response partly a selected decision.
Host
The deviations matter even if they were sensible in the moment. Random assignment did not require officers to ignore danger. But the analysis must recognize that the cases moved into arrest could already have had worse prospects.
Economist
Right. We should not confuse an imperfectly followed research protocol with a useless study. The random instruction remains available. The outcome was a later suspected domestic assault at the same address within six months. We will ask whether the instruction changed that outcome through its effect on police response.
Host
Now for the tool: a causal chain reaction. We have an assignment, then an action, then an outcome. How does IV use that sequence without pretending that officers made random decisions?
Economist
It begins by comparing everyone according to assignment. First, measure how much assignment changes the chance of receiving treatment. Next, measure how much assignment changes the outcome. If the instrument affects the outcome only by changing treatment, the second comparison runs through the first. Dividing separates the treatment effect from the strength of the nudge.
Host
That division is sometimes called the Wald ratio. I want the direction clear. We divide the reduced form by the first stage. We do not divide by the share of people who happened to obey a particular instruction.
Economist
Correct. The first stage is a difference in treatment rates across instrument groups. With no treatment recipients in the control group, that difference equals uptake in the offered group. With crossover in both directions, it does not. The subtraction matters because some people would receive treatment without the offer.
Host
The map also mentions an older problem: separating supply from demand. How did that lead to this method?
Economist
Philip Wright developed IV in work published in 1928. Market prices and quantities reflect both supply and demand moving together. A supply shifter that does not directly shift demand can help trace demand. That early insight has the same core logic: find an outside change with a defensible single channel.
Host
What makes that channel defensible? A variable that predicts treatment is easy to find. Family income might predict school attendance very well, yet it also relates to test scores in many other ways.
Economist
That brings us to identifying assumptions. Identification means connecting an observed comparison to the causal effect we want. Relevance comes first: the instrument must change treatment. A relationship with no real treatment movement cannot identify the effect. But relevance alone says nothing about whether the movement is causally useful.
Host
Independence is the next condition. Does random assignment supply it automatically, or are there details we still need to check?
Economist
Independence means the instrument is unrelated to the potential outcomes being compared, possibly after suitable controls. A properly run lottery supports that condition within its randomization groups. If different applicant groups face different winning chances, compare like groups or control for those groups. Checking baseline balance can reveal problems, but balanced observables do not prove independence.
Host
And exclusion requires the offer to affect scores only through attendance. Suppose winning also changes what parents do outside school, even when the child never attends. That would be a second route.
Economist
Yes. That hypothetical route would threaten exclusion if it changed outcomes outside the defined treatment. Random assignment of an offer does not eliminate that problem. It makes the offer effect credible, but calling the scaled result an attendance effect requires the additional channel assumption. No universal statistical test can establish exclusion.
Host
Should we also worry about one student affecting another student, or different people receiving very different versions of treatment? The school experience could involve many ingredients.
Economist
We need a clear treatment definition and a stable comparison. If one applicant's offer changes another applicant's learning through an unmodeled spillover, the simple chain needs more work. A school attendance effect can include its normal curriculum and environment. It does not separately identify the contribution of each ingredient in that school package.
Host
Who LATE speaks for is the next question. A complier sounds like someone we can identify by checking whether they obeyed. Is the label really that simple?
Economist
No. A complier is defined by two possible responses to the instrument. The child would attend when offered a seat and would not attend without one. We only observe one of those possibilities for any child. Therefore, observing a winner enroll does not reveal whether that child is a complier or an always-taker.
Host
Let me try the other types. An always-taker would find a place whether the lottery offers one or not. A never-taker would stay away either way. What exactly is a defier?
Economist
A defier would attend without an offer but stay away because an offer arrived. The monotonicity assumption rules out that opposite response. It allows the instrument to leave some people unchanged. It requires that, when the instrument changes treatment, it pushes in a common direction. It is about counterfactual responses, not moral character.
Host
Could observed crossovers prove that defiers exist? A losing applicant who enrolls looks as if they are doing the opposite of assignment.
Economist
That person could instead be an always-taker. To be a defier, they would also have to refuse attendance if offered a seat. We cannot observe that alternative. Crossover patterns can inform our judgment, but they cannot reveal everyone's type. Under independence and monotonicity, the first-stage treatment difference gives the complier share for a binary treatment.
Host
Why does the ratio single out compliers rather than everyone who attends? I understand that the instrument moves them, but how do the unchanged groups disappear?
Economist
Always-takers and never-takers receive the same treatment under either assignment. With exclusion, their outcomes therefore do not change because of assignment. The reduced form comes from compliers' treatment changes. Dividing by their share turns that overall outcome change into their average treatment effect. Without monotonicity, opposite responses can spoil this simple interpretation.
Host
Now return to the Minneapolis case study. The map calls the combined non-arrest response coddling. That is a loaded word. What does it mean in this calculation?
Economist
Here it is only the study's shorthand for advice or separation together. We will say non-arrest response as well. Treatment means that combined response, and the comparison is arrest. The instrument is assignment to the combined response rather than assignment to arrest. Reversing the treatment coding would reverse the sign, so keep this definition fixed.
Host
Assignment to non-arrest increased actual non-arrest by about 79 percentage points. That sounds strong. What are the less rounded inputs before we examine repeat violence?
Economist
The probability of a non-arrest response was 0.797 when assigned to that response. It was 0.011 when assigned to arrest. Subtracting gives a first stage of 0.786. Those are probabilities, so the difference describes treatment movement. It is not yet a difference in repeat violence.
Host
For the worked calculation, the reduced form is 0.114: assignment to non-arrest raised the probability of a later suspected assault by that amount. We now divide 0.114 by 0.786. What should the result mean?
Economist
The ratio is about 0.145, or 14.5 percentage points. Under independence, exclusion, and monotonicity, receiving the non-arrest response increased repeat violence by that amount for complier incidents. Equivalently, arrest reduced it relative to that combined alternative for these incidents. This is a difference in probabilities, not a 14.5 percent relative increase.
Host
Why is the naive received-treatment gap only 8.7 percentage points? It compares the same broad responses. I would have expected merely a noisier version of the same answer.
Economist
The groups differ before treatment. Officers were more likely to arrest especially aggressive suspects even when assignment called for another response. Those arrested cases could have higher repeat-violence risk anyway. That selection makes arrest look less protective in the naive comparison. Using random assignment removes this particular selection mechanism, subject to the IV assumptions.
Host
Does the local effect equal the effect on everyone who received the non-arrest response? This seems close to an experiment where only people offered treatment could receive it.
Economist
Almost. There were very few non-arrest responses among cases assigned to arrest, so there were almost no always-takers of the non-arrest treatment. With exactly no always-takers, treated people are compliers, and LATE equals the average effect on the treated. Here that is an approximation. It does not identify the effect for incidents officers would arrest under either assignment.
Host
The policy implication seems important but bounded. This study supported taking arrest seriously. It did not randomly compare every possible police response for every kind of domestic violence.
Economist
Exactly. The findings concern eligible incidents, participating officers, the measured follow-up outcome, and the combined alternative to arrest. Serious injuries were outside the experiment. Generalizing to different places or practices needs evidence and judgment. Identifying a credible local effect does not automatically settle every policy question.
Host
Next comes the KIPP Lynn charter lottery. KIPP means Knowledge Is Power Program. What made this school useful for causal learning, beyond the fact that families wanted to attend?
Economist
KIPP Lynn was a public charter school with a structured environment and extended instruction time. Demand exceeded available places after 2005, and admissions lotteries supplied random offers. Compare applicants who won and lost, rather than comparing KIPP students with all nearby children. Baseline characteristics and earlier scores looked reasonably balanced by offer status. The school could be chosen by motivated families, yet the offer comparison within the applicant pool remained useful.
Host
The first stage is 0.741. That means an offer raised attendance by 74.1 percentage points. The reduced form for math is 0.355 standard deviations. What does a standard deviation add to our interpretation?
Economist
A standard deviation measures the spread of scores in the reference population. Expressing a gain in those units puts different test scales on a common footing. It does not mean a percentage of questions answered correctly. Dividing the math offer effect by the attendance first stage gives about 0.48 standard deviations for lottery compliers.
Host
That is a large math gain over one school year. Should we describe it as the effect on all KIPP children, or on any child who might be placed in any charter school?
Economist
Neither follows from this design. It is an attendance effect for applicants whose enrollment changed with the KIPP Lynn offer. The analysis conditions on lottery groups with comparable offer chances, and uses applicants with the necessary test data. Similar rates of complete data across winners and losers support the comparison. The reading result was less precise and did not establish a clear gain.
Host
We have also moved from the raw admissions story to a particular analysis sample. So we should not mix headline enrollment counts with the adjusted first stage used for the score estimate.
Economist
Right. The published score comparison accounts for application groups, including year, grade, and sibling-related differences in offer chances. Its first stage belongs with its reduced form. This is a general habit: keep the outcome, sample, controls, and treatment definition aligned when taking an IV ratio. Otherwise a correct division can still answer the wrong question.
Host
For school policy, lottery compliers sound relevant to offering more places. But they need not resemble students whose families would never apply or children who would attend regardless.
Economist
That is the external validity question: will an estimate travel to other people or settings? Compliers depend on the instrument, and effects can differ across students. The lottery result does not isolate longer hours from discipline or teaching practices. Comparing similar designs across populations can help judge whether the overall attendance effect travels.
Host
Our final case study asks whether extra siblings reduce children's schooling. Parents might spread time and money more thinly as families grow. What is the quantity-quality argument here?
Economist
The phrase describes a possible tradeoff between the number of children and investment in each child. Quality here means outcomes such as schooling, not a judgment about anyone's value. A negative association is not enough. Parents choosing larger families also differ in education, resources, and preferences, which may affect their children independently.
Host
The Israeli study follows Jewish adult firstborns from families with at least two children. Ordinary least squares, or OLS, finds about minus 0.15 years of schooling per extra sibling with controls. What variation can challenge a causal reading of that association?
Economist
One instrument is twins at the second birth. A twin birth can unexpectedly increase completed family size. Without controls, the first stage is about 0.32 children. It is less than a whole child because many families would have had another birth anyway. The instrument changes eventual family size only for some families, even though every twin birth adds two babies immediately.
Host
The other instrument uses the sex of the first two children. If parents prefer a mixture, a same-sex pair can encourage another birth. How large is that shift?
Economist
The unadjusted first stage is about 0.08 children. Firstborns with same-sex rather than mixed-sex siblings therefore came from slightly larger families on average. The corresponding schooling differences were close to zero. That contrasts with the negative ordinary regression association, but a near-zero estimate is not proof that effects are exactly absent.
Host
These are naturally occurring influences rather than offers drawn by researchers. What assumptions deserve special scrutiny before we treat either one as a useful instrument?
Economist
Twinning is related to maternal age and other characteristics. Fertility treatment can complicate its interpretation in other samples too. Controls can address observed differences; they do not guarantee independence. For sibling sex composition, exclusion could fail if bedroom sharing or other household arrangements change schooling directly. Calling an instrument natural does not make its channel automatically valid.
Host
Can we learn anything about exclusion without pretending to test it directly? The original discussion looks at groups whose family size barely responds to the instruments.
Economist
Yes. If there is essentially no first stage in a group, the proposed family-size channel predicts essentially no reduced form there. An outcome response without treatment movement would raise concern about another channel. The study found no clear such warning in its checks. That supports the argument but cannot prove exclusion, particularly when estimates are imprecise.
Host
When the study combines instruments with controls, the schooling estimate is about 0.24 years per extra sibling, with a standard error of 0.13. How should a listener interpret that positive estimate?
Economist
A standard error describes sampling uncertainty. The estimate is not precise enough to establish a positive schooling effect by conventional standards. Its uncertainty is nevertheless hard to reconcile with the very precise negative ordinary regression estimate. The evidence weakens the claim that extra siblings cause the observed schooling disadvantage in this population. It does not prove universal harmlessness.
Host
And family size counts children instead of switching a treatment simply on or off. Does local still mean something when the instrument changes the size of a family?
Economist
Yes, with a more detailed interpretation. The estimate averages effects of family-size changes induced by these instruments, across affected families and child-number margins. It need not represent every possible extra sibling in every household. Twins and sex composition can move different families. Pooling them is most persuasive when their separate findings tell a compatible story.
Host
The inference toolkit now gives the computation a name: two-stage least squares. We have been dividing effects. What do the two regressions do, especially when controls or multiple instruments enter?
Economist
In the first stage, predict treatment from the instruments and the chosen controls. In the second stage, relate the outcome to that predicted treatment, keeping the same controls. With one instrument, the result matches the covariate-adjusted reduced form divided by the adjusted first stage. Multiple instruments can combine useful treatment variation and improve precision when their causal interpretations are compatible.
Host
So the predictions are not a machine learning goal in themselves. They select the part of treatment variation tied to the instruments, after accounting for the controls. What can go wrong with doing both regressions manually?
Economist
The coefficient can be correct while the reported standard error is wrong. Ordinary second-stage regression treats the fitted treatment as if it were the observed treatment. Proper IV software calculates prediction errors using actual treatment, rather than fitted treatment alone. A prediction error is the observed outcome minus the model's prediction. The uncertainty calculation therefore differs from an ordinary second-stage regression. You do need dedicated IV software, such as the estimator used in the companion notebook.
Host
What is a weak instrument? I hear people use small first stage as if it were a complete diagnosis. But a small treatment shift might be measured very precisely in a large study.
Economist
Weakness concerns how clearly the data distinguish the treatment movement from noise, not merely its size in familiar units. The first-stage F-statistic summarizes evidence that the instruments predict treatment, conditional on controls. Below 10 is a traditional warning rule, especially in the chapter's discussion of many weak instruments. Above 10 is not a universal guarantee.
Host
Why might collecting many weak instruments make the answer look reassuringly similar to ordinary regression? That sounds like agreement, which I usually welcome.
Economist
Many weak instruments can overfit the first stage. In finite samples, this can pull two-stage estimates toward the ordinary regression result. Agreement then does not show that selection bias was absent. An alternative called limited information maximum likelihood is less vulnerable to this many-instrument bias. Comparing estimates and inspecting reduced forms can reveal fragility; no estimator repairs a false exclusion assumption.
Host
Let me recap The Investigation. We retain the random influence, check its treatment movement, and scale its outcome effect. Police response, school attendance, and family size all use that reasoning, while the credible population and uncertainty differ.
Economist
And choosing a design comes before choosing software. A well-run randomized trial identifies an assignment effect, and under full compliance can identify the treatment effect for its study population. Ordinary regression needs a defensible treatment comparison after observed controls. IV needs valid outside variation and usually identifies a local effect. The method's name never replaces its assumptions.
The Resolution
Host
The Resolution begins with a devil's advocate claim. Once we have a random instrument, surely the resulting treatment effect belongs to everyone. My prediction is that a larger sample will eventually make that generalization valid.
Economist
That prediction confuses precision with population coverage. More observations can sharpen the local estimate without changing whose treatment responds. Always-takers and never-takers still supply no instrument-driven treatment changes. Extending the estimate to them requires additional assumptions or evidence. A precisely estimated complier effect remains a complier effect.
Host
The second claim is that a huge sample cures a weak instrument. Our map warns against that reassurance. How should we state the warning without making sample size sound irrelevant?
Economist
A large sample does not guarantee a strong design. With many weak instruments, finite-sample bias can remain serious. A valid instrument with a fixed, nonzero treatment effect can become more informative as the sample grows. So it is too strong to say that size can never help. Inspect the actual first-stage strength and the inference procedure rather than trusting the observation count.
Host
A small reduced form also needs care. It might mean the treatment has little effect, or it might reflect a small first stage. We need both links before deciding what the ratio says.
Economist
Exactly. Read the first stage and reduced form separately. If treatment barely moves, the ratio can be unstable and ordinary confidence intervals misleading. If treatment clearly moves but outcomes hardly respond, the evidence more directly challenges a large treatment effect in that setting. Even then, assess the uncertainty before declaring an exact zero.
Host
Let us collect the takeaways in the map's order. Comparing treatment received can revive selection bias. The IV ratio divides the reduced form by the first stage. We also need relevance, independence, exclusion, and monotonicity for the familiar local interpretation.
Economist
Use two-stage least squares through proper IV software, and examine the first-stage diagnostic. KIPP Lynn provided evidence of substantial math gains for lottery compliers. The family-size experiments challenged a strong negative causal account of schooling differences. Their conclusion is absence of convincing schooling harm in those comparisons, with uncertainty and local limits, rather than proof for all families.
Host
The thesis is a useful sentence to carry away: a random nudge, scaled by how far it moves treatment, reveals the causal effect for those it moves. What condition should I mentally attach before repeating it?
Economist
Attach the assumptions. The nudge must be independent of the underlying outcome differences, move treatment, and affect the outcome only through treatment. For the standard complier interpretation, it must not push some people the opposite way. That sentence describes a justified causal chain, not a shortcut from any convenient correlation.
Host
To recap The Resolution, local is a description of who we learned about, not an apology for the study. Strong first stages and credible channels matter more than an impressive sample count. Honest uncertainty belongs in every practical conclusion.
Economist
And a useful policy answer can still be local. We learn what changed for the people affected by a real offer or assignment. The next task is to ask whether those people and that treatment match the policy under consideration. Generalization is another question to investigate, not an automatic reward for obtaining a causal estimate.
Wrap-up
Host
Return to our learning goals. I can now explain why received-treatment comparisons can restore selection bias. I can interpret the ratio and identify its complier population. And I can ask whether the instrument and the uncertainty calculation deserve trust.
Economist
The core thesis returns: a random nudge, scaled by its treatment movement, identifies an average effect among those moved, when the assumptions hold. Assignment and receipt are different. The first stage links them, and the reduced form links assignment to outcomes. The ratio is useful because the causal argument earns it, not because division is difficult.
Host
Here is our closing think question. A new school lottery raises attendance, but merely winning also causes families to buy tutoring even if their children never attend. Can we interpret the usual IV ratio as the effect of school attendance? Pause the episode and think before continuing.
Economist
No. The tutoring response creates an outcome channel outside school attendance. The offer comparison can still identify the effect of winning the lottery. But its ratio to the attendance effect mixes channels and need not identify an attendance effect. A stronger first stage or a larger sample cannot repair that exclusion failure.
Host
For practice, open the Colab notebook and the lecture slides. Rebuild the police first stage and the ratio, keeping assignment separate from action. Then compare the school and family-size assumptions. Ask which channel you believe and which population each estimate describes.
Economist
In Chapter 4, Regression Discontinuity Designs, the next question is whether legal access to alcohol raises death rates. A policy threshold will provide a new way to construct a credible comparison. Keep today's habit: first understand what changes, then ask why that change can reveal causation. Thank you for listening, and goodbye.