moderndid.honest_did#

moderndid.honest_did(event_study, event_time=0, sensitivity_type='smoothness', grid_points=100, **kwargs)[source]#

Compute sensitivity analysis for event study estimates.

Implements the approach of [1] for robust inference in difference-in-differences and event study designs. This method relaxes the parallel trends assumption by allowing for bounded violations, providing confidence intervals that remain valid under specified deviations from exact parallel trends.

The event-study coefficient vector \(\boldsymbol{\beta}\) admits the causal decomposition

\[\begin{split}\boldsymbol{\beta} = \begin{pmatrix} \boldsymbol{\tau}_{pre} \\ \boldsymbol{\tau}_{post} \end{pmatrix} + \begin{pmatrix} \boldsymbol{\delta}_{pre} \\ \boldsymbol{\delta}_{post} \end{pmatrix},\end{split}\]

where \(\boldsymbol{\tau}\) represents treatment effects and \(\boldsymbol{\delta}\) represents differential trends between treated and comparison groups. The conventional parallel trends assumption imposes \(\boldsymbol{\delta}_{post} = \mathbf{0}\), which this method relaxes by assuming \(\boldsymbol{\delta} \in \Delta\) for a researcher-specified set \(\Delta\).

The smoothness restriction bounds the discrete second derivative of the trend by a constant \(M\)

\[\Delta^{SD}(M) = \big\{\boldsymbol{\delta}: |(\delta_{t+1} - \delta_t) - (\delta_t - \delta_{t-1})| \le M, \forall t \big\}.\]

The relative magnitudes restriction bounds post-treatment violations relative to the maximum pre-treatment violation

\[\Delta^{RM}(\bar{M}) = \big\{\boldsymbol{\delta}: |\delta_{t+1} - \delta_t| \le \bar{M} \cdot \max_{s<0} |\delta_{s+1} - \delta_s|, \forall t \ge 0 \big\}.\]
Parameters:
event_studyAGGTEResult or similar

Event study result object containing influence functions and estimates.

event_timeint, default=0

Event time to compute sensitivity analysis for. Default is 0 (on impact).

sensitivity_type{‘smoothness’, ‘relative_magnitude’}, default=’smoothness’

Type of sensitivity analysis:

  • ‘smoothness’: Allows violations of linear trends in pre-treatment periods

  • ‘relative_magnitude’: Based on relative magnitudes of deviations from parallel trends

grid_pointsint, default=100

Number of grid points for underlying test inversion.

**kwargsAdditional parameters

Additional parameters passed to sensitivity analysis functions:

  • method : CI method (‘FLCI’, ‘Conditional’, ‘C-F’, ‘C-LF’)

  • m_vec : Vector of M values for smoothness bounds

  • m_bar_vec : Vector of Mbar values for relative magnitude bounds

  • monotonicity_direction : ‘increasing’ or ‘decreasing’

  • bias_direction : ‘positive’ or ‘negative’

  • alpha : Significance level (default 0.05)

Returns:
HonestDiDResult

NamedTuple containing:

  • robust_ci: DataFrame with sensitivity analysis results

  • original_ci: Original confidence interval

  • sensitivity_type: Type of analysis performed

References

[1]

Rambachan, A., & Roth, J. (2021). A more credible approach to parallel trends. Review of Economic Studies.

Examples

The honest_did function performs sensitivity analysis on event study estimates to assess the robustness of results to violations of parallel trends. We demonstrate this below with a staggered treatment adoption design.

First, we need to compute an event study. The function requires an event study object that contains the dynamic influence functions, which we obtain by first computing group-time effects and then aggregating them.

In [1]: import numpy as np
   ...: from moderndid import att_gt, aggte, load_mpdta
   ...: from moderndid.didhonest import honest_did
   ...: df = load_mpdta()
   ...: gt_result = att_gt(
   ...:     data=df,
   ...:     yname="lemp",
   ...:     tname="year",
   ...:     gname="first.treat",
   ...:     idname="countyreal",
   ...:     est_method="dr",
   ...:     boot=False
   ...: )
   ...: es_result = aggte(gt_result, type="dynamic")
   ...: print(es_result)
   ...: 
==============================================================================
 Aggregate Treatment Effects (Event Study)
==============================================================================

 Overall summary of ATT's based on event-study/dynamic aggregation:

┌─────────┬────────────┬────────────────────────┐
│     ATT │ Std. Error │ [95% Conf. Interval]   │
├─────────┼────────────┼────────────────────────┤
│ -0.0772 │     0.0200 │ [ -0.1164,  -0.0381] * │
└─────────┴────────────┴────────────────────────┘


 Dynamic Effects:

┌────────────┬──────────┬────────────┬────────────────────────────┐
│ Event time │ Estimate │ Std. Error │ [95% Pointwise Conf. Band] │
├────────────┼──────────┼────────────┼────────────────────────────┤
│         -3 │   0.0305 │     0.0150 │ [-0.0100,  0.0710]         │
│         -2 │  -0.0006 │     0.0133 │ [-0.0364,  0.0352]         │
│         -1 │  -0.0245 │     0.0142 │ [-0.0628,  0.0139]         │
│          0 │  -0.0199 │     0.0118 │ [-0.0518,  0.0119]         │
│          1 │  -0.0510 │     0.0169 │ [-0.0965, -0.0055] *       │
│          2 │  -0.1373 │     0.0364 │ [-0.2354, -0.0391] *       │
│          3 │  -0.1008 │     0.0344 │ [-0.1933, -0.0083] *       │
└────────────┴──────────┴────────────┴────────────────────────────┘

------------------------------------------------------------------------------
 Signif. codes: '*' confidence band does not cover 0

------------------------------------------------------------------------------
 Data Info
------------------------------------------------------------------------------
 Control Group: Never Treated
 Anticipation Periods: 0

------------------------------------------------------------------------------
 Estimation Details
------------------------------------------------------------------------------
 Estimation Method: Doubly Robust

------------------------------------------------------------------------------
 Inference
------------------------------------------------------------------------------
 Significance level: 0.05
 Analytical standard errors
==============================================================================
 Reference: Callaway and Sant'Anna (2021)

Now we can apply the honest DiD sensitivity analysis. We can examine how robust our estimated treatment effects are when we allow for potential violations of the parallel trends assumption. We’ll analyze the on-impact effect (event_time=0) using smoothness restrictions, which bound how much pre-treatment trends can deviate.

In [2]: hd_result = honest_did(
   ...:     event_study=es_result,
   ...:     event_time=0,
   ...:     sensitivity_type="smoothness",
   ...:     m_vec=[0.01, 0.02, 0.03]
   ...: )
   ...: 
   ...: hd_result
   ...: 
Out[2]: 
HonestDiDResult(robust_ci=shape: (3, 5)
┌───────────┬──────────┬────────┬─────────┬──────┐
│ lb        ┆ ub       ┆ method ┆ delta   ┆ m    │
│ ---       ┆ ---      ┆ ---    ┆ ---     ┆ ---  │
│ f64       ┆ f64      ┆ str    ┆ str     ┆ f64  │
╞═══════════╪══════════╪════════╪═════════╪══════╡
│ -0.047947 ┆ 0.023344 ┆ FLCI   ┆ DeltaSD ┆ 0.01 │
│ -0.063082 ┆ 0.032342 ┆ FLCI   ┆ DeltaSD ┆ 0.02 │
│ -0.078015 ┆ 0.039492 ┆ FLCI   ┆ DeltaSD ┆ 0.03 │
└───────────┴──────────┴────────┴─────────┴──────┘, original_ci=OriginalCSResult(lb=np.float64(-0.043111064411112364), ub=np.float64(0.0032474308325936008), method='Original', delta=None), sensitivity_type='smoothness')

The output shows the original confidence interval (which assumes parallel trends hold exactly) and robust confidence intervals under different smoothness bounds. The m_vec parameter specifies different bounds on how much the linear trend can change between consecutive periods. Larger values of \(M\) allow for bigger violations of parallel trends. If the robust CIs remain informative even under reasonable violations, this provides evidence that the results are credible.

On this page