Pausal Zivference
@pausalz.bsky.social
Paul Zivich, Assistant (to the Regional) Professor
Computational epidemiologist, causal inference researcher, amateur mycologist, and open-source enthusiast.
https://github.com/pzivich
#epidemiology #statistics #python #episky #causalsky
Computational epidemiologist, causal inference researcher, amateur mycologist, and open-source enthusiast.
https://github.com/pzivich
#epidemiology #statistics #python #episky #causalsky
The idea is we can divide our parameter into two pieces: the mean in the positive region and the mean in the negative region. Here, X* denote positivity (1: yes, 0: no)
October 17, 2025 at 2:34 PM
The idea is we can divide our parameter into two pieces: the mean in the positive region and the mean in the negative region. Here, X* denote positivity (1: yes, 0: no)
These are the estimating functions for the setup, with R: assigned treatment with 1 as new, A: adherence, L: bespoke IV, Y: outcome
The random variables with hats are the model predicted versions
The random variables with hats are the model predicted versions
October 17, 2025 at 12:20 AM
These are the estimating functions for the setup, with R: assigned treatment with 1 as new, A: adherence, L: bespoke IV, Y: outcome
The random variables with hats are the model predicted versions
The random variables with hats are the model predicted versions
The following is the estimating function for two relatively simple linear regression models.
Here, \beta_1 is the average causal effect (since we are in the special case of linear models without interactions).
Here, \beta_1 is the average causal effect (since we are in the special case of linear models without interactions).
October 15, 2025 at 6:38 PM
The following is the estimating function for two relatively simple linear regression models.
Here, \beta_1 is the average causal effect (since we are in the special case of linear models without interactions).
Here, \beta_1 is the average causal effect (since we are in the special case of linear models without interactions).
The following is the kind of structure that proximal causal inference can help us address
We have an unmeasured var, U, that is a confounder. So it seems like we are doomed for identification
But not with proximal causal inference! We can leverage the pair of proxies (Z,W)
We have an unmeasured var, U, that is a confounder. So it seems like we are doomed for identification
But not with proximal causal inference! We can leverage the pair of proxies (Z,W)
October 15, 2025 at 6:38 PM
The following is the kind of structure that proximal causal inference can help us address
We have an unmeasured var, U, that is a confounder. So it seems like we are doomed for identification
But not with proximal causal inference! We can leverage the pair of proxies (Z,W)
We have an unmeasured var, U, that is a confounder. So it seems like we are doomed for identification
But not with proximal causal inference! We can leverage the pair of proxies (Z,W)
This is a quick sketch of the concept behind vertical and horizontal stacking in the context of using g-computation to estimate the parameters of a marginal structural model
October 14, 2025 at 6:25 PM
This is a quick sketch of the concept behind vertical and horizontal stacking in the context of using g-computation to estimate the parameters of a marginal structural model
I forgot to include the visualization for forward-mode autodiff...
Essentially, we break everything into functions and evaluate those functions using those object pairs from before. That gives us the derivative of the function at a particular point
Essentially, we break everything into functions and evaluate those functions using those object pairs from before. That gives us the derivative of the function at a particular point
October 13, 2025 at 1:23 PM
I forgot to include the visualization for forward-mode autodiff...
Essentially, we break everything into functions and evaluate those functions using those object pairs from before. That gives us the derivative of the function at a particular point
Essentially, we break everything into functions and evaluate those functions using those object pairs from before. That gives us the derivative of the function at a particular point
Numerical approximation does what it sounds like it does
The computer approximates what the slope would be at a particular point. We do this by evaluating the function at two points (depending on what approximation method we use). Then we do 'rise over run' to compute the slope (and approx dx)
The computer approximates what the slope would be at a particular point. We do this by evaluating the function at two points (depending on what approximation method we use). Then we do 'rise over run' to compute the slope (and approx dx)
October 13, 2025 at 1:21 PM
Numerical approximation does what it sounds like it does
The computer approximates what the slope would be at a particular point. We do this by evaluating the function at two points (depending on what approximation method we use). Then we do 'rise over run' to compute the slope (and approx dx)
The computer approximates what the slope would be at a particular point. We do this by evaluating the function at two points (depending on what approximation method we use). Then we do 'rise over run' to compute the slope (and approx dx)
For the example function I gave previously, we can see the derivative is a bit of a beast. It has lots of terms (thanks to the chain rule)
October 13, 2025 at 1:21 PM
For the example function I gave previously, we can see the derivative is a bit of a beast. It has lots of terms (thanks to the chain rule)
As you might recall from a calculus class, the derivative can be thought of the slope of a line at a particular point. You also might remember that are a multitude of rules for computing a derivative
Below is a weird function we will do the derivative of
Below is a weird function we will do the derivative of
October 13, 2025 at 1:21 PM
As you might recall from a calculus class, the derivative can be thought of the slope of a line at a particular point. You also might remember that are a multitude of rules for computing a derivative
Below is a weird function we will do the derivative of
Below is a weird function we will do the derivative of
Give it a read, there are some fun visualization in there, like this one
October 9, 2025 at 3:34 PM
Give it a read, there are some fun visualization in there, like this one
Now the trick is that the code it output is actually wrong in a subtle way. I actually hadn't noticed the error it introduce because it won't output an error. The red boxes are the error.
I won't get into the finer details, but this causes the whole procedure to under-estimate the variance
I won't get into the finer details, but this causes the whole procedure to under-estimate the variance
October 9, 2025 at 3:05 PM
Now the trick is that the code it output is actually wrong in a subtle way. I actually hadn't noticed the error it introduce because it won't output an error. The red boxes are the error.
I won't get into the finer details, but this causes the whole procedure to under-estimate the variance
I won't get into the finer details, but this causes the whole procedure to under-estimate the variance
and like the summary was fine. It was very basic and cursory, but no errors
It didn't highlight that I had code provided as part of the paper. So, I followed up by asking it for code. It generated a new example (using the details from the paper example)
It didn't highlight that I had code provided as part of the paper. So, I followed up by asking it for code. It generated a new example (using the details from the paper example)
October 9, 2025 at 3:05 PM
and like the summary was fine. It was very basic and cursory, but no errors
It didn't highlight that I had code provided as part of the paper. So, I followed up by asking it for code. It generated a new example (using the details from the paper example)
It didn't highlight that I had code provided as part of the paper. So, I followed up by asking it for code. It generated a new example (using the details from the paper example)
Aptos is such a bad font, let along one to set as the default...
September 18, 2025 at 4:52 PM
Aptos is such a bad font, let along one to set as the default...
I do this with the following line from above. What is does is it takes the original risk set matrix (which includes events in their interval) and then I subtract off a matrix that indicates where the events happened in time
That zeroes out the contributions to the pooled logit model for events
That zeroes out the contributions to the pooled logit model for events
September 9, 2025 at 12:29 PM
I do this with the following line from above. What is does is it takes the original risk set matrix (which includes events in their interval) and then I subtract off a matrix that indicates where the events happened in time
That zeroes out the contributions to the pooled logit model for events
That zeroes out the contributions to the pooled logit model for events
Okay so now we can look at how the elements from the EF are constructed. We construct 2 separate time design matrices (one for censoring times, the other for event times)
Here is where we exclude the events from contributing to the model (for the rows that correspond to the event times)
Here is where we exclude the events from contributing to the model (for the rows that correspond to the event times)
September 9, 2025 at 12:29 PM
Okay so now we can look at how the elements from the EF are constructed. We construct 2 separate time design matrices (one for censoring times, the other for event times)
Here is where we exclude the events from contributing to the model (for the rows that correspond to the event times)
Here is where we exclude the events from contributing to the model (for the rows that correspond to the event times)
For IPCW, we need to ensure they are aligned with the events times. To do this, I do a loop over the matrices and the unique event times. From the IPCW matrix, I create a new matrix that assigns the weights for each corresponding time by looking up the nearest time
(there is likely a better way)
(there is likely a better way)
September 9, 2025 at 12:29 PM
For IPCW, we need to ensure they are aligned with the events times. To do this, I do a loop over the matrices and the unique event times. From the IPCW matrix, I create a new matrix that assigns the weights for each corresponding time by looking up the nearest time
(there is likely a better way)
(there is likely a better way)
I'll skip ahead to defining the EF, so we can see the forest a bit before looking at trees
We fit a pooled logit model (given some input design matrices) and then generate predictions. To lag, we simply add a row of 1's at the top of the predicted Pr of uncensored and drop the last row
We fit a pooled logit model (given some input design matrices) and then generate predictions. To lag, we simply add a row of 1's at the top of the predicted Pr of uncensored and drop the last row
September 9, 2025 at 12:29 PM
I'll skip ahead to defining the EF, so we can see the forest a bit before looking at trees
We fit a pooled logit model (given some input design matrices) and then generate predictions. To lag, we simply add a row of 1's at the top of the predicted Pr of uncensored and drop the last row
We fit a pooled logit model (given some input design matrices) and then generate predictions. To lag, we simply add a row of 1's at the top of the predicted Pr of uncensored and drop the last row
The pooled logit algorithm in the pre-print thus needs to be modified so both of these are incorporated
The following is some Python code that does this process. First, is the setup
The following is some Python code that does this process. First, is the setup
September 9, 2025 at 12:29 PM
The pooled logit algorithm in the pre-print thus needs to be modified so both of these are incorporated
The following is some Python code that does this process. First, is the setup
The following is some Python code that does this process. First, is the setup
Okay so I am wrong somewhere
September 1, 2025 at 1:41 PM
Okay so I am wrong somewhere
Moment of truth...
September 1, 2025 at 1:40 PM
Moment of truth...
Wolfram Alpha is really saving me today...
The Normal CDF = 0.5 * (1 + erf([x-mu] / [sigma*sqrt(2)]) so that's where the input expression comes from
The Normal CDF = 0.5 * (1 + erf([x-mu] / [sigma*sqrt(2)]) so that's where the input expression comes from
September 1, 2025 at 1:38 PM
Wolfram Alpha is really saving me today...
The Normal CDF = 0.5 * (1 + erf([x-mu] / [sigma*sqrt(2)]) so that's where the input expression comes from
The Normal CDF = 0.5 * (1 + erf([x-mu] / [sigma*sqrt(2)]) so that's where the input expression comes from
So adding in those derivatives (where `m` is `mu` and `s` is `sigma` from before)
September 1, 2025 at 1:20 PM
So adding in those derivatives (where `m` is `mu` and `s` is `sigma` from before)
I could probably do the derivatives myself, but I think Wolfram Alpha is far more reliable than me. So I use it fairly regularly or to check my work
September 1, 2025 at 1:19 PM
I could probably do the derivatives myself, but I think Wolfram Alpha is far more reliable than me. So I use it fairly regularly or to check my work
This is a first draft of the truncated normal estimating equation. Next step is figuring out the derivatives (since I need them for this EE)
September 1, 2025 at 1:10 PM
This is a first draft of the truncated normal estimating equation. Next step is figuring out the derivatives (since I need them for this EE)
So if I run the usual mean estimator, the estimates for mu and sigma are off. Bumping up the previous N to 10k, the estimates are 0.52 and 0.49 (where the truth is 0, 1).
September 1, 2025 at 1:00 PM
So if I run the usual mean estimator, the estimates for mu and sigma are off. Bumping up the previous N to 10k, the estimates are 0.52 and 0.49 (where the truth is 0, 1).