Citations for articles mentioned are at the bottom of this page.
This week we focused on reading literature reviews about data privacy, specifically differential privacy, to gain a better understanding of the subject. The first article was "Differential Privacy: A Primer for a Non-Technical Audience," 1 which explains the general function of differential privacy, along with its advantages and disadvantages. The second article was "The use of differential privacy for census data and its impact on redistricting: The case of the 2020 U.S. Census," 2 which focuses on the advantages and disadvantages of the U.S. Census Bureau adapting a differential privacy approach. This paper focuses on the development of the disclosure avoidance system (DAS), which the Census Bureau hopes will better protect the privacy of individuals in the Census. The third article we explored this week was "Statistical Data Privacy: A Song of Privacy and Utility," 3 which examines the differences between differential privacy and statistical disclosure control.
This week we read a rebuttal to the Kenny et al. article, titled "Statistical Inference is Not a Privacy Violation." 4 This post offers a contrasting view of the differential privacy approach suggested by the U.S. Census Bureau. We then discussed potential research questions we would be interested in exploring.
To better understand the algorithm used by the DAS, we read "The 2020 Disclosure Avoidance System TopDown Algorithm." 5 This paper discusses the processed administered by the DAS. We also examined "Transparent Privacy is Principled Privacy" 6 as an accompanying read to better understand the importance of transparency and its role in differential privacy.
This week, we attempted to replicate the figures presented in the Kenny et al. article by using the replication material provided on Harvard Dataverse, titled "Replication Data for: The use of differential privacy for census data and its impact on redistricting: The case of the 2020 U.S. Census" 7 . We successfully reproduced Figures 1, 2, 5, and a few others that were produced by the authors but not included in the paper. We then decided our overall research goal for the rest of the summer. Since the U.S. Census Bureau did not release the 2010 Census before post-processing and following noise injection, our goal is to simulate noisy data using the 2010 Census and compare it to the post-processed Census to determine whether the biases listed in the Kenny et al. article are truly due to post-processing or if they are simply a consequence of injecting noise.
This week, we created a GitHub repository to organize and keep track of all of our code and progress. We then replicated Figures 1 and 2 in the Kenny et al. article using the datasets provided on the IPUMS website 8 which includes all iterations of the Census DAS demonstration data files. While each data point in Figures 1 and 2 represent precincts, the IPUMS data files only organizes data by state, county, tract groups, tracts, and blocks. We found that datasets for blocks contained many zeroes as values, so we focused on tracts. We then created figures that plotted the same parameters as Figures 1 and 2 in the Kenny et al. article to see if we could replicate their figures using the IPUMS datasets.
We had planned to spend this week producing noisy measurements, reproducing our figures, and comparing them to the figures using the DAS demonstration data. However, as we further analyzed the figures presented in the Kenny et al. article, we noticed that some of the figures, specifically Figures 1 and 2, were misleading. These figures were meant to illustrate the difference in Census values between the post-processed Census and the swapped 2010 Census, which was taken as the ground truth. However, rather than plotting the true difference in values, they plotted the fitted difference/error. The fitted error was produced by fitting a generalised additive model (gam) and plotting the outputs of that model against either non-white percentage (Figure 1) or Herfindahl-Hirschman Index (HHI), which is a measure of diversity. We were suspicious that plotting the fitted model would be misleading, especially because the gam used non-white percentage and HHI as predictor variables. After noticing this, my research group and I decided to put a hold on producing noisy measurements, and we set out to determine the extent to which this choice in plotting affected the overall results of the study. We first reproduced Figures 1 and 2 by plotting the true error in place of the fitted error. We found that these new graphs were similar to the original figures, but they were less intense in shape. They also more closely resembled our figures that were created using the IPUMS demonstration data, portraying that the trends shown in the figures are consistent among other geographic levels, specifically tracts.
Now that we reproduced the Kenny et al. figures to allow for better comparison, we resumed our goal of creating noisy measurements. Since the DAS injects Guassian noise into the data, as explained in the Abowd et al. article, we worked on writing code that injects Gaussian noise and reproduces our figures using the noisy measurements in place of the post-processed ones. We also started brainstorming analyses we could do to further explore the effect of plotting fitted error in place of the true error in the original Kenny et al. figures. We started by pertrubing the error so that it became disconnected from the other data. The purpose of this was to eliminate any trends that were potentially present. Then, we plotted the fitted error using this pertrubed error to see whether trends are created where they should not be.
This week, being our last week, was dedicated to finishing up my presentation on our research project, as well as writing up a paper to summarize our findings. We plan to continue working on this research project through the Fall 2022 semester.