NSW Health Wiped Out Covid, Vaccine Data Used In Weekly Reports
Like many Australians locked down during Covid lockdown, Xin Yin Ooi found herself “obsessed” with the health authorities’ weekly updates, looking at the number of cases, vaccinations, hospitalizations and deaths.
As a data analyst, the Sydney woman was naturally interested in the detailed statistical breakdowns provided in NSW Health’s weekly observation report, so when she noticed a strange figure in one table, she decided to request the underlying figures to check the work herself.
The only problem? The data has been “erased”.
“It was a big shock,” Ms. Ooi said.
“During the early days of Omicron, our Prime Minister, Minister of Health, Chief Medical Officer every day at a press briefing, I distinctly remember saying, “The data shows that two doses are not enough, you need three to deal with Omicron.” They kept repeating ‘data, data’ – it’s unbelievable that they deleted the data.”
Earlier this month, the New South Wales Civil and Administrative Tribunal (NCAT) ruled in favor NSW Health in a freedom of information battle with Ms. Ooi, who first filed a public access to government information (GIPA) request with the agency over a year ago.
‘Long Journey’
Ms. Ooi asked NSW Health to provide “anonymized patient-level data” that underpins two tables in a follow-up report published in the first week of February 2022.
Table 5 of this report provides a breakdown of hospitalizations, ICU admissions, and deaths between November 26, 2021 and February 5, 2022, sorted by vaccination status—one, two, or three or more “effective doses.” ”, without an “effective dose” or unknown.
Table 6 presents the “proportion of cases with a severe outcome” either in the intensive care unit or with death, disaggregated by age and vaccination status, but shows only “three or more effective doses”, “two effective doses”, or “less than two effective doses. “.
Ms. Ooi also requested any “other relevant information that can help me understand this data,” stating in her March 2022 statement that “my first analysis goal is to ungroup the last two columns in table six into [a] a more advanced category of vaccination.”
“It’s actually a very important table — it’s been tracking cases since Omicron arrived in New South Wales, at the time it was a new virus and everyone wanted to know what it was doing,” she said.
“What’s interesting is that you actually see that when you go from zero dose to one dose, there’s actually an increase in [in hospitalisation] from 1 percent to 2.7 percent if you look at this column. I am a data analyst by profession, I wanted to know why so I did GIPA. That was the start of my long journey in search of it.”
NCAT hearing
The following month, NSW Health denied the application, saying it did not have the requested information. Ms. Ooi sought feedback from the Information Commissioner, who confirmed NSW Health’s decision in June.
The case was then referred to the NCAT for administrative review in December.
Miss Ooi posted a long story last week Thread on Twitter stating their concerns following the NCAT decision in favor of NSW Health.
“In 2021, our government informed us that Covid-19 is an unprecedented pandemic that requires an unprecedented solution, which is to vaccinate the entire population, with a few exceptions – the elderly, young, middle-aged, everyone, with Covid-19 vaccines who have just received TGA pre-approval,” she wrote.
“We were told that this is our way out of the pandemic. That vaccines will “stop the spread”. Later, our government stated that even if vaccination does not stop Spreading, this will reduce the likelihood of hospitalization, admission to the intensive care unit, or death. Logically, NSW Health, which has been leading our response to Covid, should be very interested (and obligated) to check if the vaccination program is working and if it is achieving its goals.”
At the hearing, where Ms. Ooi introduced herself, Dr. Kristin Selvi of NSW Health’s Covid Influenza unit explained how the weekly reports were designed in such a way that no “interim records” were kept after the tables were created.
“Difficult process”
Dr Selvi told the tribunal that the former Public Health Response Branch (PHRB), which produced reports prior to the restructuring of the agency in March 2022, will extract and link data from three separate sources – the Notifiable Conditions Information Management System (NCIMS), the Australian Immunization Register (AIR) and the Patient Flow Portal, an operational database of hospital patients in the New South Wales Health System.
“As Dr. Selvi explained, the process of extracting data from each of the sources and linking the datasets was an extremely complex process and took several hours,” NCAT senior member Elizabeth Bishop said in her decision.
“Once the data was extracted, the analyst ran the code to create tables with the output, which were then verified by the epidemiologist. This process took several more hours.”
Medical personnel in New South Wales used the statistical programming language R for the analysis required for tabulation.
“A temporary database (data frame) is created that is stored in the computer’s RAM and exists only for the duration of the R session,” Ms. Bishop said.
“Intermediate tables are not created, and only the required output is created, after which the session ends and the data frame is erased.”
Dr Selvi said no interim records of the tribunal were kept because “the purpose of the analysis was to create a point-in-time snapshot reflected in the published tables” and “there was no need to keep copies”.
“Also, if more analysis is needed later, more relevant information … can be extracted at this stage,” she said.
Privacy issues
She added that “data access from AIR is permitted for very limited purposes only and is not to be retained except for the prescribed purpose” and that “associated patient information is extremely sensitive and retaining these records would violate confidentiality.” concern.”
GIPA’s application was denied because NSW Health argued that in order to provide the requested information, it would have to repeat the same complex process and “create a new record”, which is outside the scope of the law.
Ms Bishop said she was “satisfied with the balance of probabilities” that NSW Health “did not retain the patient data underlying tables five and six and therefore does not contain the information requested by the applicant” and also found that “interim data records were not kept (in any form) and therefore are not in the possession of the respondent.”
“For the sake of completeness, the Respondent has stated that it is not required to create a new record to satisfy the Applicant’s request for information, although in theory it could create a new record containing the requested information,” she said.
“This will require re-retrieval of data (including data stored in non-responder AIR), complex compilation and analysis of data, interpretation and creation of a new record. In addition, there is uncertainty about whether the same information can be retrieved again when the related data that was used to create tables five and six was at a point in time.”
“Bad Practice”
Ms Ooi, however, argued that the NSW Health process presented a “big accountability issue.”
“We’ve had three years of this pandemic and they say they never left a record of the data they used for their report, which they kept releasing every week,” she said.
“There is no verification, we can’t verify the correctness, because they erased everything.”
She said that in her work, she will always save this data so that she can go back and “make sure it’s accurate” if something goes wrong.
“Ask any data analyst, would you do it?” She said.
“Create a complex analysis that takes hours to generate output and never save your analysis, never save intermediate work, just publish the final result? You can’t trace it, there’s no audit trail. It’s just bad practice, especially during a pandemic and a vaccination program that makes things worse – you’re erasing the data that you base your policy on.”
A spokeswoman for NSW Health declined to comment.
Not black and white
Associate Professor James Trower, head of epidemiological modeling at Monash University, said that while he does not want to comment on the legal details of the case, it appears to be a “little technical discussion” of the data.
“I agree with the broader contention that Australian jurisdictions should have been providing more disaggregated surveillance data publicly throughout the pandemic,” he said in an email after reviewing the NCAT decision and Ms Ooi’s Twitter thread.
“We know that unvaccinated people predominate in deaths and hospitalizations, but we need to know more about the individual characteristics of these people in order to calculate the true efficacy of a vaccine. More information about this would be very helpful.”
He noted that New South Wales “is actually the leader in providing this data in the country, so it’s absolutely not an issue that is limited to NSW.”
“However, Australia lags far behind some other foreign countries such as the UK,” he said. “We need to think about how to improve this for future pandemics. The Australian Centers for Disease Control and Prevention (CDC) could be a key tool to achieve this.”
Professor Trawer added: “I note that most of Xin Yin Ooi’s tweets focus on data identifiability. In fact, there is no simple clear separation between identifying and de-identifying data. Even aggregated tabular data can be associated with privacy issues.”
Originally published as NSW Health ‘erased’ data used in weekly Covid surveillance reports