Data in Public Health: Why we Need a Change in Thinking

The last two decades have brought about a transformation in how we think about data. Social media primed us to believe that sharing is great, from our breakfast to our wedding. Data intensive applications necessitated cloud technology, and now it’s harder than ever to keep track of where we’ve left our digital footprint. The World Economic Forum estimated that in 2020 we would collectively produce 44 zettabytes of data per day, and this value is bound to only trend upwards. Although artificial intelligence provides hope in being able to sort through all this information, at present we are dealing with an infodemic, not just in disease information as the World Health Organization defines the term, but in all aspects of life. Data has become so ubiquitous as to often become meaningless.

One exception to this trend is public health. On the frontend, communications departments are now drown out by the onslaught of the information age and struggle to get effective messaging across. On the backend, many local health departments were so technologically underfunded they resorted to fax machines to send COVID-19 case counts. Contrary to even a start-up with a similar budget, public health just doesn’t seem to be in the same league when it comes to software and information systems.

This stems from a fundamental divide between how data is seen in the private sector compared to the public sector. For many companies, gathering more data directly impacts the bottom line. Being able to better predict what a customer would like to purchase, estimate shipping times more efficiently, or drive marketing projects to reach more eyeballs all either decrease cost or increase revenue. This makes it easy for top executives to see the value in investing in data systems. On the public side however, gathering more data is often costly, especially when systems start from an antiquated state. Once gathered, this data doesn’t increase tax revenue. While it may decrease costs in the long run, bureaucratic processes and frequent changes in political spending priorities can make upfront investment both daunting and fraught with uncertainty. In addition to these hurdles, no administrator wants to be responsible for making a systems change that results in a security breach, downtime, or in a worst case scenario, loss of life. With the absence of competition that demands innovation on the private side, government moves slowly and prefers stability.

As health systems scale to improve services, this thinking needs to be adapted to accept the realities of the current pace of change. Tracking diseases in excel can be done, but it definitely shouldn’t be done, as demonstrated by the blunders of the National Health Service in the United Kingdom during the pandemic. Having public officials that are not trained on modern data systems leads to chaos, grotesque inefficiency, and risks systems losing compatibility with other contractors and stakeholders.

Preferably, the drive to modernize services can also be used as an opportunity to re-imagine the tarnished reputation that data is beginning to develop overall as something used solely for corporate greed. Government could be a leader in developing open-source software for the common good, leading to international cooperation in health systems. A Japanese software library for pairing those suffering from homelessness to services could be adopted by Brazil. A German program to enable patients easier access to their medical records could be adapted for Australia. The possibilities are vast and the efficiencies from reducing knowledge silos for tasks that all governments share – providing healthcare and social services – are hard to overstate.

Likewise, public health officials should be trained in open-source languages to alleviate the research and healthcare industry from paying for expensive software licenses. Substituting R for Stata saves hundreds of dollars per year. Multiply that by the hundreds of researchers using the software and the savings begin to become substantial and worthwhile. The concepts of collaboration and transparency are already familiar to academics by virtue of the scientific process, there is no reason this should not extend to software as well.

I’m personally optimistic about just how much room for improvement there is. While there are certainly valid criticisms to be made about the current performance of health systems around the world, the fact that they have been able to get as far as they have using older data systems indicates that performance may be drastically improved once newer pipelines kick in.

Data doesn’t have to be “the new oil” or “liquid gold”. Rather, data can be the essential tool in shaping a better tomorrow for goals that matter to us all: health, happiness, and hope for the future. The field of public health has never been shy from a challenge, so I say let’s add improving data systems to the list.