The Critical Challenge of Effective Recordkeeping in Government

The importance of recordkeeping in government has never been more pressing and yet, time and again, we're confronted with stories that reveal just how disastrous poor recordkeeping can be. From the destruction of vital documents to the exposure of personal data in critical abuse cases, the consequences of poor recordkeeping are profound, and affect vulnerable stakeholders most acutely. This isn't just a case of poor organisation and compliance; it affects human rights, national security and historical legacy.

The Case of Abuse in Care: A Tragic Example of Record Failures

In New Zealand, the Abuse in Care Royal Commission has uncovered a disturbing truth: inadequate government recordkeeping has deeply harmed survivors of abuse within state care. Anahera Morehu, Chief Archivist of Te Rua Mahara o te Kāwanatanga Archives, recently delivered an apology on behalf of Archives New Zealand, acknowledging the systemic failures that have stripped many of their right to access their own stories. This apology is part of a broader reflection on how poor recordkeeping practices have contributed to the loss of personal identities, severed cultural connections, and delayed justice for thousands of individuals.

The revelations from the Commission have highlighted several key issues, including the precipitous destruction or improper handling of records, gaps in documentation that obscure the true extent of the abuse, and the inability to trace the rightful entitlement of survivors due to incomplete or lost files.  

A Widespread Challenge

New Zealand isn’t the only country grappling with the consequences of inadequate recordkeeping. In Australia, a recent audit by the Australian National Audit Office (ANAO) revealed that more than half of federal government agencies have never (or rarely) transferred vital records to the National Archives. Shockingly, 93% of records in agencies remain unsentenced, meaning they are not assigned a retention period or preservation rule. That means these records are at risk of being deleted when they should be retained, or, alternately, never deleted. Keeping information that no longer has a business use clutters up systems, making important information harder to find, and leaves government agencies exposed to (avoidable) privacy breaches, leaks and legal issues.

But the issue doesn’t just affect internal processes. When records go missing or get mishandled, entire systems of social governance can break down. Poorly managed records have contributed to disastrous incidents globally, such as the wrongful deportation of members of the Windrush generation in the UK, the Manchester Arena bombing, and the unlawful detention of Vivian Solon and Cornelia Rau in Australia.

AI as a Solution

As the volume of data grows exponentially, technology can offer a way forward. Artificial intelligence (AI) is starting to be used in government portfolios like Defence, Treasury, and Education to help manage records more effectively. Rather than relying on manual processes, which are prone to human error, AI automates the classification of records, ensuring that they're properly stored and archived.

In Australia, these AI systems have proven to be highly accurate, with agencies reporting up to 98% accuracy in classifying records. This means faster processing times, better compliance, and a significant reduction in the risk of mishandling records.  

The adoption of AI for recordkeeping can provide a solution to the chronic inefficiencies and resource limitations that many government agencies face. Instead of relying on outdated, labour-intensive processes, AI can automate classification and retention, freeing up staff to focus on more strategic tasks while ensuring records are properly handled and retained.

Strengthening Recordkeeping Practices

So, what can be done to improve government recordkeeping? Well, both New Zealand and Australia are already taking steps toward reform. In New Zealand, for instance, Te Rua Mahara Archives has recognised the significance of this issue and is working on amendments to the Public Records Act as part of the Responding to Abuse in Care Legislation Amendment Bill to better regulate recordkeeping practices and ensure that personal data is properly handled.

In Australia, the implementation of AI is already proving effective in certain sectors, but broader adoption is needed across all levels of government. Further investment in technology and training is essential to ensure that the management of public records is not only efficient but also accountable and transparent. The key to fixing this issue lies in embracing modern, ethical solutions and holding institutions accountable for their role in protecting personal data.

The challenges posed by poor recordkeeping are significant, and the consequences of inaction are too great to ignore. As the world grapples with these challenges, we need both governmental bodies and technological innovators to collaborate and build systems that protect the future while learning from the mistakes of the past.

‍