Sustainable & Scalable Digital Preservation at The National Archives UK
Digitally preserving national records dating back over 1,000 years
The National Archives UK is the official archive of the UK government, housing records that span nearly 1,000 years of history. From Domesday Book and Magna Carta to modern government files and archived websites, The National Archives hold over 11 million records, a collection that continues to grow rapidly in the digital age.
As digital records began to outpace paper documents in both volume and complexity, The National Archives recognized the urgent need for a robust, scalable and secure Digital Preservation management system.
Since 2001, the archive's collaboration with Preservica has evolved to address the challenges of Digital Preservation while embracing modern innovations such as cloud technology, advanced security protocols and open-source integrations. This pivotal collaboration positioned The National Archives to not only address its immediate preservation challenges but to redefine digital archival practices for the modern era.
The Challenge: Legacy systems and growing complexity
The organisation's journey into Digital Preservation began in the early 2000s with a bespoke, on-premise system designed to manage the growing influx of digital records. While innovative at the time, the system became increasingly difficult to manage as digital demands evolved, eventually requiring a more scalable and modern approach to meet current needs.
The system required highly specialized personnel to maintain its operations, which introduced vulnerabilities. Staff turnover often left gaps in knowledge, creating risks for continuity. The on-premise system was resource-intensive, relying on inefficient and costly hardware that was environmentally unsustainable. Additionally, the heavily-customised system diverged from the standard product set making software updates complex and risky, and as a result, were often not implemented.
As the volume of digital records increased—spanning everything from government inquiries to modern SharePoint files—the legacy system struggled to scale. These challenges underscored the need for a fresh, modern approach.
All photos provided courtesy of The National Archives
The Solution: Modernizing Digital Preservation with a scalable, secure and sustainable cloud-based solution
In 2023, The National Archives made a pivotal decision to transition away from its legacy on-site Preservica software to Preservica’s fully managed Digital Preservation service. The Archives upgraded its system by transitioning to this cloud-based platform, approaching the process as an opportunity to modernize and streamline operations while benefiting from a fresh start.
Key features of the solution included:
1. Cloud-First Approach
Aligning with the UK government’s cloud-first policy, The National Archives transitioned to a serverless, scalable system that reduced reliance on physical infrastructure.
2. Operational Sustainability
By outsourcing maintenance and upgrades to Preservica, The National Archives significantly reduced the operational burden on their internal teams.
3. Enhanced Security
Preservica’s cybersecurity protocols and accreditations reassured The National Archives that even its most sensitive records could be safely stored. Notably, Preservica staff do not have access to The National Archives data, a feature that provided additional confidence. Custom security enhancements were also made, benefitting all users.
4. Custodial Copies for Risk Mitigation
While Preservica manages The National Archives' primary digital repository, the latter maintains a “custodial copy” of their data for added security and flexibility. The Preservica API enables effective synchronisation with The National Archives custodial storage facility. Custodial copies utilise the vendor-neutral Oxford Common File Layout (OCFL) specification, with the process facilitated by open-source software developed by The National Archives and shared on GitHub. This dual operational/custodial approach allows The National Archives to benefit from outsourcing while mitigating the risks by retaining full control over their records.
With these foundational upgrades in place, The National Archives has been able to modernize its Digital Preservation strategy, leading to significant improvements in scalability, efficiency, security and innovation.
Results: Scalability, efficiency, security and innovation
The transition to a fully managed, cloud-based Digital Preservation management system has transformed The National Archives' operations, offering scalability, improved efficiency with resource-heavy infrastructure and enhanced security for sensitive data.
The shift to a serverless cloud model has made the system more scalable and better equipped to adapt cost-effectively to growing demands over time including:
- Scalability and Flexibility: The National Archives new system can accommodate the growing volume and variety of digital records, including modern formats and high-volume transfers, including from public inquiries.
- Improved Efficiency: By moving away from resource-heavy on-premise infrastructure, The National Archives has achieved significant operational efficiencies. The serverless cloud model ensures resources are used only when needed, reducing environmental impact and costs.
- De-Risked Operations: By leveraging Preservica for system maintenance and upgrades, The National Archives has reduced its reliance on specialized internal staff, ensuring continuity and mitigating the risk of knowledge loss. Preservica’s economies of scale also provide a more sustainable and efficient operational model.
- Enhanced Security Confidence: The new Digital Preservation strategy has enhanced The National Archives security posture, allowing even the most sensitive records to be safely stored in the cloud. Custom security measures co-developed with Preservica have set a new standard for Digital Preservation.
Use Case: Streamlining the preservation of court documents and tribunal decisions at The National Archives
One of the most impactful applications is the long-term preservation of the court judgments and tribunal decisions, published via The National Archives’ Find Case Law service. Initially, the organisation chose this as a test case due to the impact of the records and the consistent, daily flow of new documents. The simple, intuitive process perfectly demonstrates how Preservica’s system operates automatically.
Court clerks upload digital court judgment documents using The National Archives’ bespoke Transfer Digital Records service. These are processed – for example, to create multiple versions for publication - before being reviewed by archivists. Once the review is complete, the documents are published to The National Archives’ Find Case Law service for global access. Simultaneously, these documents enter Preservica’s ecosystem, where they are actively maintained for long-term integrity and usability.
What sets this process apart is its hands-off nature. Once the documents are uploaded, there’s no need for archivists to manually intervene in the preservation ingest process. Within minutes of uploading, the documents are securely stored in Preservica, organized correctly and actively maintained to ensure their longevity and integrity. The system works entirely in the background with minimal human intervention, allowing staff to focus on other priorities. This is a dramatic shift from the labor-intensive processes of the past, which required archivists to handle data and perform numerous checks, a time-consuming and error-prone effort.
This transformation has significantly reduced processing time—from weeks or even months of manual work down to same-day preservation—while enhancing accuracy and reliability. The simplicity and efficiency of Preservica’s platform have proven to be a game changer for The National Archives, offering a solution that’s amenable to future-proofing, that streamlines operations, reduces human error and allows for active maintenance of critical national records with virtually no manual effort.
Lessons from The National Archives for institutions navigating the future of archival sustainability
The National Archives’ experience provides valuable lessons for other institutions embarking on a Digital Preservation journey. Transitioning to a modern system often requires leaving legacy infrastructure behind, which, while challenging, can pave the way for a sustainable solution.
By partnering with a trusted provider, The National Archives has reduced operational risks and freed up internal resources for strategic priorities, all while maintaining control through custodial copies to mitigate outsourcing risks.
Finally, The National Archives scalable system and agile approach exemplify the importance of flexibility in addressing the changing digital landscape.
Looking ahead, The National Archives remains dedicated to evolving its partnership with Preservica, regularly reassessing its requirements to ensure the system aligns with its organizational objectives. “We appreciate the flexibility of a solution that allows us to adapt and explore alternatives as needed, but our focus remains on working closely and collaboratively with Preservica to meet our goals,” Daly shared, emphasizing their proactive approach to future planning while valuing the system’s current alignment with their needs.
The National Archives’ experience with Preservica underscores the transformative potential of a well-executed Digital Preservation strategy. By embracing a cloud-first, managed, scalable solution and nurturing a trusted partnership, The National Archives has not only secured its invaluable digital records but also established a model for innovation and sustainability within the archival community. For more information, visit The National Archives UK.
Discover how Preservica can accelerate your Digital Preservation efforts and enhance the management of your valuable collections. Contact us today to learn more about our customized solutions and start your journey toward seamless Digital Preservation.