Converting NSRL RDS Deltas

TL;DR

You can convert Delta files into database that can be imported into tools that support that format
Download the program here
Set the input file to the “RDS_*_delta.zip” file and specify and output folder
Click Convert and wait for it to finish
There should be a file with the same name as the zip file but with a “.db” extension
This can then be imported into your forensic tool of choice

Requirements:
.NET 6 runtime – https://dotnet.microsoft.com/en-us/download/dotnet/6.0
Download the “.NET 6.0 Desktop Runtime” option

This post has been edited to change links to version 1.2 after an issue with previous versions. If the program hits an entry it cannot add, it will show an error message but will continue importing after you click OK.

In Detail

The National Software Reference Library (NSRL) collects hashes of software an incorporates it in to a Reference Data Set (RDS). This data set is used by Digital Investigators to remove hashes of files that are known not to contains Child Sexual Abuse Material (CSAM) and to reduce time processing evidence.

Historically, this data was provided in large text files but, since at least March 2022, NSRL has moved their hash sets from a text based list of hashes to a SQLite database file. When updates are made to the RDS, a Delta is provided in a zip file.

This zip file contains a schema for the SQLite database, a verification hash for the file once the updates have been applied and a text file with SQL commands to update the database to include the updates. Periodically (appears to be yearly) a new update is provided with all the previous data from the last ‘full’ database and all the Delta files in between.

Importing the full database can be a time consuming process due to the number of hashes that are included in it. Therefore, adding the Delta to the last ‘full’ database and importing that into a forensic tool or hash manager may be inefficient.

In my previous post, https://wordpress.com/post/askclees.com/324, I showed how I created a program to convert the full database into a text file suitable for legacy tools. In this post, I will demonstrate a program I have created that will convert the Delta file (in zip format) into a RDS format database so then only the new hashes have to be imported.

The program can be downloaded from here and requires the .NET runtime 6 to work. If you don’t have it, it will prompt you to download it when you try and run it. If you are running on an air gapped machine, you can download it from https://dotnet.microsoft.com/en-us/download/dotnet/6.0. You only need the Desktop Runtime version

When you start the program it should look like this:

You then need to select the RDS Delta to convert and a folder to output the results, below shows the Android

Once running, you should get a screen similar to below to show you how many records needs processing

Once completed, a message box will come up saying the conversion is complete and then you should have a database in the output directory

This should now import into any tool that supports the RDS SQLite database format. If you need to be able to import this into a legacy tool, see my previous post on converting the RDS to legacy hash format.