How I've optimized my file system backup setup as a Developer on Windows.

As I'm going through my almost-yearly backups and clean-up of my devices I've noticed Windows displaying the actual folder size was around 30GB - but the size on disk was somewhere around 75GB. WTF. That's more than double!

When you have a difference this large it usually means you have a large number of small files, which increases cluster overhead due to how NTFS works.

Windows' NTFS will usually space for files in clusters of 4k. But if your files are small then that, then windows simply reserves the leftover space, but it remains unutilised.

ie: 10 files of 1byte each will have total size in disk of 40k

This behavior is known as "slack space".

My backup setup had too many small files.

Freeing up to 37GB of node modules

So while doing a quick investigation I quickly realized I had a bunch of useless "node_modules" folders which were copied to my backup disk along with their parent repositories.

I thought I could run a few prompt commands to perhaps delete them.

Out of curiosity I've first executed some type of count command to get a rough number of total files on my backup setup:

dir /s /a-d | find /c "."

The result was 950516 files.

Then I've executed the following command to delete any all node_modules folders recursively.

for /f "delims=" %d in ('dir "node_modules" /s /b /ad') do rd /s /q "%d"

It took a while and the command is not perfect. But after finalizing I've executed the count command again and had the a total file count of 276669!

Lastly, I've checked again the total folder size and the actual size in disk:

• Folder size: 26GB
• Size in disk: 38GB

Not bad, not bad ...

Conclusion

• Do not forget to exclude node_modules from your backups if you can.
• Avoid too many small files.