diskDedupe 4+

This utility is extremely fast and flexible. Unlike other duplicate file finders, this one has a few unique features including: ability to scan only files above or below a minimum size, and a simplified deduplication process that doesn't require you to pick a canonical original.

Under the hood, the speed comes from the APFS file cloning capability, which is sort of like really efficient hardlinks on HFS but allows copies to diverge efficiently without clobbering one copy or the other if a cloned copy is modified or deleted. This takes the guesswork out of deduplication, and for its use case beats the pants off of any other duplicate file utility I know of.

The only real limitation of this utility is that it ONLY works on APFS file systems. You can't use it on HFS disks. However, it *does* work reliably on mounted APFS disk *images* (including sparsebundles) even if they're located on HFS or other filesystems, so it's a great way to get the benefits of APFS cloning and deduplication even if you have spinning disks or RAID systems that don't do APFS well.

One sparsebundle was reduced from 9 TB to only 1.4 TB after deduplication with this utility followed by hdiutil compaction. Your data patterns may vary, but this was pretty compelling for me.

In short, I highly recommend it.

This app works perfectly! I would only advise against using it on files in iCloud Drive, because iCloud Drive storage acts differently than any other "normal" APFS drive. When I used it on iCloud Drive, iCloud preserved all of the original files I "deduplicated", as well as all of the copy-on-write duplicates that this app created, so I essentially ended up with two copies of each of the files (the original plus the new copy-on-write file). Other than that, the app works great on any APFS drive I've tested it on!

It does what it says it does. It finds duplicate files and combines them so you don't use storage you don't need to. It uses something called "hard links" which have multiple file pointers pointing to the same data.

Backup before you do it, but I haven't had any problems using it and because many of my files and photos were duplicates, it's saved a lot of space.

I wish it were a little faster, but I have a lot of files! Maybe it could use the APFS FileVault file hashes instead of calculating them on its own.