Couple of tools for Duplicate Photos and Media Titles

Despite saving nearly all of my photos stored online straight from my phone, I still many photos and other media stored locally. In part, this is due to having photos from other cameras now and over the years or other media (music library for example).

However, curation is a bit of a headache so I only try to clean every once in a while. Here are a few of the commands that I use to remove duplicates, fix titles or make other changes.

Music and Movie Titles

To check titles on media files like movies or music, I print out the filename (easy to find via ls!) and the title using exiftool like this (the line break between File Name and Title is important to grep -F):

ls -1 |  while read a; do echo "$a"; exiftool "$a" | grep -F "File Name
Title" ; echo " "; done

If I want to update the title:

exiftool -Title=Zombie_Island my_homemade_zombie_movie.m4v

Duplicate Photos

To find duplicate photos, you could rely on the filename, but you might find that the same filename (image_001.jpg) was used for a number of images. So, the best thing to do is compare checksums on the files:

find . -type f -exec md5sum {} \; | sort | uniq --all-repeated=separate -w 15 > dupes.txt

and fine the duplicates to keep:
grep -A1 ^$ dupes.txt | grep / > dup-keep.txt
wc -l dup-keep.txt #to check how many

find the files to move to a holding directory prior to deleting rather than deleting straight away - something could go wrong!:
grep -vf dup-keep.txt dupes.txt > dup-move.txt # slow if there are many...
grep -f dup-keep.txt dup-move.txt  # should be empty

remove the empty lines and check:
grep -v ^$ dup-move.txt > dup-m1.txt
grep -c / dup-move.txt  dup-m1.txt
grep -c / dup-m1.txt
mv dup-m1.txt dup-move.txt

test out "moving" the duplicates into the duplicates directory. I say "moving" as it's really a copy and delete:

mkdir duplicates

cut -d" " -f3-99 dup-move.txt | head | cpio -pvd duplicates
cut -d" " -f3-99 dup-move.txt | head | while read a; do echo $a; \rm "$a" ; done

 
If that looks good, then do the rest:
cut -d" " -f3-99 dup-move.txt | cpio -pvd duplicates
cut -d" " -f3-99 dup-move.txt | while read a; do echo $a; \rm "$a" ; done

Check for duplicates in the local directory again:

mv duplicates ../
find . -type f -exec md5sum {} \; | sort | uniq --all-repeated=separate -w 15 > dupes.txt