On 9/28/24 21:06, Michael via PLUG-discuss wrote:
> About a year ago I messed up by accidently copying a folder with other
> folders into another folder. I'm running out of room and need to find that
> directory tree and get rid of it. All I know for certain is that it is
> somewhere in my home directory. I THINK it is my pictures directory with
> ARW files.
> chatgpt told me to use fdupes but it told me to use an exclude option
> (which I found out it doesn't have) to avoid config files (and I was
> planning on adding to that as I discovered other stuff I didn't want). then
> it told me to use find but I got an error which leads me to believe it
> doesn't know what it's talking about!
> coul;d someone help me out?
>
First, someone said you need to run updatedb before running find. No,
sorry, updatedb is for using locate, not find. Find actively walks the
directory tree. Locate searches the text (I think) database built by
updatedb.
Ok, now to answer the question. I've got a similar situation, but in
spades. Every time I did a backup, I did an entire copy of everything,
so I've got ... oh, 10, 20, 30 copies of many things. I'm working on
scripts to help reduce that, but for now doing it somewhat manually, I
suggest the following command:
cd (the directory of interest, possibly your home dir) ; find . -type f
-print0 | xargs -0 md5sum | sort > list.of.files
this will create a list of files, sorted by their md5sum. If you want
to be lazy and not search that file for duplicate md5sums, consider
uniq. Like this:
uniq -c -d -w list.of.files
This will print the list of files which are duplicates. For example,
out of a list of 42,279 files in a certain directory on my computer,
here's the result:
2 73d249df037f6e63022e5cfa8d0c959b
_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160321-223138.png
5 9b162ac35214691461cc0f0104fb91ce
_files/melissa/Documents/EPHESUS/Office Stuff/SPD/SPD SUMMER 2016 (1).pdf
3 b396af67f2cd75658397efd878a01fb8
_files/dads_zipdisks/2003-1/CLASS at VBC Sp-03/CLASS BKUP - Music
Reading & Sight Singing Class/C & D Major & Minor Scales & Chords.mct
2 cd83094e0c4aeb9128806b5168444578
_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160318-222051.png
2 d1a5a1bec046cc85a3a3fd53a8d5be86
_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160410-145331.png
2 fa681c54a2bd7cfa590ddb8cf6ca1cea
_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160312-113340.png
Originally the _files directory had MANY duplicates, now I've managed to
get that down to the above list...
Anyway, there you go. Happy scripting.
---------------------------------------------------
PLUG-discuss mailing list:
PLUG-discuss@lists.phxlinux.org
To subscribe, unsubscribe, or to change your mail settings:
https://lists.phxlinux.org/mailman/listinfo/plug-discuss
%20to%20avoid%20config%20files%20(and%20I%20was%0A>%20>%20planning%20on%20adding%20to%20that%20as%20I%20discovered%20other%20stuff%20I%20didn't%20want).%20then%0A>%20>%20it%20told%20me%20to%20use%20find%20but%20I%20got%20an%20error%20which%20leads%20me%20to%20believe%20it%0A>%20>%20doesn't%20know%20what%20it's%20talking%20about!%0A>%20>%20coul;d%20someone%20help%20me%20out?%0A>%20>%0A>%20First,%20someone%20said%20you%20need%20to%20run%20updatedb%20before%20running%20find.%C2%A0%20No,%20%0A>%20sorry,%20updatedb%20is%20for%20using%20locate,%20not%20find.%C2%A0%20Find%20actively%20walks%20the%20%0A>%20directory%20tree.%C2%A0%20Locate%20searches%20the%20text%20(I%20think)%20database%20built%20by%20%0A>%20updatedb.%0A>%20%0A>%20%0A>%20Ok,%20now%20to%20answer%20the%20question.%C2%A0%20I've%20got%20a%20similar%20situation,%20but%20in%20%0A>%20spades.%C2%A0%20Every%20time%20I%20did%20a%20backup,%20I%20did%20an%20entire%20copy%20of%20everything,%20%0A>%20so%20I've%20got%20...%20oh,%2010,%2020,%2030%20copies%20of%20many%20things.%20I'm%20working%20on%20%0A>%20scripts%20to%20help%20reduce%20that,%20but%20for%20now%20doing%20it%20somewhat%20manually,%20I%20%0A>%20suggest%20the%20following%20command:%0A>%20%0A>%20%0A>%20cd%20(the%20directory%20of%20interest,%20possibly%20your%20home%20dir)%20;%20find%20.%20-type%20f%20%0A>%20-print0%20%7C%20xargs%20-0%20md5sum%20%7C%20sort%20>%20list.of.files%0A>%20%0A>%20this%20will%20create%20a%20list%20of%20files,%20sorted%20by%20their%20md5sum.%C2%A0%20If%20you%20want%20%0A>%20to%20be%20lazy%20and%20not%20search%20that%20file%20for%20duplicate%20md5sums,%20consider%20%0A>%20uniq.%C2%A0%20Like%20this:%0A>%20%0A>%20uniq%20-c%20-d%20-w%20list.of.files%0A>%20%0A>%20%0A>%20This%20will%20print%20the%20list%20of%20files%20which%20are%20duplicates.%C2%A0%20For%20example,%20%0A>%20out%20of%20a%20list%20of%2042,279%20files%20in%20a%20certain%20directory%20on%20my%20computer,%20%0A>%20here's%20the%20result:%0A>%20%0A>%20%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%202%2073d249df037f6e63022e5cfa8d0c959b%20%0A>%20_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160321-223138.png%0A>%20%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%205%209b162ac35214691461cc0f0104fb91ce%20%0A>%20_files/melissa/Documents/EPHESUS/Office%20Stuff/SPD/SPD%20SUMMER%202016%20(1).pdf%0A>%20%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%203%20b396af67f2cd75658397efd878a01fb8%20%0A>%20_files/dads_zipdisks/2003-1/CLASS%20at%20VBC%20Sp-03/CLASS%20BKUP%20-%20Music%20%0A>%20Reading%20&%20Sight%20Singing%20Class/C%C2%A0%20&%20D%20Major%20&%20Minor%20Scales%20&%20Chords.mct%0A>%20%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%202%20cd83094e0c4aeb9128806b5168444578%20%0A>%20_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160318-222051.png%0A>%20%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%202%20d1a5a1bec046cc85a3a3fd53a8d5be86%20%0A>%20_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160410-145331.png%0A>%20%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%202%20fa681c54a2bd7cfa590ddb8cf6ca1cea%20%0A>%20_files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160312-113340.png%0A>%20%0A>%20Originally%20the%20_files%20directory%20had%20MANY%20duplicates,%20now%20I've%20managed%20to%20%0A>%20get%20that%20down%20to%20the%20above%20list...%0A>%20%0A>%20Anyway,%20there%20you%20go.%C2%A0%20Happy%20scripting.%0A>%20%0A>%20---------------------------------------------------%0A>%20PLUG-discuss%20mailing%20list:%20PLUG-discuss@lists.phxlinux.org%0A>%20To%20subscribe,%20unsubscribe,%20or%20to%20change%20your%20mail%20settings:%0A>%20https://lists.phxlinux.org/mailman/listinfo/plug-discuss%0A>%20 "Reply to this message")
(text/plain)