thank you so much! After running it I find it only finds the duplicates in
~. I need to find the duplicates across all the directories under home.
after looking at the man file and searching for recu it seems it recurses
by default unless I am reading it wrong.
I tried the uniq command but:

 uniq -c -d -w list.of.files
 uniq: list.of.files: invalid number of bytes to compare

isn't uniq used to find the differences between two files? I have
a very rudimentary understanding of linux so I'm sure I'm wrong

all the files in list.of.files are invisible files. (prefaced with a
period))
and isn't there a way to sort things depending on their column (column1
md5sum, column2 file name)

On Mon, Sep 30, 2024 at 2:56 AM Rusty Carruth via PLUG-discuss <
plug-discuss@lists.phxlinux.org> wrote:

>
> On 9/28/24 21:06, Michael via PLUG-discuss wrote:
> > About a year ago I messed up by accidently copying a folder  with other
> > folders into another folder. I'm running out of room and need to find
> that
> > directory tree and get rid of it. All I know for certain is that it is
> > somewhere in my home directory. I THINK it is my pictures directory with
> > ARW files.
> > chatgpt told me to use fdupes but it told me to use an exclude option
> > (which I found out it doesn't have) to avoid config files (and I was
> > planning on adding to that as I discovered other stuff I didn't want).
> then
> > it told me to use find but I got an error which leads me to believe it
> > doesn't know what it's talking about!
> > coul;d someone help me out?
> >
> First, someone said you need to run updatedb before running find.  No,
> sorry, updatedb is for using locate, not find.  Find actively walks the
> directory tree.  Locate searches the text (I think) database built by
> updatedb.
>
>
> Ok, now to answer the question.  I've got a similar situation, but in
> spades.  Every time I did a backup, I did an entire copy of everything,
> so I've got ... oh, 10, 20, 30 copies of many things. I'm working on
> scripts to help reduce that, but for now doing it somewhat manually, I
> suggest the following command:
>
>
> cd (the directory of interest, possibly your home dir) ; find . -type f
> -print0 | xargs -0 md5sum | sort > list.of.files
>
> this will create a list of files, sorted by their md5sum.  If you want
> to be lazy and not search that file for duplicate md5sums, consider
> uniq.  Like this:
>
> uniq -c -d -w list.of.files
>
>
> This will print the list of files which are duplicates.  For example,
> out of a list of 42,279 files in a certain directory on my computer,
> here's the result:
>
>        2 73d249df037f6e63022e5cfa8d0c959b
>
> _files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160321-223138.png
>        5 9b162ac35214691461cc0f0104fb91ce
> _files/melissa/Documents/EPHESUS/Office Stuff/SPD/SPD SUMMER 2016 (1).pdf
>        3 b396af67f2cd75658397efd878a01fb8
> _files/dads_zipdisks/2003-1/CLASS at VBC Sp-03/CLASS BKUP - Music
> Reading & Sight Singing Class/C  & D Major & Minor Scales & Chords.mct
>        2 cd83094e0c4aeb9128806b5168444578
>
> _files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160318-222051.png
>        2 d1a5a1bec046cc85a3a3fd53a8d5be86
>
> _files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160410-145331.png
>        2 fa681c54a2bd7cfa590ddb8cf6ca1cea
>
> _files/from_ebay_pc/pics_and_such_from_work/phone_backup/try2_nonptp_or_whatever/Pictures/Screenshots/Screenshot_20160312-113340.png
>
> Originally the _files directory had MANY duplicates, now I've managed to
> get that down to the above list...
>
> Anyway, there you go.  Happy scripting.
>
> ---------------------------------------------------
> PLUG-discuss mailing list: PLUG-discuss@lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>


-- 
:-)~MIKE~(-: