cp: Argument list too long

[David Mandala]

You may want to not have so many files in a single directory. Linux
stores directories in a liner list so accessing large directores is hard
and slow (as you've noticed). There is a patch for the kernel that adds
btree capabilities to the directories but you must build the kernel with
it and you still can get grief with utilities that are not expecting so
much data on the command line.

Also you may need to just do $(ls) instead of $(ls *.pdf) and add an if
statement where the mv or cp is so you only cp or mv a pdf file.

here is a corrected version using an if statement, make sure to type the
spaces correctly in the if statement, the [] braces must have spaces
before and after them.

for file in $(ls); do filename=$file; if [ ${filename##*.} = "pdf" ];
then mv $file /newdir/; fi; done



On Mon, 2002-12-09 at 14:54, David Mandala wrote:
> This should work from the command line, make sure to replace the
> /newdir/ with a proper path for you. Can use cp also. Make sure to type
> this all on one command line.
> 
> for file in $(ls *.pdf); do  mv $file /newdir/; done
> 
> Warning I can't test this with as many files as you have but this should
> work.
> 
> Cheers,
> 
> Davidm
> 
> On Mon, 2002-12-09 at 14:15, Bill Earl wrote:
> > Hello everyone!
> >  
> > I need to copy, and occasionally move, some directories that contain
> > thousands of files, (like around 250,000 pdfs), and the problem I'm
> > having is that in the Bash shell, both cp and mv commands seem to die
> > at somewhere around 3,000 files, so that if I enter
> >  
> > cp /mydir/myfiles/*.pdf  /newdir/
> >  
> > I get the following error:
> >  
> > bash: /bin/cp: Argument list too long.  
> >  
> > I noticed that even just trying to list this many files can cause this
> > error, depending on the syntax used. "ls /mydir/myfiles/*.pdf"
> > generates the Argument list too long error, but "ls /mydir/myfiles"
> > displays fine.
> >  
> > I can get around it using a python script and more fiddling around
> > than I really want to do. I'd really like to be able to move these
> > large numbers of files around just through the shell.
> >  
> > Is anyone aware of a separate file copying or moving (I need to do
> > both, depending on the situation) utility that can handle very large
> > numbers of files?  Would switching to a different shell solve this (if
> > so, which one?), or is it a limit on the cp and mv commands?
> >  
> > Thanks!
> >  
> > Bill Earl
> > Network Admin
> > Spectrum Data Solutions, LLC
> > www.bestbill.com
-- 
David IS Mandala
gpg fingerprint 8932 E7EF CCF5 1B8C 1B5C  A92E C678 795E 45B2 D952
Phoenix, AZ (480) 460-7546 HP, (602) 321-8277 CP
http://www.them.com/~davidm/