Re: OK, how do I count files in a directory QUICK!!!

Top Page
Attachments:
Message as email
+ (text/plain)
Delete this message
Reply to this message
Author: Matt Graham
Date:  
To: Main PLUG discussion list
Subject: Re: OK, how do I count files in a directory QUICK!!!
From: "" <>
> I'll keep it easy: It is a directory that only contains files.
> So, you'd say: what's wrong with "ls|wc -l" ?
> There are almost a million files in that directory.
> This count has to be placed in a loop in a shell script to report a
> second-to-second delta.


The directory organization is complete pants if there are that many files in
one dir. Time to split some things up, eh?

> The truth is that find takes some 3 seconds to do the count.
> What about ls without sorting? That was almost 15 seconds.


That's a bit strange. I'd think "ls -1U --color=never" would spit everything
out almost as fast as possible, considering.

> Now, directories are files. It would be great if I could "count
> lines" on that [directory] or somehow interrogate it "how many
> lines do you have?" without actually hitting the filesystem.
> I'm considering writing a little C utility to do just that, but...
> "struct stat" (my first shot) doesn't contain that


#include<dirent.h>
#include<stdio.h>

int main(int argc, char **argv)
{

DIR *dirp;
struct dirent *entry;
int count=0;

if(argc != 2){
printf("usage: checkdir <directory to check>\n");
exit(1);
}

dirp=opendir(argv[1]);
if(dirp==NULL){
printf("couldn't open %s\n",argv[1]);
exit(1);
}
while((entry=readdir(dirp))!=NULL){
count++;
}
printf("%d files in %s\n",count,argv[1]);
exit(0);

}

...this counts all files; all dirs have at least 2 entries in them, . and .. ,
so subtract 2 or do count-=2 or whatever. Note that this does no filetype
checking at all and checking of errors is minimal. I can't see how to do what
you want any faster without going below glibc, which is pretty hairy.

Also, see if you can rearrange the dir structure. Having a million files in
one dir can cause silliness. HTH,


--
Matt G / Dances With Crows
The Crow202 Blog: http://crow202.org/wordpress/
There is no Darkness in Eternity/But only Light too dim for us to see

---------------------------------------------------
PLUG-discuss mailing list -
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss