[ubuntu-us-ma] 10,000+ files in a folder slows down ...

[James Gray]

If you don't find the command line an attractive option there are
other file managers (instead of nautilus) that you can install which
may perform faster.  A few searches in synaptic (or google) will turn
up some alternatives.

On Thu, Jun 25, 2009 at 1:28 PM, Daniel
Hollocher<danielhollocher at gmail.com> wrote:
> I ran into and reported a bug on gedit for large numbers of files.
> The problem ended up being the data structure and the sort method.  I
> won't go into it, but basically the design was good for folders with
> less than 100 files.  There was another bug too, which was multiplying
> the slow down, and that bug was easy to fix and was fixed.  The data
> structure / sorting issue was not fixed as of jaunty (AFAIK).
>
> So, by my estimates, gedit went from being 10-100 times slower than
> nautilus and is now on par with nautilus after the simple bug fix.  I
> suspect that nautilus too is using some poor design, since its now
> just as fast as gedit.
>
> O well, that was my experience.  Quick answer is use the cli.  Its
> much easier to do just what you want and no more.  Using a GUI, all
> sorts of extras are done behind the scenes (sometimes that helps, in
> this case it hurts).
>
> http://www.commandlinefu.com/ is a cool website that displays
> interesting CLI tools, and ways to use them.
>
> Dan
>
> On Thu, Jun 25, 2009 at 11:22 AM, John Abreau<jabr at blu.org> wrote:
>> That's really not an Ubuntu issue. A folder with that many files will
>> have performance issues on any OS. And not just from a GUI, either;
>> you'll have performance issues on a command line, too.
>>
>> Typically, folders get this large when an automated process keeps
>> adding new files to the folder over long periods of time.  When I write
>> scripts that need to do this, I avoid the problem by writing the files to
>> a subfolder.  My preferred method is to make yearly/daily folders, e.g.
>> instead of creating the file
>>
>>    /path/to/dir/foo12345.jpg
>>
>> I'll make it
>>
>>    /path/to/dir/2009/2009-06-25/foo12345.jpg
>>
>> The process is unlikely to run for 10,000+ years, and each yearly folder
>> will have at most 366 daily subfolders.
>>
>> I've never run into a situation where I expected the process to generate
>> 10,000+ files per day, but if the situation ever comes up, I could extend
>> the logic to a third level of hourly or minutely subsubfolders.
>>
>>
>> On Thu, Jun 25, 2009 at 8:29 AM, Brian DeLacey<bdelacey at gmail.com> wrote:
>>> Does anyone have experience with folders containing lots of files in Ubuntu?
>>> Like 10,000+? The files are relatively small - the total disk size is under
>>> 1G.
>>>
>>> Ubuntu's GUI seems to have a difficult time managing folders of this size
>>> (eg. attempting to "select all" and copy so many individual/small files to
>>> another folder hangs - or takes incredibly long.)
>>>
>>> Any suggestions on how to manage this large number of small files
>>> efficiently?
>>>
>>> Thanks,
>>> Brian
>>>
>>> --
>>> Ubuntu-us-ma mailing list
>>> Ubuntu-us-ma at lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-ma
>>>
>>>
>>
>>
>>
>> --
>> John Abreau / Executive Director, Boston Linux & Unix
>> AIM abreauj / JABBER jabr at jabber.blu.org / YAHOO abreauj / SKYPE zusa_it_mgr
>> Email jabr at blu.org / WWW http://www.abreau.net / PGP-Key-ID 0xD5C7B5D9
>> PGP-Key-Fingerprint 72 FB 39 4F 3C 3B D6 5B E0 C8 5A 6E F1 2C BE 99
>>
>> --
>> Ubuntu-us-ma mailing list
>> Ubuntu-us-ma at lists.ubuntu.com
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-ma
>>
>
>
>
> --
> In science and in mind, the impossible and the hasn't-happened-yet are
> indistinguishable.
>
> --
> Ubuntu-us-ma mailing list
> Ubuntu-us-ma at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-ma
>