On Tue, 15 Mar 2005 10:53:44 -0500, Mark Weaver <mdw1982@xxxxxxxxxxx> wrote:
This is just a heads up. The first indexing run of the Fedora Core archives ran for over 15 hours and wasn't finished when I was forced to kill the process because it was tying up resources on my workstation. I was ssh'd into the box doing the indexing.
Perhaps you were running short on RAM? Are you using the -e option to limit the RAM requirements?
well, actually that is definitely a problem on my server here at the house, but I'm also testing the indexing on a far superior machine and so far this process has been running for the past 4.5 hours.
If it does then I can easily pass off the configuration to someone with far more bandwidth and processor power on a server that is publically accessible for the purposes of hosting a remote archive search. Judging from the activity level of this list which directly dictates the frequency of which the index would have to be refreshed I simply don't have the bandwidth or the processing power to host a live remote search host for the archives.
Are you using the hypermail indexer to index the archived mailbox files? You would create several index files using the list archive files (https://www.redhat.com/archives/fedora-list/). Then you'd specify multiple index files at search time. This way you don't have to constantly reindex everything. Do incremental updates every 15 minutes or hour or however often. Then do a full reindex every month, week or however often is needed to reduce CPU usage.
Don't hesitate asking questions on the Swish-e mailing list: http://www.swish-e.org/discuss.html
This is awesome information. guess I hadn't read far enough into the Readme file to find out about these things but thank you! I will definitely look into this. I wasn't even aware there were such options available for Swish-e.
I will have to experiement with the mailing lists I've got going here. one is just about as large as the FC archives so it will make a good guinea pig.
-- Mark ----------------------------------------------------------- Paid for by Penguins against modern appliances(R) Linux User Since 1996 Powered by Mandrake Linux 8.2, 10.0 & RH Fedora Core 3
