Christopher Ness wrote:
On Mon, 2004-03-29 at 18:04, Horacio Reyes wrote:
I am using scp to copy large amount of small files between 2 servers with a firewall in the middle, but the copy takes all CPU, this is the situation:
- The files are copied in batch of 100 tif images, in a directory with
more than 5 million files.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You need to use a method that can properly handle that many files. I hope you do not need to copy them ALL very often.
Do you have the diskspace to make a tarball and optionally compress the tarball before sending?
I'm not sure if it's the small files or the firewall that is causing your slow transfer. If it's the number of files this could be a solution.
Cheers,
Chris
