Horacio Reyes wrote:
Hi,
I am using scp to copy large amount of small files between 2 servers with a firewall in the middle, but the copy takes all CPU, this is the situation:
- The files are copied in batch of 100 tif images, in a directory with more than 5 million files. - the server that initiate the scp is a 1.2Ghz P3 with 2G in ram, the receiver is a double Xeon 2.6ghz with 2G in ram. The first server takes no time to send the file, but the receiver takes more than 200segs to copy. They are in a 100mb lan. - the top comand shows 98% free cpu before the scp, when scp start it takes 100% of the 2 xeon cpu. The problem is not the ram, both servers have more than 1G free when scp starts.
I tried with all the SCP flags (-q, -c blowfish, etc), and nothing works....
Regards, Horacio Reyes Rios Eng. Director.
I have found thet SCP does not like lots of small files, if you can tar the files together into on big file it will probably transfer much quicker..
