RE: Copy with SCP - low performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



We are trying the sftp to solve the problem. With sftp you can configure the buffer (in bytes) so it does not make one conection per file (instead the 100 conection with scp) but one only conection for the whole transfer, the performance look good. I´ll put the new script in production and we will see...

Regards,
Horacio.
-------------

Hi,

I am using scp to copy large amount of small files between 2 servers with a firewall in the middle, but the copy takes all CPU, this is the situation:

- The files are copied in batch of 100 tif images, in a directory with more than 5 million files.
- the server that initiate the scp is a 1.2Ghz P3 with 2G in ram, the receiver is a double Xeon 2.6ghz with 2G in ram. The first server takes no time to send the file, but the receiver takes more than 200segs to copy. They are in a 100mb lan.
- the top comand shows 98% free cpu before the scp, when scp start it takes 100% of the 2 xeon cpu. The problem is not the ram, both servers have more than 1G free when scp starts.

I tried with all the SCP flags (-q, -c blowfish, etc), and nothing works....

Regards,
Horacio Reyes Rios
Eng. Director.
----------



**************************************************************************
Nota: La información transmitida a través de este correo electrónico es confidencial y está dirigida únicamente a su destinatario. Su reproducción, lectura o uso está prohibido a cualquier persona o entidad diferente. Aunque Computec S.A. ha realizado su mejor esfuerzo para asegurar que el presente mensaje y sus archivos anexos se encuentran libre de virus y defectos que puedan llegar a afectar los computadores o sistemas que lo reciban, no se hace responsable por la eventual transmisión de virus o programas dañinos por este conducto, y por lo tanto es responsabilidad del destinatario confirmar la existencia de este tipo de elementos al momento de recibirlo y abrirlo. Ni Computec S.A. ni ninguna de sus divisiones o dependencias aceptan responsabilidad alguna por eventuales daños o alteraciones derivados de la recepcion o uso del presente mensaje.
**************************************************************************
Note: The information transmitted through this E-mail is confidential and is intended to reach only its addressee. Reproduction and usage of this message or its contents, in whole or in part, are strictly forbidden without the prior written consent of Computec S.A. Computec S.A. has made its best efforts to ensure that this message and any attached files are free of any virus or other potentially harmful content, but makes no representations to this effect.  Addressee shall assume full responsibility for ensuring that opening or reading this message and any attachments will not result in harm to their system.  Neither Computec nor any of its divisions accepts any responsibility for damages arising from opening or reading this message or any attachments thereto.
**************************************************************************



[Index of Archives]     [Current Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux