On Thu, 2004-09-02 at 01:55, Chris Hewitt wrote: > On Wed, 2004-09-01 at 23:58, Rodolfo J. Paiz wrote: > > On Wed, 2004-08-18 at 13:30, Thom Paine wrote: > > > On Wed, 2004-08-18 at 14:14, Rodolfo J. Paiz wrote: > > > > > > > I used to run a mirror and I kept the whole tree on my disk at home, so > > > > I have about 60GB of the Red Hat FTP tree local. <grin> I can share my > > > > rsync script and my include/exclude files, if you like. No secret. > > > > > I wonder if you would be kind enough to send me a copy too, please. Here you go, files attached. Sent to the list as well since the file sizes are so darn small. The actual script is "mirror" and is called with one parameter: which repository you want to mirror. So, "mirror redhat" and "mirror fedora" are your two current options. Of course, since these take a while to run I usually do "./mirror fedora &" so I can get on with my life. In order to reduce load on the mirror servers, I run these once a day. Then, in order to "give a little back" I configure all of my office machines and some of my more clueful friends' boxes to hit this server instead of other mirror servers, so my box at least reduces general demand on the public servers by 25 machines or so. Makes me happy. Note that my exclude files are carefully tailored to the stuff *I* want. Customize heavily before using, and test the file lists you get with the "-n" (dry-run) parameter to ensure that you will rsync what you want. One mistake can mean an additional few *gigabytes* of downloading, which is not what you want and is not nice on the public servers either. For example, I do not download any languages or architectures I don't use, I do not mirror test versions, and I only keep a select few older versions online. "mirror all" currently does not work as desired... don't know why, haven't spent the time to debug it yet. I strongly recommend that you spend some quality time with the rsync man page. There is a *ton* of useful stuff in there but you have to dig somewhat. And careful for one common mistake: --bw-limit used BYTES not bits, so to limit traffic to 64Kbps you're going to want --bw-limit=8. Place these files in their own subdirectory (anywhere... I have them in my home dir) since each run will generate a report file. I allow for one report daily to avoid overcrowding; running a second mirror job on the same day will overwrite the previous file. It should be trivial to modify this for other rsync-based mirror sites, but yell if you want any help. Cheers! -- Rodolfo J. Paiz <rpaiz@xxxxxxxxxxxxxx> Simpaticus.com
# Top-level directories in /pub/fedora/linux/core. Basically, exclude # everything except "1" and "2" for the moment. debug/ development/ SRPMS/ test/ testing/ x86_64/ # Exclude the DVD and SRPM ISO images. FC2-i386-DVD.iso
# Top-level directories in /pub/redhat. Basically, exclude everything # except "linux" and "mirror-tools" for the moment. ccm/ ccvs/ gnupro/ interchange/ rhdb/ stronghold/ support/ tux/ # Versions of Red Hat for which I don't want any files at all. 1.0/ 1.1/ 2.0/ 2.1/ 3.0.3/ 4.0/ 4.1/ 4.2/ 5.0/ 5.1/ 5.2/ 6.0/ 6.1/ 7.0/ 7.1/ 7.2/ beta/ code/ enterprise/ preview/ rawhide/ # Updates for other products. updates/other_prod/ # Languages I don't speak or don't want. de/ fr/ it/ ja/ # Architectures I don't want. alpha/
Attachment:
mirror
Description: application/shellscript
