Actually, after some further troubleshooting, it seems to be a networking issue. I tried downloading a file using wget, and it starts downloading fast, but as the transfer continues, the K/s starts to get lower and lower until "--.--K/s" appears. If I break out of the transfer, and restart, same result. The yum transfer seems to have the same problem. I'm SSHed into the server, and there is no sluggishness there. Does anyone have any idea why my server's transfer would start out fast at the beginning but only after 48K or so, it starts to get exponentially slower until it crawls?
The times I've seen this I've assumed that the server was artificially limiting the bandwidth, and I've tried finding another mirror. My $.02 --wpd