On Thu, 2005-03-31 at 23:21 +0100, Mike Fleetwood wrote: > On Wed, 30 Mar 2005, Paul Howarth wrote: > > > On Wed, 2005-03-30 at 12:46 -0800, Kam Leo wrote: > >> On Wed, 30 Mar 2005 08:40:09 +0100, Paul Howarth <paul@xxxxxxxxxxxx> wrote: > >>> On Tue, 2005-03-29 at 22:35 +0100, Mike Fleetwood wrote: > >>>> Recently "yum check-update" and "yum update" has started producing the > >>>> error "Cannot find a valid baseurl for repo: ...". Googling suggests > >>>> that other occurrences have been caused by network connectivity errors > >>>> or miss configured yum repository locations. I am convinced that in my > >>>> case it is yum 2.2's inability to successfully download files through a > >>>> HTTP proxy that is the problem. To work around the problem I > >>>> reinstalled yum 2.1 and upgrade using "yum --exclude=yum update". > >>> > >>> yum 2.2 provides a way to specify the proxy globally or separately for > >>> each repo if required (see "man yum.conf"). Did you try that? > >> > >> The OP commented out the baseurl line in > >> /et/cyum.repos.d/fedora-updates.repo leaving only mirrorlist as the > >> repositoryid. Per 'man yum.conf': > > > > This is perfectly normal. The baseurl is in fact commented out by > > default. See: > > http://www.mirrorservice.org/sites/download.fedora.redhat.com/pub/fedora/linux/core/3/i386/os/Fedora/RPMS/fedora-release-3-8.i386.rpm//etc/yum.repos.d/fedora-updates.repo?extract=true > > > >> Either the mirrorlist list is defective or there is a bug in yum-2.2. > > > > Clearly the proxy configuration using environment variables, which used > > to work, is no longer doing so. I would however be interested to hear if > > the yum.conf way of specifying proxies now works in yum 2.2. > > Executive summary: > I was partially wrong. Yum only doesn't work properly with HTTP > proxies requiring authorisation. Yum 2.1 copes, 2.2 doesn't. Not > using mirrorlist _might_ work around it. > > > Details: > I have been doing lots of packet tracing, discovery bad assumptions > I have made, and found something closer to my actual problem with yum. > > My initial problem was with yum 2.2 on FC3 at work where I have to use > a Microsoft ISA web proxy with password authentication. Yum 2.1 often > reported errors left, right and centre but would eventually succeed in > downloading the required RPMs. When I upgraded to yum 2.2 it never > succeeded and still doesn't. More on this later ... > > Flash back to home were I did some brief testing on FC3 which is behind > Stone (http://www.gcd.org/sengoku/stone/) web proxy. It must have > failed for network problems a couple of times and I assumed that yum > 2.2 was the problem. Sorry for that. Yum 2.1 and 2.2 successfully > work though this web proxy which doesn't require authentication. > > Packet tracing of yum 2.1 and 2.2 using HTTP_PROXY environment variable > to define the Stone web proxy, which doesn't require authentication, > shows that yum works OK. Yum 2.1 actually downloads the mirror list > for all repos even if they aren't enabled. It does only download > repomd.xml for enable repos though. Yum 2.2 only gets the mirror list > for repos that are actually enabled. An improvement. > > Now on to packet tracing of yum 2.1 and 2.2 at work through my > companies MS ISA web proxy which requires authentication ... > I have found that yum doesn't send the HTTP headers correctly when yum > is using an authenticating web proxy. This is when defining the web > proxy using something like: > export HTTP_PROXY=http://username:password@xxxxxxxxxxxxxxxxxxxx:8080/ > MS ISA web proxy offers to keep the TCP connection open so it can be > used to send more HTTP request over, with the "Keep-Alive" header > which yum tries to use. A typical TCP conversation goes like this: > yum> GET ... > yum> Host: ... > yum> Accept-Encoding: identity > yum> Proxy-authorization: Basic xxxxxxxxxxxxxxxxx > yum> <--- Blank line which shouldn't be here > yum> User-agent: urlgrabber/2.9.2 > yum> > proxy< HTTP/1.1 200 OK > proxy< ... > proxy< Content-Length: ... > proxy< ... > proxy< Keep-Alive: ... > proxy< > proxy< [Requested URL contents ...] > proxy< ... > proxy< HTTP/1.1 400 Bad Request ( The data is invalid. ) > proxy< ... > proxy< Connection: close > proxy< Proxy-Connection: close > proxy< ... > yum> GET ... > yum> Host: ... > yum> Accept-Encoding: identity > yum> Proxy-authorization: Basic xxxxxxxxxxxxxxxxx > yum> <--- Blank line which shouldn't be here > yum> User-agent: urlgrabber/2.9.2 > yum> > And the connection closes or times out. > > So Yum (or urlgrabber) is incorrectly inserting a new line after the > Proxy-authorization header before adding the User-agent header. MS > ISA web proxy thinks that it's 2 separate requests, the first works and > the second results in an error message. Yum doesn't realise as it only > reads the specified number of bytes for its first request. It then > reuses the connection, sending another request. It reads the > previously buffer / unread data and see the error from the proxy. The > difference between yum 2.1 and 2.2 is that the particular files which > they try to read from closed sockets differ, probably because yum 2.1 > reads mirror lists for repos its not using and yum 2.2 doesn't. Yum > 2.1 only ever sees the failures on files it can and does re-try, so it > eventually succeeds. Where as as yum 2.2 sees the failures on getting > the mirror lists which it doesn't re-try so always fails. > > This explanation suggests not using mirrorlist parameter and switching > back to baseurl instead as a possible work around for yum 2.2. I will > try this in a later. > > With all this I have only done one or two tests with yum 2.2 specifying > proxy parameters in the configuration file. Results seem to indicate > that it just ignores these but don't count on this being the truth. Excellent detective work! I'd suggest raising a bug on yum for this, and/or contacting the author, Seth Vidal <skvidal at phy dot duke dot edu>, who I've always found to be very responsive. Paul. -- Paul Howarth <paul@xxxxxxxxxxxx>
