Waqas Toor wrote:
i think Matthew Miller is a bit close to my point what i am asking. sorry if i couldnt make you guyz understand. now the complete scenario the thing is i want a script that will parse a webpage to get the latest version , now i can do it easly in php or pearl or python etc ... but the thing is the freakin requirement of my client is that to write only bash script.. now how to parse a webpage to get the latest version, say the webpage in some part says ... current version: 2.3.2 now in php i can parse the webpage to get a this point. now as i am totally dumb in bash etc so needed help ... i hope i am making my problem more clear :)
I have a bash script that I use to keep track of the latest versions of a number of packages that I build myself. It has a data directory where it downloads the "download" page for each application, plus a "versions" file where it keeps track of the current versions of everything.
When run, it makes a new copy of the "versions" file and compares it with the previous version to tell me what's new.
It's all very hacky but it works for me. Script attached. Paul.
#!/bin/bash DATADIR=~paul/lib/swv BITTORRENT_URL='http://www.bittorrent.com/dl/' BLUEFISH_URL='http://pkedu.fbt.eitn.wau.nl/~olivier/downloads/' BWWHOIS_URL='http://whois.bw.org/dist/current/HISTORY' CURL_URL='http://curl.haxx.se/' DOVECOT_URL='http://dovecot.org/releases/' DKMS_URL='http://linux.dell.com/dkms/permalink/' FETCHYAHOO_URL='http://fetchyahoo.sourceforge.net/' FIRESTUFF_URL='http://firestuff.org/' GETMAIL_URL='http://pyropus.ca/software/getmail/old-versions/' GOTMAIL_URL='http://sourceforge.net/project/showfiles.php?group_id=96810&package_id=103482' GREPMAIL_URL='http://grepmail.sourceforge.net/' GTKWAVE_URL='http://home.nc.rr.com/gtkwave/' GTORRENTVIEWER_URL='http://sourceforge.net/project/showfiles.php?group_id=120398&package_id=131216' HOTBABE_URL='http://dindinx.net/hotbabe/download.php' LAM_URL='http://sourceforge.net/project/showfiles.php?group_id=73243&package_id=73321' LAT_URL='http://dev.mmgsecurity.com/projects/lat/' LIBIDN_URL='http://josefsson.org/libidn/releases/' LIBSPF2_URL='http://libspf2.org/download.html' MILTER_REGEX_URL='http://www.benzedrine.cx/milter-regex.html' MOD_FCGID_URL='http://fastcgi.coremail.cn/download.htm' MOIN_URL='http://sourceforge.net/project/showfiles.php?group_id=8482&package_id=8623' NMAP_URL='http://download.insecure.org/nmap/dist/' OPENVPN_URL='http://openvpn.net/release/' OPENVPN_WIN_URL='http://openvpn.se/files/install_packages_source/' PHP_GTK_URL='http://gtk.php.net/download.php' PROFTPD_URL='http://www.proftpd.org/' PROXYCHECK_URL='http://www.corpit.ru/mjt/proxycheck.html' PPTP_URL='http://sourceforge.net/project/showfiles.php?group_id=33063&package_id=25151' PPTPD_URL='http://sourceforge.net/project/showfiles.php?group_id=44827&package_id=51373' PYCRYPTO_URL='http://www.amk.ca/python/code/crypto.html' RBLCHECK_URL='http://sourceforge.net/project/showfiles.php?group_id=2081&package_id=2015' RBLDNSD_URL='http://www.corpit.ru/mjt/rbldnsd.html' SENDMAIL_URL='http://www.sendmail.org/current-release.html' SA_MILTER_URL='http://savannah.nongnu.org/download/spamass-milt/' SMBLDAP_TOOLS_URL='http://sourceforge.net/project/showfiles.php?group_id=166108' SPFMILTER_URL='http://www.acme.com/software/spfmilter/' TIDY_DOCS_URL='http://tidy.sourceforge.net/docs/' TIDY_SRC_URL='http://tidy.sourceforge.net/src/' TORRENTSNIFF_URL='http://www.highprogrammer.com/alan/perl/torrentsniff.html' TWISTED_URL='http://twistedmatrix.com/trac/wiki/TwistedProject' TWISTED_CONCH_URL='http://twistedmatrix.com/trac/wiki/TwistedConch' TWISTED_FLOW_URL='http://twistedmatrix.com/trac/wiki/TwistedFlow' TWISTED_LORE_URL='http://twistedmatrix.com/trac/wiki/TwistedLore' TWISTED_MAIL_URL='http://twistedmatrix.com/trac/wiki/TwistedMail' TWISTED_NAMES_URL='http://twistedmatrix.com/trac/wiki/TwistedNames' TWISTED_NEWS_URL='http://twistedmatrix.com/trac/wiki/TwistedNews' TWISTED_PAIR_URL='http://twistedmatrix.com/trac/wiki/TwistedPair' TWISTED_RUNNER_URL='http://twistedmatrix.com/trac/wiki/TwistedRunner' TWISTED_WEB_URL='http://twistedmatrix.com/trac/wiki/TwistedWeb' TWISTED_WEB2_URL='http://twistedmatrix.com/trac/wiki/TwistedWeb2' TWISTED_WORDS_URL='http://twistedmatrix.com/trac/wiki/TwistedWords' WEBLINT_PP_URL='http://sourceforge.net/project/showfiles.php?group_id=36221&package_id=67423' XXDIFF_URL='http://sourceforge.net/project/showfiles.php?group_id=2198&package_id=2150' ZOPE_INTERFACE_URL='http://zope.org/Products/ZopeInterface' # Run from data directory cd $DATADIR # Remove old data find . -name '[A-Z]*' -mmin +360 -exec rm -f {} \; # Grok current versions if [ "$1" != "nodownload" ]; then curl --silent --location --output bittorrent "$BITTORRENT_URL" & curl --silent --location --output bluefish "$BLUEFISH_URL" & curl --silent --location --output bw-whois "$BWWHOIS_URL" & curl --silent --location --output curl "$CURL_URL" & curl --silent --location --output dkms "$DKMS_URL" & curl --silent --location --output dovecot "$DOVECOT_URL" & curl --silent --location --output fetchyahoo "$FETCHYAHOO_URL" & curl --silent --location --output firestuff "$FIRESTUFF_URL" & curl --silent --location --output getmail "$GETMAIL_URL" & curl --silent --location --output gotmail "$GOTMAIL_URL" & curl --silent --location --output grepmail "$GREPMAIL_URL" & curl --silent --location --output gtkwave "$GTKWAVE_URL" & curl --silent --location --output gtorrentviewer "$GTORRENTVIEWER_URL" & curl --silent --location --output hot-babe "$HOTBABE_URL" & curl --silent --location --output lam "$LAM_URL" & curl --silent --location --output lat "$LAT_URL" & curl --silent --location --output libidn "$LIBIDN_URL" & curl --silent --location --output libspf2 "$LIBSPF2_URL" & curl --silent --location --output milter-regex "$MILTER_REGEX_URL" & curl --silent --location --output mod_fcgid "$MOD_FCGID_URL" & curl --silent --location --output moin "$MOIN_URL" & wait curl --silent --location --output openvpn "$OPENVPN_URL" & curl --silent --location --output openvpn-win "$OPENVPN_WIN_URL" & curl --silent --location --output nmap "$NMAP_URL" & curl --silent --location --output php-gtk "$PHP_GTK_URL" & curl --silent --location --output pptp "$PPTP_URL" & curl --silent --location --output pptpd "$PPTPD_URL" & curl --silent --location --output proftpd "$PROFTPD_URL" & curl --silent --location --output proxycheck "$PROXYCHECK_URL" & curl --silent --location --output python-crypto "$PYCRYPTO_URL" & curl --silent --location --output rblcheck "$RBLCHECK_URL" & curl --silent --location --output rbldnsd "$RBLDNSD_URL" & curl --silent --location --output sendmail "$SENDMAIL_URL" & curl --silent --location --output smbldap-tools "$SMBLDAP_TOOLS_URL" & curl --silent --location --output spamass-milter --max-time 30 "$SA_MILTER_URL" & curl --silent --location --output spfmilter "$SPFMILTER_URL" & curl --silent --location --output tidy-docs "$TIDY_DOCS_URL" & curl --silent --location --output tidy-src "$TIDY_SRC_URL" & curl --silent --location --output torrentsniff "$TORRENTSNIFF_URL" & curl --silent --location --output twisted-conch --max-time 30 "$TWISTED_CONCH_URL" & curl --silent --location --output twisted-core --max-time 30 "$TWISTED_URL" & curl --silent --location --output twisted-flow --max-time 30 "$TWISTED_FLOW_URL" & curl --silent --location --output twisted-lore --max-time 30 "$TWISTED_LORE_URL" & curl --silent --location --output twisted-mail --max-time 30 "$TWISTED_MAIL_URL" & curl --silent --location --output twisted-names --max-time 30 "$TWISTED_NAMES_URL" & curl --silent --location --output twisted-news --max-time 30 "$TWISTED_NEWS_URL" & curl --silent --location --output twisted-pair --max-time 30 "$TWISTED_PAIR_URL" & curl --silent --location --output twisted-runner --max-time 30 "$TWISTED_RUNNER_URL" & curl --silent --location --output twisted-web --max-time 30 "$TWISTED_WEB_URL" & curl --silent --location --output twisted-web2 --max-time 30 "$TWISTED_WEB2_URL" & curl --silent --location --output twisted-words --max-time 30 "$TWISTED_WORDS_URL" & curl --silent --location --output weblint++ "$WEBLINT_PP_URL" & curl --silent --location --output xxdiff "$XXDIFF_URL" & curl --silent --location --output zope-interface "$ZOPE_INTERFACE_URL" & wait fi mv -f versions versions.last ( # Extract latest BitTorrent versions sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' bittorrent | awk '/BitTorrent-[1-9][0-9.]*\.tar\.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[.]/) if (rel[2] % 2 == 1) { # printf "Found developer release %d:%d:%d\n", rel[1], rel[2], rel[3] if (rel[1] > maxdrel[1]) { split(tarball, maxdrel, /[.]/) maxdname = tarball } else if (rel[1] == maxdrel[1] && rel[2] > maxdrel[2]) { split(tarball, maxdrel, /[.]/) maxdname = tarball } else if (rel[1] == maxdrel[1] && rel[2] == maxdrel[2] && rel[3] > maxdrel[3]) { split(tarball, maxdrel, /[.]/) maxdname = tarball } } else { # printf "Found stable release %d:%d:%d\n", rel[1], rel[2], rel[3] if (rel[1] > maxsrel[1]) { split(tarball, maxsrel, /[.]/) maxsname = tarball } else if (rel[1] == maxsrel[1] && rel[2] > maxsrel[2]) { split(tarball, maxsrel, /[.]/) maxsname = tarball } else if (rel[1] == maxsrel[1] && rel[2] == maxsrel[2] && rel[3] > maxsrel[3]) { split(tarball, maxsrel, /[.]/) maxsname = tarball } } } END { printf "%-30s Stable: %-18s Developer: %s\n", "BitTorrent", maxsname, maxdname }' # Extract latest bluefish version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' bluefish | awk '/bluefish-[1-9].*\.tar\.bz2/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.bz2$", "", tarball) split(tarball, rel, /[.]/) if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } } END { printf "%-30s Stable: %s\n", "bluefish", maxname }' # Extract latest bw-whois version awk '/^version / { printf "%-30s Stable: %s\n", "bw-whois", $2; exit }' bw-whois # Extract latest curl version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' curl | awk '/^The most recent stable version of curl is/ { versionstr = $0 sub("^.*version *", "", versionstr) sub(",.*$", "", versionstr) printf "%-30s Stable: %s\n", "curl", versionstr }' # Extract latest dkms version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' dkms | awk '/dkms-[1-9].*\.tar\.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[.]/) if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } } END { printf "%-30s Stable: %s\n", "dkms", maxname }' # Extract latest dovecot version sed -e 's/[<]a href="\([^"]*\)"[>]/\1 /' -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' dovecot | tee dovecot.txt | awk '/dovecot-[0-9].*\.tar\.gz / { tarball = $1 sub("^[^-]*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[.]/) if (tarball ~ /(alpha|beta)/) { if (rel[1] > maxdevrel[1]) { split(tarball, maxdevrel, /[.]/) maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] > maxdevrel[2]) { split(tarball, maxdevrel, /[.]/) maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] == maxdevrel[2] && rel[3] > maxdevrel[3]) { split(tarball, maxdevrel, /[.]/) maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] == maxdevrel[2] && rel[3] == maxdevrel[3] && rel[4] > maxdevrel[4]) { split(tarball, maxdevrel, /[.]/) maxdevname = tarball } } else { if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] == maxrel[3] && rel[4] > maxrel[4]) { split(tarball, maxrel, /[.]/) maxname = tarball } } } END { if (maxdevname != "") { printf "%-30s Stable: %-18s Developer: %s\n", "dovecot", maxname, maxdevname } else { printf "%-30s Stable: %s\n", "dovecot", maxname } }' # Extract latest dsbl-testers version sed -e 's/[<][^>]*[>]//g' -e 's/^[| ]*//' firestuff | awk '/^Current Version: dsbl-testers-/ { thisrel = $3 sub("^dsbl-testers-", "", thisrel) printf "%-30s Stable: %s\n", "dsbl-testers", thisrel }' # Extract latest ecncheck version sed -e 's/[<][^>]*[>]//g' -e 's/^[| ]*//' firestuff | awk '/^Current Version: ecncheck-/ { thisrel = $3 sub("^ecncheck-", "", thisrel) printf "%-30s Stable: %s\n", "ecncheck", thisrel }' # Extract latest fetchyahoo version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' fetchyahoo | awk '/^Current Version:/ { printf "%-30s Stable: %s\n", "fetchyahoo", $3; exit }' # Extract latest firedns version sed -e 's/[<][^>]*[>]//g' -e 's/^[| ]*//' firestuff | awk '/^Current Version: firedns-/ { thisrel = $3 sub("^firedns-", "", thisrel) printf "%-30s Stable: %s\n", "firedns", thisrel }' # Extract latest firestring version sed -e 's/[<][^>]*[>]//g' -e 's/^[| ]*//' firestuff | awk '/^Current Version: firestring-/ { thisrel = $3 sub("^firestring-", "", thisrel) printf "%-30s Stable: %s\n", "firestring", thisrel }' # Extract latest getmail versions sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' getmail | awk '/getmail-[4-9].*\.tar\.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[.]/) if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } } /getmail-3.*\.tar\.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[.]/) if (rel[1] > maxrel3[1]) { split(tarball, maxrel3, /[.]/) maxname3 = tarball } else if (rel[1] == maxrel3[1] && rel[2] > maxrel3[2]) { split(tarball, maxrel3, /[.]/) maxname3 = tarball } else if (rel[1] == maxrel3[1] && rel[2] == maxrel3[2] && rel[3] > maxrel3[3]) { split(tarball, maxrel3, /[.]/) maxname3 = tarball } } END { printf "%-30s Stable: %s\n", "getmail3", maxname3 printf "%-30s Stable: %s\n", "getmail", maxname }' # Extract latest gotmail version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' gotmail | awk '/ \[Notes\] / { thisrel = $1 printf "%-30s Stable: %s\n", "gotmail", thisrel exit }' # Extract latest grepmail version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' grepmail | egrep 'recent version [(].*[)]' | sed -e 's/.*recent version [(]\(.*\)[)].*/\1/' | awk '{ printf "%-30s Stable: %s\n", "grepmail", $1 }' # Extract latest gtkwave version fgrep 'You can grab version ' gtkwave | sed -s 's/.* version *\([^ ]*\).*/\1/' | awk '{ printf "%-30s Stable: %s\n", "gtkwave", $1 }' # Extract latest GTorrentViewer version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' gtorrentviewer | awk '/ \[Notes\] / { thisrel = $1 printf "%-30s Stable: %s\n", "GTorrentViewer", thisrel exit }' # Extract latest hot-babe version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' hot-babe | awk '/^hot-babe-.*.tar.bz2/ { tarball = $1 sub("^hot-babe-", "", tarball) sub(".tar.bz2$", "", tarball) printf "%-30s Stable: %s\n", "hot-babe", tarball; exit }' # Extract latest lam version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' lam | awk '/ \[Notes\] / { thisrel = $1 sub("^ldap-account-manager-", "", thisrel) printf "%-30s Stable: %s\n", "ldap-account-manager", thisrel exit }' # Extract latest lam version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' lat | awk '/^lat-.*.tar.gz/ { tarball = $1 sub("^lat-", "", tarball) sub(".tar.gz$", "", tarball) printf "%-30s Stable: %s\n", "lat", tarball; exit }' # Extract latest libidn version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' libidn | awk '/libidn-[0-9].*\.tar\.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[.]/) if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } } END { printf "%-30s Stable: %s\n", "libidn", maxname }' # Extract latest libspf2 version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' libspf2 | awk '/^libspf2-.*.tar.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) printf "%-30s Stable: %s\n", "libspf2", tarball; exit }' # Extract latest milter-regex version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' milter-regex | awk '/^milter-regex-.*.tar.gz/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.gz$", "", tarball) printf "%-30s Stable: %s\n", "milter-regex", tarball; exit }' # Extract latest moin version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' moin | awk ' / \[Notes\] / { thisrel = $1 sub("^moin-", "", thisrel) if (thisrel ~ /-rc[0-9]+$/ && devrel == "") { devrel = thisrel } if (thisrel !~ /-rc[0-9]+$/ && relrel == "") { relrel = thisrel } } END { printf "%-30s Stable: %-18s Developer: %s\n", "moin", relrel, devrel }' # Extract latest mod_fcgid version sed -e 's/.*\(mod_fcgid\.[0-9.]*\.tar\.gz\)/\1\n/g' mod_fcgid | awk '/^mod_fcgid\..*.tar.gz/ { tarball = $1 sub("^[^.]*[.]", "", tarball) sub("[.]tar[.]gz.*$", "", tarball) printf "%-30s Stable: %s\n", "mod_fcgid", tarball; exit }' # Extract latest nmap version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' nmap | awk '/nmap-[3-9].*\.tar\.bz2/ { tarball = $1 sub("^[^-]*-", "", tarball) sub(".tar.bz2$", "", tarball) split(tarball, rel, /[.]/) if (tarball ~ /(ALPHA|Alpha|BETA)/) { if (rel[1] > maxdevrel[1]) { split(tarball, maxdevrel, /[.]/) maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] > maxdevrel[2]) { split(tarball, maxdevrel, /[.]/) maxdevname = tarball } } else { if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } } } END { if (maxdevname != "") { printf "%-30s Stable: %-18s Developer: %s\n", "nmap", maxname, maxdevname } else { printf "%-30s Stable: %s\n", "nmap", maxname } }' # Extract latest openvpn version sed -e 's/.*[<]a href="\([^"]*\).*/\1/g' -e 's/^[[:space:]]*//' openvpn | awk '/openvpn-[1-9].*\.tar\.gz/ { tarball = $1 sub("^[^-]*-", "", tarball) sub(".tar.gz$", "", tarball) split(tarball, rel, /[._]/) if (tarball ~ /(alpha|beta|rc)/) { reltype = rel[3] sub("(alpha|beta|rc)", "", rel[3]) gsub("[0-9]*", "", reltype) if (rel[1] > maxdevrel[1]) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] > maxdevrel[2]) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] == maxdevrel[2] && reltype > maxdevreltype) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] == maxdevrel[2] && reltype == maxdevreltype && rel[3] > maxdevrel[3]) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } } else { if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } } } END { if (maxdevname != "") { printf "%-30s Stable: %-18s Developer: %s\n", "openvpn", maxname, maxdevname } else { printf "%-30s Stable: %s\n", "openvpn", maxname } }' # Extract latest openvpn-win version sed -e 's/.*[<]a href="\([^"]*\).*/\1/g' -e 's/^[[:space:]]*//' openvpn-win | gawk '/openvpn_install_source-[1-9].*\.zip/ { tarball = $1 sub("^[^-]*-", "", tarball) sub(".zip$", "", tarball) tarball = gensub("([^-])(alpha|beta|rc)", "\\1-\\2", "g", tarball) split(tarball, rel, /[._-]/) #printf "tarball %s munged to %s\n", $1, tarball if (tarball ~ /(alpha|beta|rc)/) { reltype = rel[3] sub("(alpha|beta|rc)", "", rel[3]) gsub("[0-9]*", "", reltype) #printf "Comparing %d.%d.%s.%d against %d.%d.%s.%d for %s\n", # rel[1], rel[2], reltype, rel[3], # maxdevrel[1], maxdevrel[2], maxdevreltype, maxdevrel[3], tarball if (rel[1] > maxdevrel[1]) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] > maxdevrel[2]) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] == maxdevrel[2] && reltype > maxdevreltype) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } else if (rel[1] == maxdevrel[1] && rel[2] == maxdevrel[2] && reltype == maxdevreltype && rel[3] > maxdevrel[3]) { maxdevrel[1] = rel[1] maxdevrel[2] = rel[2] maxdevrel[3] = rel[3] maxdevreltype = reltype maxdevname = tarball } } else { #printf "Comparing %d.%d.%d against %d.%d.%d\n", # rel[1], rel[2], rel[3], # maxrel[1], maxrel[2], maxrel[3] if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[._-]/) maxrel[3] = maxrel[3] + 0 maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[._-]/) maxrel[3] = maxrel[3] + 0 maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[._-]/) maxrel[3] = maxrel[3] + 0 maxname = tarball } } } END { if (maxdevname != "") { printf "%-30s Stable: %-18s Developer: %s\n", "openvpn-win", maxname, maxdevname } else { printf "%-30s Stable: %s\n", "openvpn-win", maxname } }' # Extract latest php-gtk version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' php-gtk | awk '/^php-gtk-[^ ]* Source/ { thisrel = $1 sub("^php-gtk-", "", thisrel) printf "%-30s Stable: %s\n", "php-gtk", thisrel exit }' # Extract latest pptp version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' pptp | awk '/ \[Notes\] / { thisrel = $1 sub("^pptp-", "", thisrel) printf "%-30s Stable: %s\n", "pptp", thisrel exit }' # Extract latest pptpd version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' pptpd | awk ' / \[Notes\] / { thisrel = $1 sub("^pptpd-", "", thisrel) if ($2 != "(stable)" && devrel == "") { devrel = thisrel } if ($2 == "(stable)" && relrel == "") { relrel = thisrel } } END { printf "%-30s Stable: %-18s Developer: %s\n", "pptpd", relrel, devrel }' # Extract latest proftpd versions sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' proftpd | awk '/^Stable:/ { stable = $2 } /^Candidate:/ { relcand = $2 } END { printf "%-30s Stable: %-18s Release Candidate: %s\n", "proftpd", stable, relcand }' # Extract latest proxycheck version egrep '^Latest version' proxycheck | sed -e 's/[<][^>]*[>]//g' -e 's/.* source code is *\([^, ]*\).*/\1/' | awk '{ printf "%-30s Stable: %s\n", "proxycheck", $1 }' # Extract latest python-crypto version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' -e 's/^[^>]*[>]//' python-crypto | tee ~/fred | awk '/^pycrypto-[^ ]*\.tar\.gz/ { thisrel = $1 sub("^pycrypto-", "", thisrel) sub(".tar.gz.*", "", thisrel) printf "%-30s Stable: %s\n", "python-crypto", thisrel exit }' # Extract latest rblcheck version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' rblcheck | awk '/ \[Notes\] / { thisrel = $1 sub("^rblcheck-", "", thisrel) printf "%-30s Stable: %s\n", "rblcheck", thisrel exit }' # Extract latest rbldnsd version egrep '^Last version, *[^,]*,' rbldnsd | sed -e 's/[<][^>]*[>]//g' -e 's/.* version, *\([^,]*\),*/\1/' | awk '{ printf "%-30s Stable: %s\n", "rbldnsd", $1 }' # Extract latest sendmail version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' sendmail | awk '/^SendmailTM/ { printf "%-30s Stable: %s\n", "sendmail", $NF; exit }' # Extract latest smbldap-tools version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' smbldap-tools | awk '/smbldap-tools-.*\.tgz/ { thisrel = $1 sub("^smbldap-tools-", "", thisrel) sub("[.]tgz", "", thisrel) printf "%-30s Stable: %s\n", "smbldap-tools", thisrel exit }' # Extract latest spamass-milter version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' spamass-milter | awk '/spamass-milter-[0-9].*\.tar\.bz2/ { tarball = $1 sub("^.*-", "", tarball) sub(".tar.bz2$", "", tarball) split(tarball, rel, /[.]/) if (rel[1] > maxrel[1]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] > maxrel[2]) { split(tarball, maxrel, /[.]/) maxname = tarball } else if (rel[1] == maxrel[1] && rel[2] == maxrel[2] && rel[3] > maxrel[3]) { split(tarball, maxrel, /[.]/) maxname = tarball } } END { printf "%-30s Stable: %s\n", "spamass-milter", maxname }' # Extract latest spfmilter version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' spfmilter | awk '/^Fetch version/ { ver = $3; sub("[.]*$", "", ver); printf "%-30s Stable: %s\n", "spfmilter", ver; exit }' # Extract latest tidy versions sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' tidy-docs tidy-src | awk '/^tidy_docs_[0-9][0-9][0-9][0-9][0-9][0-9].tgz/ { tarball = $1 sub("tidy_docs_", "", tarball) sub(".tgz", "", tarball) if (tarball > docver) { docver = tarball } } /^tidy_src_[0-9][0-9][0-9][0-9][0-9][0-9].tgz/ { tarball = $1 sub("tidy_src_", "", tarball) sub(".tgz", "", tarball) if (tarball > srcver) { srcver = tarball } } END { printf "%-30s Source: %-18s Documentation: %s\n", "tidy", srcver, docver }' # Extract latest torrentsniff version sed -e 's/[<][^>]*[>]//g' -e 's/^[[:space:]]*//' torrentsniff | awk '/^Download TorrentSniff/ { printf "%-30s Stable: %s\n", "torrentsniff", $3; exit }' # Extract latest python-twisted-conch version awk '/href="http:\/\/.*\/TwistedConch-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedConch-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-conch", thisrel exit }' twisted-conch # Extract latest python-twisted-core version awk '/href="http:\/\/.*\/Twisted-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/Twisted-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-core", thisrel exit }' twisted-core # Extract latest python-twisted-flow version #awk '/href="http:\/\/.*\/TwistedFlow-.*[.]tar[.]bz2/ { # thisrel = $0 # sub("^.*/TwistedFlow-", "", thisrel) # sub("[.]tar[.]bz2.*$", "", thisrel) # printf "%-30s Stable: %s\n", "python-twisted-flow", thisrel # exit # }' twisted-flow # Extract latest python-twisted-lore version awk '/href="http:\/\/.*\/TwistedLore-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedLore-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-lore", thisrel exit }' twisted-lore # Extract latest python-twisted-mail version awk '/href="http:\/\/.*\/TwistedMail-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedMail-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-mail", thisrel exit }' twisted-mail # Extract latest python-twisted-names version awk '/href="http:\/\/.*\/TwistedNames-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedNames-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-names", thisrel exit }' twisted-names # Extract latest python-twisted-news version awk '/href="http:\/\/.*\/TwistedNews-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedNews-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-news", thisrel exit }' twisted-news # Extract latest python-twisted-pair version #awk '/href="http:\/\/.*\/TwistedPair-.*[.]tar[.]bz2/ { # thisrel = $0 # sub("^.*/TwistedPair-", "", thisrel) # sub("[.]tar[.]bz2.*$", "", thisrel) # printf "%-30s Stable: %s\n", "python-twisted-pair", thisrel # exit # }' twisted-pair # Extract latest python-twisted-runner version awk '/href="http:\/\/.*\/TwistedRunner-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedRunner-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-runner", thisrel exit }' twisted-runner # Extract latest python-twisted-web version awk '/href="http:\/\/.*\/TwistedWeb-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedWeb-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-web", thisrel exit }' twisted-web # Extract latest python-twisted-web2 version #awk '/href="http:\/\/.*\/TwistedWeb2-.*[.]tar[.]bz2/ { # thisrel = $0 # sub("^.*/TwistedWeb2-", "", thisrel) # sub("[.]tar[.]bz2.*$", "", thisrel) # printf "%-30s Stable: %s\n", "python-twisted-web2", thisrel # exit # }' twisted-web2 # Extract latest python-twisted-words version awk '/href="http:\/\/.*\/TwistedWords-.*[.]tar[.]bz2/ { thisrel = $0 sub("^.*/TwistedWords-", "", thisrel) sub("[.]tar[.]bz2.*$", "", thisrel) printf "%-30s Stable: %s\n", "python-twisted-words", thisrel exit }' twisted-words # Extract latest weblint++ version sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' weblint++ | awk '/ \[Notes\] / { thisrel = $1 sub("^weblint..-", "", thisrel) printf "%-30s Stable: %s\n", "weblint++", thisrel exit }' # Extract latest xxdiff versions sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' xxdiff | awk '/ \[Notes\] / { thisrel = $1 split(thisrel, rel, /[.]/) relminor = rel[2] # printf "Found release %s -> MAJ: %s MIN: %s REV: %s\n", thisrel, rel[1], rel[2], rel[3] if (relminor % 2 == 1 && devrel == "") { devrel = thisrel } if (relminor % 2 == 0 && relrel == "") { relrel = thisrel } } END { printf "%-30s Stable: %-18s Developer: %s\n", "xxdiff", relrel, devrel }' # Extract latest zope interface versions sed -e 's/[<][^>]*[>]//g' -e 's/[&]nbsp;/\n/g' -e 's/^[[:space:]]*//' zope-interface | awk 'NR == 1, /^Available Releases/ { next } /^(Development|Stable)$/ { reltype = $1 next } /^ZopeInterface-.*[.]tgz/ { tarball = $1 sub("^ZopeInterface-", "", tarball) sub("[.]tgz$", "", tarball) if (reltype == "Development" && devrel == "") { devrel = tarball } if (reltype == "Stable" && relrel == "") { relrel = tarball } } END { printf "%-30s Stable: %-18s Developer: %s\n", "python-zope-interface", relrel, devrel }' ) > versions # Compare with versions from last run diff versions.last versions
