Re: automated test? (was Re: Linux 2.6.17.7)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, 25 Jul 2006, Matthias Andree wrote:

On Tue, 25 Jul 2006, Arjan van de Ven wrote:

well you can do such a thing withing statistical bounds; however... if
the patch already is in -git (as is -stable policy normally).. it should
have been found there already...

The sad facts I learned from Debian bug #212762 (not kernel related) that
culminated in CVE-2005-2335 (remote root exploit against older
fetchmail) and from various qmail bugs Guninski discovered:

- a bug need not necessarily be found soon after introduction

- a bug report may not convey the hint "look at this NOW, the shit
 already hit the fan"
 (sorry, I meant to write: look NOW, it's urgent and important)

- an automated test to catch non-trivial mistakes is non-trivial in
 itself, and - what I've seen with another project I was involved with,
 and more often than I found amusing - is that the test itself can be
 buggy causing bogus results.

That doesn't mean I object to automated tests, but "it should have been
found by now" (because the source is open, someone could have tested it,
whatever) just doesn't work.

what I was intending with my origional question was a series of simple 'does it compile' tests that try all the config options that are affected by the patchset in question. the purpose being to catch simple errors like the one here where the patch was diffed against the wrong tree and the result doesn't compile

i.e. if the change is the the e1000 driver, try compiling a kernel with it on, off, and as a module

obviously such a test can't be done on the huge -rc patches (they would approach an exhaustive test of all config permutations), but for -stable patches (or better still the indivdual patches that form a -stable release)

so the first part of the question boils down to

1. Given a patch that modifies file X is it possible to know what config options could be affected?

and the second part would be

2. would it make sense for the LTP or one of the big compile farms to do a series of compiles prior to the release of a -stable kernel to catch mistakes like this?

David Lang
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]
  Powered by Linux