At the bottom is a rough draft from Christian and needs lots of work. Divide this into smaller sub-jobs which could actually done by different people:
Merge this with the posting from Jesse:
As we get closer to pushing out our first round of updates, we need to (re)visit our publishing policy. Warren had persuaded me to agree with him that an extra level of testing needs to happen prior to a public release. Currently the process in full is: 1) file a bug with packages fixing a certain flaw 2) QA the package for source aspects and general build issues 3) push the package into updates-testing once two people give "PUBLISH" votes. 4) once in updates-testing, one test of package in full production is enough to release the package into testing. The whole "updates-testing" thing is the new wrinkle that Warren talked to me about. What I'm struggling with is how to handle the -testing aspects. A) Is the updates-testing package signed, and if so, with what key? B) How is the package announced? C) How can we verify that the tester is really testing the package? D) Since we release the package across multiple releases, how long do we wait on a specific release to be tested before releasing the rest of the packages? Please comment.
And with Warrens comments:
We must be able to use our discretion, and do MORE testing on all
distributions especially for the important packages. This probably
means get GPG clearsigned "name-version-release works for me on RH7.3
after 1 hour of functionality testing. I also verified that the binary
ldd output matches RH's package" from multiple people for packages like
screen, but if apache needs upgrading even more feedback should be
submitted since it is such a complex and important package.
Another good test is using ldd and compare the output of the binaries in
RH's package and Legacy's package. It is entirely possible that
Legacy's binary in updates-testing could be missing functionality since
BuildRequires are missing.
http://www.fedora.us/pipermail/fedora-devel/2004-January/002526.html
"Careful - Missing BuildRequires for Reproducibility"
Read this thread for details about the many missing BuildRequires in
many RH packages. Multiple packages currently in LEGACY QA have this
problem and need to be fixed.
http://www.fedora.us/wiki/PendingRepository
fedora.us QA pending channel detailed on this page serves a similar need
to Legacy's updates-testing channel, but fedora.us never had this
particular problem of missing BuildRequires since we were strict on
BuildRequires requirements from the beginning.
>
> The whole "updates-testing" thing is the new wrinkle that Warren talked to
> me about. What I'm struggling with is how to handle the -testing aspects.
>
> A) Is the updates-testing package signed, and if so, with what key?
FedoraLegacy.org eventually, when the build server is in production.
> B) How is the package announced?
Has the template been finalized? Perhaps consider using sha1sum instead
of md5sum since it is less susceptible to the birthday attack?
> C) How can we verify that the tester is really testing the package?
That is why multiple testers are needed. One of the testers should be
the packager of course. People that have submitted good testing results
and especially found ERRORS in the past are to be trusted more than new
people.
> D) Since we release the package across multiple releases, how long do we
> wait on a specific release to be tested before releasing the rest of the
> packages?
>
Use best judgement. Wait for enough clearsigned feedback based upon the
importance of a package to avoid regressions. Something like apache
would need far more testing than something like screen. But NEVER push
it too soon.