[Date Prev][Date Next] [Chronological] [Thread] [Top]

Bug and Test reporting (was RE: GEMS 1.7.4)

Here are some guidelines for submitting test results and bug reports:
  • Choose a descriptive subject that can be used to track the progress of the bug or feature.  This is the same as for any post to the support or rcr lists.  "GEMS 1.7.4" is not a descriptive subject.  "GEMS 1.7.4 testing" is also not a descriptive subject.  We use the subjects to track bugs, features, and operations.
  • It is not necessary to submit only one test result in a post;  that would be pedantic.  It is necessary that you keep the report to at least one topic.  If you are doing testing on download and you find three problems, post those three problems under the heading "1.7.4 download test results".
  • Post test results separately from questions on how GEMS should behave.  If you are unclear on how GEMS should behave while testing, post a separate message to support asking what the correct behavior should be.  If GEMS deviates from the intended behavior, include that in your test results.  Either way contact Rita to clarify the behavior in our documentation.  Never post a given behavior as a bug unless you know it is not intended, since it is very confusing to those reading the report.
  • Post test failure conditions separated clearly from success conditions.  Strictly speaking, development is not interested in the success conditions;  we only need to know what went wrong to fix it.  You might want to include in the post your test procedures, including those items that pass, for the interest of other ESG staff that want to reproduce your results.  Just keep them separate.
  • Narrow down the cause of the bug to the extent of your ability.  This means eliminating variables that are not relevant to the problem.  As a simple hypothetical example, you might get an "out of sync" error when downloading memory cards.  The database you are using might happen to use Kansas rotation.  Don't include that fact in the report though unless you think it is relevant to the "out of sync" error.  If you are unsure, then re-run the test using no rotation or some other rotation and find out.
I will be setting up a new mailing list today for discussion and tracking of bugs and testing.  I will follow up to that list on Nel's results, one topic at a time.  Everyone on the support list will automatically get subscribed to the bug list, so don't worry about missing anything.