Release & QA

Release Engineering and QA. Working to improve the quality of code and reduce the time it takes users to get it.

Start:	2013-08-21
Team members:	Antoine Musso, Sam Reed, Bryan Davis, Željko Filipin, Andre Klapper, Rummana Yasmeen
Lead:	Greg Grossmeier, Chris McMahon

This is the team responsible for Release Engineering/Management and QA at the Wikimedia Foundation. We predominately use the QA mailing list along with conversations on the appropriate IRC channels (especially #wikimedia-tech ^connect, #wikimedia-operations ^connect, #wikimedia-qa ^connect, and #wikimedia-dev ^connect).

Status

Quarterly Reviews

We maintain a list of projects that would be great to have done, but are not on our roadmap in in the near term. See the wishlist here.

See /Meetings for notes from the Showcase meetings of the Release and QA Team.

For past quarter's checkins, see the archives. See the WMF Engineering 13-14 goals page for the yearly view.

Status: in-progress - Process through all (useful) pain points from the Dev/Deploy review session - (Greg)
Status: Not done - Integrate HHVM support into our deployment systems - (Bryan, Greg, ytbh RelEngineer, others from Platform)
Status: Not done - start the scap(py) & trebuchet integration conversation (stretch goal)

Status: Not done - Support HHVM deployment tooling and puppet configuration testing - (Bryan, Antoine, ytbh RelEngineer)
Status: Not done - Swift cluster in beta (stretch goal)

Status: in-progress - Successfully support the release of MediaWiki 1.23 - (Antoine, Greg)
Status: in-progress - Kickoff/complete second RFP - (Greg)
Status: Not done - Investigate and create useful release/deployment metrics visualizations - (Greg)

Status: Not done - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki) - (Chris)
Status: in-progress - Retire Cloudbees Jenkins instance - (Željko)
Status: Not done - Integrate WMF Jenkins with new WMF SauceLabs account - (Željko, Antoine, Chris)
Status: in-progress - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.) - (Chris, Željko)
Status: in-progress - Add browsertests to new repos e.g. GettingStarted - (Chris, Željko)

Status: in-progress - Complete hiring and train new Release Engineer (Greg, all)
Status: in-progress - Complete hiring and train new Automation Engineer (Ruby) (Chris, all)

Status: Not done - create a plan for browser testing of MediaWiki 1.23 - Chris M
Status: Not done - Greg get firm requirements from Antoine, circle back to Mark who'll have an idea of Op's timelime for production Shinken.
Status: Not done - Figure out how to keep HHVM unit tests from delaying +2 for standard production commits (Antoine, Chris M, Zeljko)

Status: in-progress - Process through pain points
- Status: Done - config changes not being deployed after merge (RT ticket filed], successfully in Ops' court)
- Status: in-progress - WMF product should be consulted on some shellbugs
Status: in-progress - Refine Deployment_tooling/Notes/Deployment_system_requirements with Bryan
Status: Not done - onboard new Release Engineer
Status: in-progress - Successfully support the release of MediaWiki 1.23
Status: in-progress - Kickoff/complete second RFP
Status: Not done - Investigate and create useful release/deployment metrics visualizations

Status: Done - Figure out how to keep HHVM unit tests from delaying +2 for standard production commits
- bug 64687
Status: Not done - Support HHVM deployment tooling and puppet configuration testing
Status: Not done - Integrate WMF Jenkins with new WMF SauceLabs account

Status: In progress Scap refactor/python port
- Done Setup test environment in beta
- In progress bug 53972 Repository .git is synchronised fine, but is broken for submodules because of hardcoded gitdir link
  - Gerrit change 130560 Build JSON git info caches during scap
  - Done Gerrit change 130498 support in GitInfo.php
- Use trebuchet to deploy scap scripts
  - Done Gerrit change 130211 Add scap/scap trebuchet target
  - In progress Gerrit change 129814 Provision scap scripts using trebuchet
Status: Not done - Integrate HHVM support into our deployment systems
Status: Not done Next generation deployment tooling
Status: Ongoing Support train deploys
- Done Created Train deploy instructions on wikitech based on noted from Sam and experience in March

(With Guillaume) (Continued from March) Set up and facilitate the community RfC about Project management tools/Review, and bring it to a decision if possible. Status: Done - RfC started on 2014-04-14
(With Guillaume) Have another IRC office hour about Project management tools Status: Done on 2014-04-17 and 2014-04-22
Gather more feedback on rebooted Annoying little bugs after GSoC start Status: In progress - Andre sent an email to five GCI mentors on 2014-04-24 and received three answers
~~Prepare introducing "Bug of the week" by talking to development teams~~ Status: Not done -- postponed to May
Have a bugday in the second half of April Status: Done: on 2014-04-29 about General MediaWiki, see Bug management/Triage/20140429

Status: in-progress - create a plan for browser testing of MediaWiki 1.23
- Antoine fleshing out a start of this idea at Zurich
Status: in-progress - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki)
Status: in-progress - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.)

Status: in-progress - Retire Cloudbees Jenkins instance - (Željko)
Status: Not done - Integrate WMF Jenkins with new WMF SauceLabs account - (Željko, Antoine, Chris)