Release & QA

Release Engineering and QA. Working to improve the quality of code and reduce the time it takes users to get it.

Start:	2013-08-21
Team members:	Antoine Musso, Sam Reed, Bryan Davis, Mukunda Modell, Željko Filipin, Andre Klapper, Rummana Yasmeen, Dan Duvall
Lead:	Greg Grossmeier, Chris McMahon

This is the team responsible for Release Engineering/Management and QA at the Wikimedia Foundation. We predominately use the QA mailing list along with conversations on the appropriate IRC channels (especially #wikimedia-tech ^connect, #wikimedia-operations ^connect, #wikimedia-qa ^connect, and #wikimedia-dev ^connect).

Status

Template:Wikimedia project status line Template:Add new Wikimedia engineering project status update

Quarterly Reviews

Wishlist

We maintain a list of projects that would be great to have done, but are not on our roadmap in in the near term. See the wishlist here.

Meetings

See /Meetings for notes from the Showcase meetings of the Release and QA Team.

April - July '14 Goals Progress

For past quarter's checkins, see the archives.
See the WMF Engineering 13-14 goals page for the yearly view.
Mid-quarter checkin (June 27th)

Deployment tooling

Status: in-progress - Process through all (useful) pain points from the Dev/Deploy review session - (Greg)
Status: in-progress - Integrate HHVM support into our deployment systems - (Bryan, Greg, Mukunda, others from Platform)
Status: Not done - start the scap(py) & trebuchet integration conversation (stretch goal)

Beta cluster

Status: Not done - Support HHVM deployment tooling and puppet configuration testing - (Bryan, Antoine, ytbh RelEngineer)
Status: Not done - Swift cluster in beta (stretch goal)
- Bryan created preliminary puppet classes that matches labs environment
Status: In progress - assist language team to deploy cxserver - added mid-quarter

MediaWiki Release

Status: Done - Successfully support the release of MediaWiki 1.23 - (Antoine, Greg)
Status: in-progress - Kickoff/complete second RFP - (Greg)
Status: Not done - Investigate and create useful release/deployment metrics visualizations - (Greg)

Browser tests

Status: Not done - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki) - (Chris)
Status: in-progress - Retire Cloudbees Jenkins instance - (Željko)
Status: Done - Integrate WMF Jenkins with new WMF SauceLabs account - (Željko, Antoine, Chris)
Status: Done - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.) - (Chris, Željko) Calling this done for the quarter, but we'll always be using it into the future
- Used by MobileFrontend
- Used by VisualEditor
- Used by smoke tests (part of quarterly goals)
Status: in-progress - Add browsertests to new repos e.g. GettingStarted - (Chris, Željko)

Hiring

Status: Done - Complete hiring and train new Release Engineer (Greg, all)
Status: Done - Complete hiring and train new Automation Engineer (Ruby) (Chris, all)
Status: in-progress - Complete hiring of a new QA Tester (Chris, Greg) - added mid-quarter

Quarterly Review Action items

Status: Not done - create a plan for browser testing of MediaWiki 1.23 - Chris M
Status: Not done - Greg get firm requirements from Antoine, circle back to Mark who'll have an idea of Op's timelime for production Shinken.
Status: Done - Figure out how to keep HHVM unit tests from delaying +2 for standard production commits (Antoine, Chris M, Zeljko)

June Goal Progress

Greg

Status: in-progress - Process through pain points
- Status: in-progress - WMF product should be consulted on some shellbugs
Status: Done - Successfully support the release of MediaWiki 1.23
Status: in-progress - Kickoff/complete second RFP
Status: hold - Refine Deployment_tooling/Notes/Deployment_system_requirements with Bryan
Status: Not done - Investigate and create useful release/deployment metrics visualizations

Antoine

Status: Not done - Support HHVM deployment tooling and puppet configuration testing
Status: Done - Integrate WMF Jenkins with new WMF SauceLabs account
Status: In progress - RFC for Jenkins tests isolation
- wrote and published to ops and engineering lists
- agreement by End of June
- POC by End of July
Status: Not done - RFC for cross repositories testing and integration job in gate
Status: In progress - implementing cross repository cloner (aka have repositories match branches/zuul ref)
Status: In progress - assist language team to have cxserver deployed to beta cluster

Mukunda

Status: in-progress - read all the things
Status: in-progress - WMF SUL/OAuth support T40
Status: in-progress - security component support in Phab Upstream T4893, our phab T95
- local 6 line hack
- pinging evan regularly on real long term fix
Status: in-progress - puppet wmflib beginnings

Bryan

Status: In progress Scap refactor/python port
- In progress bug 53972 Repository .git is synchronised fine, but is broken for submodules because of hardcoded gitdir link
  - Done Gerrit change 130560 Build JSON git info caches during scap
  - Done Gerrit change 130498 support in GitInfo.php
  - Code is deployed in beta (and prod) but it doesn't work as expected. See notes on bug for where I'm stuck.
Status: Not done - Integrate HHVM support into our deployment systems
- Done Met with Emir from Facebook to discuss HHVM and deployment in general
Status: Not done Next generation deployment tooling
Status: Ongoing Support train deploys
- Done Helped with deployment training hangout

Reedy

Andre

(With Guillaume) (Continued from March) Set up and facilitate the community RfC about Project management tools/Review, and bring it to a decision if possible. Status: Done - RfC started on 2014-04-14
(With Guillaume) Have another IRC office hour about Project management tools Status: Done on 2014-04-17 and 2014-04-22
Gather more feedback on rebooted Annoying little bugs after GSoC start Status: In progress - Andre sent an email to five GCI mentors on 2014-04-24 and received three answers
~~Prepare introducing "Bug of the week" by talking to development teams~~ Status: Not done -- postponed to May
Have a bugday in the second half of April Status: Done: on 2014-04-29 about General MediaWiki, see Bug management/Triage/20140429

Chris

Status: in-progress - migrate to WMF Jenkins
Status: Done - create a plan for browser testing of MediaWiki 1.23
- https://bugzilla.wikimedia.org/show_bug.cgi?id=66272
- See https://gerrit.wikimedia.org/r/#/c/133507/
  - Chris Steipp had suggestions in email for further tests that map to legacy tests in /qa/browsertests repo
  - After discussion with Rob Lanphier, will port appropriate tests from /qa/browsertests to /mediawiki/core with Dan Duvall over time
Status: in-progress - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki)
- Antoine fleshing out a start of this idea at Zurich
Status: Done - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.)
- Done for Mobile
- Done for VisualEditor
- Done for browser smoke tests
Status: in-progress - hire new QA Tester

Rummana

Željko

Dan

Read all the things
Status: in-progress - VE broswer tests in Vagrant
Status: in-progress - MobileFrontend broswer tests in Vagrant

May Goal Progress