Mailing List Archive

2017-12-06 Scrum of Scrums meeting notes

= *2017-12-06* =
== Callouts ==
* Operations blocked on
<> (Flow)
* Operations DBAs s8 master switchover programmed for 9th January
* Reminder! This is your last few weeks of deployments for the
year/quarter! No non-emergency deploys starts the week of December 18th.
** MediaWiki 1.30 is about to be released in the next few days, so last
call for any blockers, phabricator tag is *#mw-1.30-release
* Wikidata welcomes secondary review from somebody with knowledge about the
recent "section editing" change:
* Have a look at newly initiated Feedback

== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
** Releasing 5.7.2 (high priority fix for ) ( )
** Continuing work on 5.7.3 (Faster article loading, other minor
enhancements) for release before the end of the year ( )
** Continuing work on 5.8 (Reading Lists) for release next year ( )

==== Android native app ====
* Blocked by:
* Blocking:
* Updates: Releasing beta 2.6.207

==== Reading Web ====
* Blocked by:
* Blocking:
* Updates:

==== Reading Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** ReadingLists performance improvements
** Wrapping up media + summary endpoints
** References needs a bit more design work

==== Multimedia ====
* Blocked by: N/A
* Blocking: N/A
* Updates: 3D nearly ready to go, probably before holidays, else shortly
thereafter. Also working on prototyping for first feature of Structured
Data on Commons.

==== Discovery ====
* Blocked by:
* Blocking:
* Updates:

===== Maps =====
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Maps mediawiki integration (Kartographer) improvements #1 on community
** Redeploying Kartotherian
** Tracking down conflicting versions + NPM

=== Community Tech ===
* Blocked by:
* Blocking:
* Updates:

=== Contributors ===
==== Editing ====
* Blocked by:
* Blocking:
* Updates:

==== Parsing ====
* Blocked by: Services (because of Cassandra3 migration) on deployment of
HTML version bump (for section, figure-inline, html5-id, interwiki-link
changes). This is just an FYI since Services & us have already synced up
about this.
* Blocking: Reading Infra on section tag markup
* Updates:
** itwiki, dewiki, and 170 small wikis got switched from Tidy to RemexHtml
yesterday - itwiki have flagged a bunch of new issues that hadn't been
caught by linting so far. Investigation ongoing and might introduce one or
two newer categories to aid editors -- we expect these categories to be
sparsely populated
** Templatedata related fixes being deployed today -- For pre-existing
transclusions, this will prevent Parsoid from normalizing parameter order
to templatedata format

==== Global Collaboration ====
* Blocked by: MW core (or Parsing?) for code review on and surrounding stack of commits
* Blocking: ops for Flow dumps
* Updates:
** New tags for
<> are on this week's train

==== UI Standardization ====
** No OOUI release this and upcoming weeks
* Ongoing:
** OOUI & based products:
*** icons: Work on icon set to be more harmonious and align to WikimediaUI
Style Guide finishing up
** Unify SVG markup across Foundation products
** Continuous work and per-project SVGO based optimizations, also initiated

== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** The prometheus druid exporter (to report druid metrics to graphana via
prometheus) got some attention and others (outside WMF) will be using it
** First productionized version of EventLogging backend on hadoop launched
this week. This means SQL-friendly tables on hadoop with eventlogging data,
and for the first time, ability to join to all mediawiki databases in one
** Wikistats 2 APIs (like Pageview API, but for edit data), alpha launch:
** Wikistats 2 UI, alpha launch:
** Decommissioning old DB hosts for EventLogging done, remember that
eventlogging DB is now only on analytics-slave, not on analytics-store
** Work on kafka jumbo cluster
** Rebooting one druid host for maintenance brought up unexpected issues
with druid and zookeeper, working on that

=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
** Annual Toolforge survey closed. 141 responses received (11% response
** Team in Austin this week doing an offsite + attending KubeCon conference
** Will upgrade labpuppetmaster* to v4 puppet packages after the offsite
*** Presuming everything in eqiad goes well next week

=== Fundraising Tech ===
* Blocked by: None
* Blocking: None
* Updates:
** New CiviCRM Import tool for Major Gifts
** Paying down some tech debt in Central Notice
** Fixes for fundraising vagrant role
** Continued improvements to Fundraising Dashboard
** Continuing to support FR-non-tech during the fundraiser

=== MediaWiki Platform ===
* Blocked by: N/A
* Blocking: Global Collaboration for code review on and surrounding stack of commits
* Updates:
** MediaWiki 1.30 Release Candidate 0 released
** MCR and Actor table reviews ongoing
** Comment table: Tried to enable WRITE_BOTH mode on testwikis, but had to
revert because it broke CentralAuth and ForeignDBViaLBRepo.
*** Will likely wait until the schema change is done everywhere before we
move anything to WRITE_BOTH.
** TemplateStyles: ParserOutput stateless transforms got merged. Some
Wikibase issues that are being dealt with.
** Firejail: will be deployed for Score
** New PSR-4 autoloader patchset ready
for review
** PoolCounter for ORES client
<> ready for testing
** MP3 transcoding deployment
** Discussions with third party MediaWiki developers regarding extension
deployment/dependency management (composer)

=== Performance ===
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Significant new performance testing documentation in progress (aimed at
developers), expected to be delivered before end of 2017
** New varnish slow log configuration in final testing
** perf testing of cross-data center database writes
** Q3 Goals planning: if you think you might need anything from the perf
team, hit me up.
** docker-based MW dev environment (in collab with WMDE)

=== Release Engineering ===
* Blocking:
** Scoring platform,
*** New scap release planned for this week that will hopefully address this
* Blocked by:
** none
* Updates:
** [MW Train] Reminder! This is your last few weeks of deployments for the
year/quarter! No non-emergency deploys starts the week of December 18th.
*** *#mw-1.30-release *MediaWiki 1.30 is about to be released in the next
few days, so last call for any blockers people might
know...forgotten *

=== Research ===
* Blocked by:
* Blocking:
* Updates:

=== Scoring Platform ===
* Blocked by:
** Parallel scap SSH issue
* Blocking:
* Updates:
** Outage on Nov 28, incident report is going out today.
** SimpleWiki deployed to ORES
*** FYI: Global Collab, please announce and update RC Filters. The
extension is only configured on the beta cluster.
** testwiki (RevIdScorer) includes thresholds now, which makes extension
development easier, against the real server or in either of our vagrant
** ORES Ext. Refactoring
** Incoming basic support of Icelandic (No RC Filters yet)

=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
* Working on improvements to LTR training
* Reindexed wikis to enable improved katakana/hiragana mapping
* Improving completion suggester interaction with namespaced prefix search
* Wikidata descriptions indexed, working on fulltext search
* Working on porting Selenium tests from Ruby to JS
* Logstash upgrade to ElasticSearch 5.5 on Thu, completing the 5.5 upgrade

=== Security ===
* Blocked by:
* Blocking:
* Updates:
** Lots of development on Phan security plugin by Bawolff
** Maintanenace work on security alert configuration (Github, NSP)
** Reviews:
*** Ex:WikipediaExtracts
*** Next set of security reviews will be scheduled this week

=== Services ===
* Blocked by: none
* Blocking: Parsing till early next week
* Updates:
** Round of Cassandra 3 bootstraps is complete
** htmlCacheUpdate for wiktionaries are on kafka queue

=== Technical Operations ===
* Blocked by:
** Global Collaboration on Flow dumps
* Blocking:
** None
* Updates:
** Q3 goal planning started, if teams have dependencies on Ops for next
quarter, reach out
** Part of DC ops in Singapore for eqsin (new caching DC's name) rollout
** s8 (wikidata) work ongoing, work ontrack.
** Operations DBAs s8 master switchover programmed for 9th January

== Wikidata ==
* Finally got rid of the manual "Wikidata" build process, Wikidata is part
of the weekly deployment now:
* Introduced a custom, "compact" entity diff serialization in the
wb_changes table. Please look out for side-effects:
* First patches demo the possibility to persist statements on sub-entities
utilizing Wikibase's standard wbsetclaim API (namely Forms on Lexemes):
* Core changed the way it handles section editing. Wikibase hooks into this
feature, and now behaves unexpected. Review welcome:

== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
Wikitech-l mailing list
Re: 2017-12-06 Scrum of Scrums meeting notes [ In reply to ]
On Wed, Dec 6, 2017 at 11:15 AM, Grace Gellerman <>

> ==== Reading Infrastructure ====
> ** ReadingLists performance improvements
> ** Wrapping up media + summary endpoints
> ** References needs a bit more design work

Also, I'm planning to deploy the ReadingLists extension to production
tomorrow [1][2]. Sorry, I forgot completely to add that.

Wikitech-l mailing list