Mailing List Archive

2017-12-06 Scrum of Scrums meeting notes
https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-12-06?veaction=edit&section=29

= *2017-12-06* =
== Callouts ==
* Operations blocked on https://phabricator.wikimedia.org/
<https://phabricator.wikimedia.org/T172025>T172025
<https://phabricator.wikimedia.org/T172025> (Flow)
* Operations DBAs s8 master switchover programmed for 9th January
* Reminder! This is your last few weeks of deployments for the
year/quarter! No non-emergency deploys starts the week of December 18th.
** MediaWiki 1.30 is about to be released in the next few days, so last
call for any blockers, phabricator tag is *#mw-1.30-release
* Wikidata welcomes secondary review from somebody with knowledge about the
recent "section editing" change: https://phabricator.wikimedia.org/T181807
* Have a look at newly initiated
https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG. Feedback
welcome!

== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
** Releasing 5.7.2 (high priority fix for
https://phabricator.wikimedia.org/T69015 ) (
https://phabricator.wikimedia.org/tag/ios-app-v5.7.2/ )
** Continuing work on 5.7.3 (Faster article loading, other minor
enhancements) for release before the end of the year (
https://phabricator.wikimedia.org/project/view/2913/ )
** Continuing work on 5.8 (Reading Lists) for release next year (
https://phabricator.wikimedia.org/project/view/3131/ )

==== Android native app ====
* Blocked by:
* Blocking:
* Updates: Releasing beta 2.6.207

==== Reading Web ====
* Blocked by:
* Blocking:
* Updates:

==== Reading Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** ReadingLists performance improvements
** Wrapping up media + summary endpoints
** References needs a bit more design work

==== Multimedia ====
* Blocked by: N/A
* Blocking: N/A
* Updates: 3D nearly ready to go, probably before holidays, else shortly
thereafter. Also working on prototyping for first feature of Structured
Data on Commons.

==== Discovery ====
* Blocked by:
* Blocking:
* Updates:

===== Maps =====
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Maps mediawiki integration (Kartographer) improvements #1 on community
wishlist:
https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Tracking
** Redeploying Kartotherian
** Tracking down conflicting versions + NPM

=== Community Tech ===
* Blocked by:
* Blocking:
* Updates:

=== Contributors ===
==== Editing ====
* Blocked by:
* Blocking:
* Updates:

==== Parsing ====
* Blocked by: Services (because of Cassandra3 migration) on deployment of
HTML version bump (for section, figure-inline, html5-id, interwiki-link
changes). This is just an FYI since Services & us have already synced up
about this.
* Blocking: Reading Infra on section tag markup
* Updates:
** itwiki, dewiki, and 170 small wikis got switched from Tidy to RemexHtml
yesterday - itwiki have flagged a bunch of new issues that hadn't been
caught by linting so far. Investigation ongoing and might introduce one or
two newer categories to aid editors -- we expect these categories to be
sparsely populated
** Templatedata related fixes being deployed today -- For pre-existing
transclusions, this will prevent Parsoid from normalizing parameter order
to templatedata format

==== Global Collaboration ====
* Blocked by: MW core (or Parsing?) for code review on
https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits
* Blocking: ops for Flow dumps
* Updates:
** New tags for https://phabricator.wikimedia.org/
<https://phabricator.wikimedia.org/T167656>T167656
<https://phabricator.wikimedia.org/T167656> are on this week's train

==== UI Standardization ====
** No OOUI release this and upcoming weeks
* Ongoing:
** OOUI & based products:
*** icons: Work on icon set to be more harmonious and align to WikimediaUI
Style Guide https://phabricator.wikimedia.org/T177432 finishing up
** Unify SVG markup across Foundation products
https://phabricator.wikimedia.org/T178867
** Continuous work and per-project SVGO based optimizations, also initiated
https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG

== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** The prometheus druid exporter (to report druid metrics to graphana via
prometheus) got some attention and others (outside WMF) will be using it
** First productionized version of EventLogging backend on hadoop launched
this week. This means SQL-friendly tables on hadoop with eventlogging data,
and for the first time, ability to join to all mediawiki databases in one
query
** Wikistats 2 APIs (like Pageview API, but for edit data), alpha launch:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats
** Wikistats 2 UI, alpha launch: https://stats.wikimedia.org/v2/
** Decommissioning old DB hosts for EventLogging done, remember that
eventlogging DB is now only on analytics-slave, not on analytics-store
anymore
** Work on kafka jumbo cluster
** Rebooting one druid host for maintenance brought up unexpected issues
with druid and zookeeper, working on that

=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
** Annual Toolforge survey closed. 141 responses received (11% response
rate).
** Team in Austin this week doing an offsite + attending KubeCon conference
** Will upgrade labpuppetmaster* to v4 puppet packages after the offsite
*** Presuming everything in eqiad goes well next week

=== Fundraising Tech ===
* Blocked by: None
* Blocking: None
* Updates:
** New CiviCRM Import tool for Major Gifts
** Paying down some tech debt in Central Notice
** Fixes for fundraising vagrant role
** Continued improvements to Fundraising Dashboard
** Continuing to support FR-non-tech during the fundraiser

=== MediaWiki Platform ===
* Blocked by: N/A
* Blocking: Global Collaboration for code review on
https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits
* Updates:
** MediaWiki 1.30 Release Candidate 0 released
** MCR and Actor table reviews ongoing
** Comment table: Tried to enable WRITE_BOTH mode on testwikis, but had to
revert because it broke CentralAuth and ForeignDBViaLBRepo.
*** Will likely wait until the schema change is done everywhere before we
move anything to WRITE_BOTH.
** TemplateStyles: ParserOutput stateless transforms got merged. Some
Wikibase issues that are being dealt with.
** Firejail: will be deployed for Score
** New PSR-4 autoloader patchset https://gerrit.wikimedia.org/r/373626 ready
for review
** PoolCounter for ORES client https://gerrit.wikimedia.org/r/
<https://gerrit.wikimedia.org/r/394407>394407
<https://gerrit.wikimedia.org/r/394407> ready for testing
** MP3 transcoding deployment
** Discussions with third party MediaWiki developers regarding extension
deployment/dependency management (composer)

=== Performance ===
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Significant new performance testing documentation in progress (aimed at
developers), expected to be delivered before end of 2017
** New varnish slow log configuration in final testing
** perf testing of cross-data center database writes
** Q3 Goals planning: if you think you might need anything from the perf
team, hit me up.
** docker-based MW dev environment (in collab with WMDE)

=== Release Engineering ===
* Blocking:
** Scoring platform, https://phabricator.wikimedia.org/
<https://phabricator.wikimedia.org/T181661>T181661
<https://phabricator.wikimedia.org/T181661>
*** New scap release planned for this week that will hopefully address this
issue.
* Blocked by:
** none
* Updates:
** [MW Train] Reminder! This is your last few weeks of deployments for the
year/quarter! No non-emergency deploys starts the week of December 18th.
*** *#mw-1.30-release *MediaWiki 1.30 is about to be released in the next
few days, so last call for any blockers people might have...you
know...forgotten *

=== Research ===
* Blocked by:
* Blocking:
* Updates:

=== Scoring Platform ===
* Blocked by:
** Parallel scap SSH issue https://phabricator.wikimedia.org/T181661
* Blocking:
* Updates:
** Outage on Nov 28, incident report is going out today.
** SimpleWiki deployed to ORES
*** FYI: Global Collab, please announce and update RC Filters. The
extension is only configured on the beta cluster.
** testwiki (RevIdScorer) includes thresholds now, which makes extension
development easier, against the real server or in either of our vagrant
roles.
** ORES Ext. Refactoring
** Incoming basic support of Icelandic (No RC Filters yet)

=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
* Working on improvements to LTR training
* Reindexed wikis to enable improved katakana/hiragana mapping
https://phabricator.wikimedia.org/T179945
* Improving completion suggester interaction with namespaced prefix search
https://phabricator.wikimedia.org/T178474
* Wikidata descriptions indexed, working on fulltext search
* Working on porting Selenium tests from Ruby to JS
* Logstash upgrade to ElasticSearch 5.5 on Thu, completing the 5.5 upgrade

=== Security ===
* Blocked by:
* Blocking:
* Updates:
** Lots of development on Phan security plugin by Bawolff
** Maintanenace work on security alert configuration (Github, NSP)
** Reviews:
*** Ex:WikipediaExtracts
*** Next set of security reviews will be scheduled this week

=== Services ===
* Blocked by: none
* Blocking: Parsing till early next week
* Updates:
** Round of Cassandra 3 bootstraps is complete
** htmlCacheUpdate for wiktionaries are on kafka queue

=== Technical Operations ===
* Blocked by:
** Global Collaboration on Flow dumps https://phabricator.wikimedia.org/
<https://phabricator.wikimedia.org/T172025>T172025
<https://phabricator.wikimedia.org/T172025>
* Blocking:
** None
* Updates:
** Q3 goal planning started, if teams have dependencies on Ops for next
quarter, reach out
** Part of DC ops in Singapore for eqsin (new caching DC's name) rollout
** s8 (wikidata) work ongoing, work ontrack.
** Operations DBAs s8 master switchover programmed for 9th January

== Wikidata ==
* Finally got rid of the manual "Wikidata" build process, Wikidata is part
of the weekly deployment now: https://phabricator.wikimedia.org/T173818
* Introduced a custom, "compact" entity diff serialization in the
wb_changes table. Please look out for side-effects:
https://phabricator.wikimedia.org/T113468
* First patches demo the possibility to persist statements on sub-entities
utilizing Wikibase's standard wbsetclaim API (namely Forms on Lexemes):
https://phabricator.wikimedia.org/T163724
* Core changed the way it handles section editing. Wikibase hooks into this
feature, and now behaves unexpected. Review welcome:
https://phabricator.wikimedia.org/T181807

== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: 2017-12-06 Scrum of Scrums meeting notes [ In reply to ]
On Wed, Dec 6, 2017 at 11:15 AM, Grace Gellerman <ggellerman@wikimedia.org>
wrote:

> ==== Reading Infrastructure ====
> ** ReadingLists performance improvements
> ** Wrapping up media + summary endpoints
> ** References needs a bit more design work
>

Also, I'm planning to deploy the ReadingLists extension to production
tomorrow [1][2]. Sorry, I forgot completely to add that.

[1] https://phabricator.wikimedia.org/T181107
[2]
https://wikitech.wikimedia.org/w/index.php?title=Deployments&diff=1777474&oldid=1777458
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l