Mailing List Archive

Community, collaboration, and cognitive biases
Hello all,

the massive thread regarding the default sidebar language link
expansion state has surfaced a number of fundamental and significant
questions regarding the working relationship between the Wikimedia
Foundation and the larger Wikimedia volunteer community. This message
represents my snapshot-in-time thinking about these questions, and I
hope others will join this thread to meaningfully advance this IMO
very important conversation.

Tl;dr version: I believe that the transparency of Wikimedia's
engineering processes, and the general quality of these processes, has
significantly improved over the last year. At the same time, I agree
with those who are observing a widening gap between staff and
volunteer contributions, and I think we need to work together to close
this gap in full awareness of the cognitive biases present in all of
us.

1) Increase in quality and transparency of our engineering processes:

I'm very grateful for the hard work accomplished by the User
Experience Team to-date. When we started the usability project, we had
never run a discrete engineering effort of comparable scale before,
and indeed, most of our larger-scale engineering projects tended to be
drowned by day-to-day emergencies and bug fixes. The team consists
primarily of people who didn't have a long prior history in Wikimedia
projects, nor with each other: they had to grow as a team, understand
our enormously complex community, while also delivering results. I
want to use this opportunity to explicitly thank Brion, who played a
key role in the orientation of the team.

The UX team has implemented a number of important practices:

* very frequent sharing of detailed work product, including mock-ups,
specs, and research results such as user test videos.
* systematic QA using an external QA firm, browser compatibility
matrix, etc.; beginnings of automated testing
* qualitative remote and in-person user testing to assess real-world
experience with the site
* moving towards data-driven decision-making using tools such as
click-tracking, systematic assessment of feedback data, various
statistics, etc.
* public and frequent blog updates both in the regular and in the
technical blog, use of the sitenotice to announce deployments
* public sandboxes demonstrating all bleeding edge improvements early
* various community-oriented discussion pages and documentation on
usability.wikimedia.org informing and supporting design and
localization
* largest systematic feedback collection about any software deployment
in the history of Wikimedia projects

Many of these were "firsts" for the Wikimedia Foundation, and they all
were firsts on this scale. Some of these practices have clearly
contributed to letting the community participate _more_ in the overall
initiative: the larger-scale outreach, the sharing of research
findings (which led to some community-driven improvements), the
carefully prepared public testbeds, etc. Some, on the other hand, have
arguably widened the staff and volunteer gap, both between staff
developers and volunteer developers, and between staff and larger
volunteer community.

I don't think Wikimedia's development processes are anywhere near
those of proprietary web companies, but it's absolutely true that
they're also not highly comparable to 100% volunteer-driven open
source projects anymore. I think there are some parallels with both
the strengths and weaknesses of open source projects with reasonably
well-funded organizations behind them, both for-profit and non-profit.

I also want to be clear that there's no doubt in my mind that our
ability to execute projects of this _type_ and at this _scale_ is
unprecedented, and a historic achievement for the Wikimedia movement.
MonoBook was implemented in 2004, and the basics of the user interface
and user experience of Wikipedia have changed very little between then
and pre-Vector.

The reasons for that are varied. Fundamentally, I believe that online
volunteer mass collaboration works best when incrementalism and
self-selection can, over a long period of time, add to an overall
product that's useful at any given time -- such as Wikipedia or
Wikimedia Commons. We've seen this work less well where we need to
collaborate in a short period of time to achieve a result of
predictably high quality (Wikinews), or where we're working over long
periods of time on a result with (perceived or actual) limited
usefulness until it is complete (Wikibooks).

I'm not saying this is inherent in volunteer mass collaboration, or
that these challenges are insurmountable, just that a) tools and
practices of volunteer collaboration tend to support self-selected
long-term incrementalism better than, shall we say, strategic
megaprojects, b) we're not necessarily even using all the tools and
practices yet that other volunteer projects successfully employ to
tackle strategic challenges.

2) Widening gap between staff and volunteer contributions:

As the Wikimedia Foundation began to professionalize, it initially
moved its focus away from some of the early experiments in volunteer
mobilization (such as the 2006 Wikimedia Foundation committees), and
moved instead towards employing and tasking WMF staff for each
critical work area. Much of 2008 was spent for this new team to get
its feet on the ground and get oriented in the world of Wikimedia, to
the detriment of developing new structures for volunteer engagement.

The strategic planning process launched in 2009 was, from the
beginning, conceptualized to mobilize the Wikimedia community in
collecting ideas and planning its own future, and has achieved
unprecedented volunteer participation in doing so. We're now shifting
towards using the StrategyWiki to help volunteers in efforts to
self-organize around particular issues, whether the WMF is giving
attention to them or not -- we'll be posting more about that soon, and
I'm pretty excited about it.

At the same time, when tackling large scale strategic projects like
the usability initiative or the bookshelf, I do believe we haven't yet
succeeded at forming the most productive collaborative relationship
possible with the larger volunteer community. I should note that we
haven't even fully succeeded at closing the gap between staff and
contractors based in San Francisco, and those who are not -- we have a
long way to go.

The staff/volunteer gap is evident when looking at the commit history
of the UsabilityInitiative codebase, for example, which is very much
dominated by WMF staff and contractors. (The notable exception, as for
all our code, is localization -- again a great example for
self-selected incrementalism.) It's also evident in the recent
conflict between a staff and a volunteer developer regarding the
language link issue, and in our overall conversation about that same
issue.

3) Closing the gap and overcoming cognitive biases:

I do think it's possible and necessary to close that gap. I think
doing so has to _start_ with discussion, but ultimately to me this
isn't so much about conversation, but about actual real-world
collaboration with practical results -- we want to get stuff done
together, and we're all working towards the same vision.

To do so, I think we have to be wary of our respective cognitive
biases and operate with the highest possible degree of self-awareness.
I find the framework of cognitive bias personally very helpful when
I assess my own actions, and the list on Wikipedia
( http://en.wikipedia.org/wiki/List_of_cognitive_biases ) is a good
starting point. I believe we've seen some examples of biases listed
there in both WMF staff and volunteer actions, such as:

* belief that the group you're a member of is diverse, whereas other
groups are not ("the staff does/is X", "the community doesn't have
the expertise to do Y") ;
* preferential treatment of people perceived to be members of
the same group, and the associated us vs. them dynamic;
* irrational escalation (tendency to defend an investment based on
cumulative prior commitment) and the bandwagon effect
(tendency to join the belief of others).

These and other biases contribute to unhealthy spirals of escalation.
There's a big picture of clear alignment of values and interests:
we've all committed ourselves to a very important mission, and we
have seen time and again that we can work together immensely
successfully in doing so. Fundamentally, I think it's the group
division that's the most important to overcome -- inside
the Wikimedia movement, but also towards readers and "outsiders."
WMF absolutely has been both part of the problem, and part of
efforts to solve it.

I entirely agree that we need to steer clear of unhealthy power dynamics.
Wikipedia editing is not a tyranny of the majority or the minority, and nor is
our engineering and design process -- ideally, the best arguments, the
best code and the best edits should prevail. I'm glad that the BOLD,
revert, discuss cycle --
http://en.wikipedia.org/wiki/Wikipedia:BOLD,_revert,_discuss_cycle --
was invoked in prior threads, which I think is a generally sensible
framework through which we get stuff done. It's not the only framework,
but it's one that we (should) stick to wherever reasonably possible.

To the extent that there is an irrational status quo bias, I think the best
way to overcome it is for us to engage the largest possible number of
people in bringing about and advocating on behalf of positive changes.

With all this in mind, here are just a few concrete ideas for closing the gap:

1) Embedding teams funded by WMF into larger, publicly visible
workgroups which include volunteers and which meet regularly e.g. via
IRC;
1 a) Outreach to grow and strengthen those workgroups with the best
skills present in the Wikimedia volunteer community;
2) Publication of mini-projects which we identify and which can be
tackled by self-organized volunteers, with mentoring by experienced
WMF staff and volunteers (happening a bit via GSoC, but not as much
outside of it yet);
3) Making development roadmaps fully transparent and open to
discussion, and sharing justifications for all key priorities;
4) Further iteration of tools and processes for rapid volunteer-driven
bug reporting, cross-browser testing, and submission of simple
patches;
5) Stimulating larger scale contests focusing on specific areas of interest;
6) More topic-focused meetings and sprints like the multimedia
usability meeting in Paris.
7) Further experimentation with tools like IdeaTorrent for large-scale
brainstorming and ranking purposes (we have a prototype running at
http://prototype.wikimedia.org/en-idea/ideatorrent/ ).

The Wikimedia Foundation regards these as crucial questions, and
we've had many conversations about them, both internally and
publicly. I hope these conversations will continue to broaden.
I personally intend to work especially on 3) in the list above, and want
to help establish 1) as a normal practice in key WMF program activities.
Howie has started a general brainstorming page here, where I've
reposted the above:

http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas

I hope you'll join me in adding to that list, and that we continue to
hold each other to the idea of building true and genuinely open forms
of collaboration, which is one of the most important cornerstones of
the Wikimedia movement.

All best,
Erik
--
Erik Möller
Deputy Director, Wikimedia Foundation

Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
Hoi,
As you may know, I had reservations about the fact that the UX was funded
for the English language Wikipedia only. This turned out well; there was
attention for right to left languages, testing environments for several
languages were created. In the tools there was time to include character
sets for several scripts. At translatewiki.net there are 137 languages with
some localisations for the UX software.

It has been a joy to see Naoko do her work; her hands on and agile
involvement has been a major contributing factor for the success of this
project. Some people will say that it is not perfect but it never is and
that is to be expected. <grin> Yes, her team gelled together ... I was happy
to observe ...

If there is one thing that will help close the gap between staff and
programming volunteers, then it is a "bugmeister" ie someone who takes the
time to analyse and comment on code provided from outside the projects. One
practical example; the Babel extension would be really welcome particularly
on smaller projects. There is nobody who has time or inclination to look at
it.. at the same time it is live on several wikis and there are several
projects who asked for it.
Thanks,
GerardM

On 9 June 2010 00:31, Erik Moeller <erik@wikimedia.org> wrote:

> Hello all,
>
> the massive thread regarding the default sidebar language link
> expansion state has surfaced a number of fundamental and significant
> questions regarding the working relationship between the Wikimedia
> Foundation and the larger Wikimedia volunteer community. This message
> represents my snapshot-in-time thinking about these questions, and I
> hope others will join this thread to meaningfully advance this IMO
> very important conversation.
>
> Tl;dr version: I believe that the transparency of Wikimedia's
> engineering processes, and the general quality of these processes, has
> significantly improved over the last year. At the same time, I agree
> with those who are observing a widening gap between staff and
> volunteer contributions, and I think we need to work together to close
> this gap in full awareness of the cognitive biases present in all of
> us.
>
> 1) Increase in quality and transparency of our engineering processes:
>
> I'm very grateful for the hard work accomplished by the User
> Experience Team to-date. When we started the usability project, we had
> never run a discrete engineering effort of comparable scale before,
> and indeed, most of our larger-scale engineering projects tended to be
> drowned by day-to-day emergencies and bug fixes. The team consists
> primarily of people who didn't have a long prior history in Wikimedia
> projects, nor with each other: they had to grow as a team, understand
> our enormously complex community, while also delivering results. I
> want to use this opportunity to explicitly thank Brion, who played a
> key role in the orientation of the team.
>
> The UX team has implemented a number of important practices:
>
> * very frequent sharing of detailed work product, including mock-ups,
> specs, and research results such as user test videos.
> * systematic QA using an external QA firm, browser compatibility
> matrix, etc.; beginnings of automated testing
> * qualitative remote and in-person user testing to assess real-world
> experience with the site
> * moving towards data-driven decision-making using tools such as
> click-tracking, systematic assessment of feedback data, various
> statistics, etc.
> * public and frequent blog updates both in the regular and in the
> technical blog, use of the sitenotice to announce deployments
> * public sandboxes demonstrating all bleeding edge improvements early
> * various community-oriented discussion pages and documentation on
> usability.wikimedia.org informing and supporting design and
> localization
> * largest systematic feedback collection about any software deployment
> in the history of Wikimedia projects
>
> Many of these were "firsts" for the Wikimedia Foundation, and they all
> were firsts on this scale. Some of these practices have clearly
> contributed to letting the community participate _more_ in the overall
> initiative: the larger-scale outreach, the sharing of research
> findings (which led to some community-driven improvements), the
> carefully prepared public testbeds, etc. Some, on the other hand, have
> arguably widened the staff and volunteer gap, both between staff
> developers and volunteer developers, and between staff and larger
> volunteer community.
>
> I don't think Wikimedia's development processes are anywhere near
> those of proprietary web companies, but it's absolutely true that
> they're also not highly comparable to 100% volunteer-driven open
> source projects anymore. I think there are some parallels with both
> the strengths and weaknesses of open source projects with reasonably
> well-funded organizations behind them, both for-profit and non-profit.
>
> I also want to be clear that there's no doubt in my mind that our
> ability to execute projects of this _type_ and at this _scale_ is
> unprecedented, and a historic achievement for the Wikimedia movement.
> MonoBook was implemented in 2004, and the basics of the user interface
> and user experience of Wikipedia have changed very little between then
> and pre-Vector.
>
> The reasons for that are varied. Fundamentally, I believe that online
> volunteer mass collaboration works best when incrementalism and
> self-selection can, over a long period of time, add to an overall
> product that's useful at any given time -- such as Wikipedia or
> Wikimedia Commons. We've seen this work less well where we need to
> collaborate in a short period of time to achieve a result of
> predictably high quality (Wikinews), or where we're working over long
> periods of time on a result with (perceived or actual) limited
> usefulness until it is complete (Wikibooks).
>
> I'm not saying this is inherent in volunteer mass collaboration, or
> that these challenges are insurmountable, just that a) tools and
> practices of volunteer collaboration tend to support self-selected
> long-term incrementalism better than, shall we say, strategic
> megaprojects, b) we're not necessarily even using all the tools and
> practices yet that other volunteer projects successfully employ to
> tackle strategic challenges.
>
> 2) Widening gap between staff and volunteer contributions:
>
> As the Wikimedia Foundation began to professionalize, it initially
> moved its focus away from some of the early experiments in volunteer
> mobilization (such as the 2006 Wikimedia Foundation committees), and
> moved instead towards employing and tasking WMF staff for each
> critical work area. Much of 2008 was spent for this new team to get
> its feet on the ground and get oriented in the world of Wikimedia, to
> the detriment of developing new structures for volunteer engagement.
>
> The strategic planning process launched in 2009 was, from the
> beginning, conceptualized to mobilize the Wikimedia community in
> collecting ideas and planning its own future, and has achieved
> unprecedented volunteer participation in doing so. We're now shifting
> towards using the StrategyWiki to help volunteers in efforts to
> self-organize around particular issues, whether the WMF is giving
> attention to them or not -- we'll be posting more about that soon, and
> I'm pretty excited about it.
>
> At the same time, when tackling large scale strategic projects like
> the usability initiative or the bookshelf, I do believe we haven't yet
> succeeded at forming the most productive collaborative relationship
> possible with the larger volunteer community. I should note that we
> haven't even fully succeeded at closing the gap between staff and
> contractors based in San Francisco, and those who are not -- we have a
> long way to go.
>
> The staff/volunteer gap is evident when looking at the commit history
> of the UsabilityInitiative codebase, for example, which is very much
> dominated by WMF staff and contractors. (The notable exception, as for
> all our code, is localization -- again a great example for
> self-selected incrementalism.) It's also evident in the recent
> conflict between a staff and a volunteer developer regarding the
> language link issue, and in our overall conversation about that same
> issue.
>
> 3) Closing the gap and overcoming cognitive biases:
>
> I do think it's possible and necessary to close that gap. I think
> doing so has to _start_ with discussion, but ultimately to me this
> isn't so much about conversation, but about actual real-world
> collaboration with practical results -- we want to get stuff done
> together, and we're all working towards the same vision.
>
> To do so, I think we have to be wary of our respective cognitive
> biases and operate with the highest possible degree of self-awareness.
> I find the framework of cognitive bias personally very helpful when
> I assess my own actions, and the list on Wikipedia
> ( http://en.wikipedia.org/wiki/List_of_cognitive_biases ) is a good
> starting point. I believe we've seen some examples of biases listed
> there in both WMF staff and volunteer actions, such as:
>
> * belief that the group you're a member of is diverse, whereas other
> groups are not ("the staff does/is X", "the community doesn't have
> the expertise to do Y") ;
> * preferential treatment of people perceived to be members of
> the same group, and the associated us vs. them dynamic;
> * irrational escalation (tendency to defend an investment based on
> cumulative prior commitment) and the bandwagon effect
> (tendency to join the belief of others).
>
> These and other biases contribute to unhealthy spirals of escalation.
> There's a big picture of clear alignment of values and interests:
> we've all committed ourselves to a very important mission, and we
> have seen time and again that we can work together immensely
> successfully in doing so. Fundamentally, I think it's the group
> division that's the most important to overcome -- inside
> the Wikimedia movement, but also towards readers and "outsiders."
> WMF absolutely has been both part of the problem, and part of
> efforts to solve it.
>
> I entirely agree that we need to steer clear of unhealthy power dynamics.
> Wikipedia editing is not a tyranny of the majority or the minority, and nor
> is
> our engineering and design process -- ideally, the best arguments, the
> best code and the best edits should prevail. I'm glad that the BOLD,
> revert, discuss cycle --
> http://en.wikipedia.org/wiki/Wikipedia:BOLD,_revert,_discuss_cycle --
> was invoked in prior threads, which I think is a generally sensible
> framework through which we get stuff done. It's not the only framework,
> but it's one that we (should) stick to wherever reasonably possible.
>
> To the extent that there is an irrational status quo bias, I think the best
> way to overcome it is for us to engage the largest possible number of
> people in bringing about and advocating on behalf of positive changes.
>
> With all this in mind, here are just a few concrete ideas for closing the
> gap:
>
> 1) Embedding teams funded by WMF into larger, publicly visible
> workgroups which include volunteers and which meet regularly e.g. via
> IRC;
> 1 a) Outreach to grow and strengthen those workgroups with the best
> skills present in the Wikimedia volunteer community;
> 2) Publication of mini-projects which we identify and which can be
> tackled by self-organized volunteers, with mentoring by experienced
> WMF staff and volunteers (happening a bit via GSoC, but not as much
> outside of it yet);
> 3) Making development roadmaps fully transparent and open to
> discussion, and sharing justifications for all key priorities;
> 4) Further iteration of tools and processes for rapid volunteer-driven
> bug reporting, cross-browser testing, and submission of simple
> patches;
> 5) Stimulating larger scale contests focusing on specific areas of
> interest;
> 6) More topic-focused meetings and sprints like the multimedia
> usability meeting in Paris.
> 7) Further experimentation with tools like IdeaTorrent for large-scale
> brainstorming and ranking purposes (we have a prototype running at
> http://prototype.wikimedia.org/en-idea/ideatorrent/ ).
>
> The Wikimedia Foundation regards these as crucial questions, and
> we've had many conversations about them, both internally and
> publicly. I hope these conversations will continue to broaden.
> I personally intend to work especially on 3) in the list above, and want
> to help establish 1) as a normal practice in key WMF program activities.
> Howie has started a general brainstorming page here, where I've
> reposted the above:
>
> http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas
>
> I hope you'll join me in adding to that list, and that we continue to
> hold each other to the idea of building true and genuinely open forms
> of collaboration, which is one of the most important cornerstones of
> the Wikimedia movement.
>
> All best,
> Erik
> --
> Erik Möller
> Deputy Director, Wikimedia Foundation
>
> Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 6:31 PM, Erik Moeller <erik@wikimedia.org> wrote:
> With all this in mind, here are just a few concrete ideas for closing the gap:
>
> 1) Embedding teams funded by WMF into larger, publicly visible
> workgroups which include volunteers and which meet regularly e.g. via
> IRC;
> 1 a) Outreach to grow and strengthen those workgroups with the best
> skills present in the Wikimedia volunteer community;
> 2) Publication of mini-projects which we identify and which can be
> tackled by self-organized volunteers, with mentoring by experienced
> WMF staff and volunteers (happening a bit via GSoC, but not as much
> outside of it yet);
> 3) Making development roadmaps fully transparent and open to
> discussion, and sharing justifications for all key priorities;
> 4) Further iteration of tools and processes for rapid volunteer-driven
> bug reporting, cross-browser testing, and submission of simple
> patches;
> 5) Stimulating larger scale contests focusing on specific areas of interest;
> 6) More topic-focused meetings and sprints like the multimedia
> usability meeting in Paris.
> 7) Further experimentation with tools like IdeaTorrent for large-scale
> brainstorming and ranking purposes (we have a prototype running at
> http://prototype.wikimedia.org/en-idea/ideatorrent/ ).

None of these strike me as essential for a successful bazaar-style
development model, except (4). I'd say some of the most important
things are (from a development point of view, not talking about
non-developer communities)

1) Rely on public mailing lists for communication as much as possible,
supplemented by IRC channels (preferably publicly logged). Private
e-mail, face-to-face meetings, and telephone meetings are impossible
for volunteers to join in on, so they should be used as little as
possible. Don't try to move everyone to San Francisco -- if you do
that, they'll inevitably rely heavily on face-to-face communication
and lock out volunteers. I get the impression this has happened with
the usability team.

2) Make sure that every paid developer spends time dealing with the
community. This can include giving support to end users, discussing
things with volunteers, reviewing patches, etc. They should be doing
this on paid time, and they should be discussing their personal
opinions without consulting with anyone else (i.e., not summarizing
official positions). Paid developers and volunteers have to get to
know each other and have to be able to discuss MediaWiki together.

3) Don't needlessly fork discussion fora. The Usability Initiative
made its own public wiki, IRC chat, etc., and those are used
overwhelmingly by paid people. This might not have happened if they
stuck to existing, established fora like wikitech-l, #mediawiki, and
mediawiki.org, where there are already a lot of community members
reading.

The basic attitude has to be that paid developers are treated
identically to volunteers, except that you can tell the former what to
do and expect them to put in more time. There should not be
communication between paid developers and the community, paid
developers should be an integral *part* of the community rather than a
separate group of people.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
> The basic attitude has to be that paid developers are treated
> identically to volunteers, except that you can tell the former what to
> do and expect them to put in more time.  There should not be
> communication between paid developers and the community, paid
> developers should be an integral *part* of the community rather than a
> separate group of people.

I do believe that it's very true and very universal (so could and
should be treated this way far beyond platform development).

I mean that "communication between us and them" is error-prone concept
in it very bottom.

The should be only community as "us".

Seemingly/IMHO Erik Zachte' 'story' is right example: he started his
statics endeavor as volunteer and as soon as WMF started to 'expect
him to put in more time' he became paid person.

Pavlo


On Wed, Jun 9, 2010 at 1:55 AM, Aryeh Gregor
<Simetrical+wikilist@gmail.com> wrote:
> On Tue, Jun 8, 2010 at 6:31 PM, Erik Moeller <erik@wikimedia.org> wrote:
>> With all this in mind, here are just a few concrete ideas for closing the gap:
>>
>> 1) Embedding teams funded by WMF into larger, publicly visible
>> workgroups which include volunteers and which meet regularly e.g. via
>> IRC;
>> 1 a) Outreach to grow and strengthen those workgroups with the best
>> skills present in the Wikimedia volunteer community;
>> 2) Publication of mini-projects which we identify and which can be
>> tackled by self-organized volunteers, with mentoring by experienced
>> WMF staff and volunteers (happening a bit via GSoC, but not as much
>> outside of it yet);
>> 3) Making development roadmaps fully transparent and open to
>> discussion, and sharing justifications for all key priorities;
>> 4) Further iteration of tools and processes for rapid volunteer-driven
>> bug reporting, cross-browser testing, and submission of simple
>> patches;
>> 5) Stimulating larger scale contests focusing on specific areas of interest;
>> 6) More topic-focused meetings and sprints like the multimedia
>> usability meeting in Paris.
>> 7) Further experimentation with tools like IdeaTorrent for large-scale
>> brainstorming and ranking purposes (we have a prototype running at
>> http://prototype.wikimedia.org/en-idea/ideatorrent/ ).
>
> None of these strike me as essential for a successful bazaar-style
> development model, except (4).  I'd say some of the most important
> things are (from a development point of view, not talking about
> non-developer communities)
>
> 1) Rely on public mailing lists for communication as much as possible,
> supplemented by IRC channels (preferably publicly logged).  Private
> e-mail, face-to-face meetings, and telephone meetings are impossible
> for volunteers to join in on, so they should be used as little as
> possible.  Don't try to move everyone to San Francisco -- if you do
> that, they'll inevitably rely heavily on face-to-face communication
> and lock out volunteers.  I get the impression this has happened with
> the usability team.
>
> 2) Make sure that every paid developer spends time dealing with the
> community.  This can include giving support to end users, discussing
> things with volunteers, reviewing patches, etc.  They should be doing
> this on paid time, and they should be discussing their personal
> opinions without consulting with anyone else (i.e., not summarizing
> official positions).  Paid developers and volunteers have to get to
> know each other and have to be able to discuss MediaWiki together.
>
> 3) Don't needlessly fork discussion fora.  The Usability Initiative
> made its own public wiki, IRC chat, etc., and those are used
> overwhelmingly by paid people.  This might not have happened if they
> stuck to existing, established fora like wikitech-l, #mediawiki, and
> mediawiki.org, where there are already a lot of community members
> reading.
>
> The basic attitude has to be that paid developers are treated
> identically to volunteers, except that you can tell the former what to
> do and expect them to put in more time.  There should not be
> communication between paid developers and the community, paid
> developers should be an integral *part* of the community rather than a
> separate group of people.
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 3:31 PM, Erik Moeller <erik@wikimedia.org> wrote:

<a long, thoughtful, and really important post>

> 7) Further experimentation with tools like IdeaTorrent for large-scale
> brainstorming and ranking purposes (we have a prototype running at
> http://prototype.wikimedia.org/en-idea/ideatorrent/ ).

I was super excited to see this go up the other day. Can we move it
to, say, strategywiki to try out? Any thoughts on scope? Not having
used this much I don't know if you'd want separate torrents by broad
topic (ideas for the wikimedia foundation, ideas for mediawiki, ideas
for the projects, ideas for outreach) or one big one that could be
sorted by topic.

-- phoebe

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 4:28 PM, phoebe ayers <phoebe.wiki@gmail.com> wrote:
> On Tue, Jun 8, 2010 at 3:31 PM, Erik Moeller <erik@wikimedia.org> wrote:
>> 7) Further experimentation with tools like IdeaTorrent for large-scale
>> brainstorming and ranking purposes (we have a prototype running at
>> http://prototype.wikimedia.org/en-idea/ideatorrent/ ).
>
> I was super excited to see this go up the other day. Can we move it
> to, say, strategywiki to try out? Any thoughts on scope? Not having
> used this much I don't know if you'd want separate torrents by broad
> topic (ideas for the wikimedia foundation, ideas for mediawiki, ideas
> for the projects, ideas for outreach) or one big one that could be
> sorted by topic.

We originally considered using IdeaTorrent for strategy's Call for
Proposals, then opted for a wiki-based solution. There were two main
obstacles: Our timing (we wanted to get started quickly, and we didn't
have time to hack things in like SUL support, etc.) and lack of
multilingual support. Our system worked fine, but I did experience
some pangs of regret for not having some of IdeaTorrent's
capabilities.

Incidentally, as Erik mentioned in his email, we're developing some
IdeaTorrent-like mechanisms on strategy wiki as a way to encourage
self-organization and activation around good ideas. For a slightly
out-of-date description, see:

http://strategy.wikimedia.org/wiki/Process/Activating_volunteers

Discussion there is encouraged.

=Eugene

--
======================================================================
Eugene Eric Kim ................................ http://xri.net/=eekim
Blue Oxen Associates ........................ http://www.blueoxen.com/
======================================================================

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 6:55 PM, Aryeh Gregor
<Simetrical+wikilist@gmail.com<Simetrical%2Bwikilist@gmail.com>
> wrote:

> 2) Make sure that every paid developer spends time dealing with the
> community. This can include giving support to end users, discussing
> things with volunteers, reviewing patches, etc. They should be doing
> this on paid time, and they should be discussing their personal
> opinions without consulting with anyone else (i.e., not summarizing
> official positions). Paid developers and volunteers have to get to
> know each other and have to be able to discuss MediaWiki together.
> [...]
>
> The basic attitude has to be that paid developers are treated
> identically to volunteers, except that you can tell the former what to
> do and expect them to put in more time. There should not be
> communication between paid developers and the community, paid
> developers should be an integral *part* of the community rather than a
> separate group of people.
>
>

I really agree with this sentiment, but it seems difficult to get staff to
really be part of the community unless they're _from_ the community. The
developers I've seen discuss their personal opinions on public fora
(especially in ways that are accepted in an open community but not in a
business environment—one example would be criticizing their co-workers) have
been those who were recruited from the community. There's nothing wrong
with having outsiders as employees, but communication is rather different in
the outside world, and I get the sense that a lot of the people hired from
elsewhere aren't necessarily familiar with the Wikimedia Way™ of discussing
things—and even if they understand that it's there, I'm not sure they always
understand that they're supposed to join in.

I recall someone once suggesting that all employees be required to choose a
Wikimedia project and get involved in it. I haven't thought through the
practical implications, but it always seemed like a cute idea, at least.
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 8:48 PM, Benjamin Lees <emufarmers@gmail.com> wrote:
> I really agree with this sentiment, but it seems difficult to get staff to
> really be part of the community unless they're _from_ the community.  The
> developers I've seen discuss their personal opinions on public fora
> (especially in ways that are accepted in an open community but not in a
> business environment—one example would be criticizing their co-workers) have
> been those who were recruited from the community.  There's nothing wrong
> with having outsiders as employees, but communication is rather different in
> the outside world, and I get the sense that a lot of the people hired from
> elsewhere aren't necessarily familiar with the Wikimedia Wayâ„¢ of discussing
> things—and even if they understand that it's there, I'm not sure they always
> understand that they're supposed to join in.

It's not specific to Wikimedia, it's practically universal in
open-source development. To get it to happen, you need pushing from
the top: formally stating it as part of people's job duties (so they
don't feel they have to do "real work" instead), and forcing them to
engage by only giving them public media to discuss things in with
their co-workers. I recall reading that IBM improved its
participation in the Linux kernel community by getting rid of all
internal communications among its kernel developers, meaning they had
to use the public project lists to bounce ideas off anyone.

It's also worth pointing out that a good way *not* to engage with the
community is to not touch preexisting code that volunteers are
familiar with. All the Usability Initiative stuff was created
separately: a new skin, and all other functionality in extensions.
There's mostly no technical reason for this; at least some could have
been integrated with the existing code. Putting most of your code in
a directory called "UsabilityInitiative" is a good way of signaling
"this is ours, not yours", whether that was the intent or not. If it
had touched code that volunteers were familiar with, they would have
been more engaged from the start, because they'd have stronger
opinions on the changes and no presumption that they shouldn't touch
it.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Wed, Jun 9, 2010 at 11:28 AM, Aryeh Gregor
<Simetrical+wikilist@gmail.com> wrote:
> ...
> It's also worth pointing out that a good way *not* to engage with the
> community is to not touch preexisting code that volunteers are
> familiar with.  All the Usability Initiative stuff was created
> separately: a new skin, and all other functionality in extensions.

add to that a new wiki, where 8 or 9 of the 10 sysops are staff members.

http://usability.wikimedia.org/wiki/Special:ListUsers/sysop

--
John Vandenberg

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
Hoi,
The water is nice, I have tried it ..
http://prototype.wikimedia.org/en-idea/ideatorrent/
I even blogged about it ...
http://ultimategerardm.blogspot.com/2010/06/ideatorrent-for-mediawiki-ideas.html
Thanks,
GerardM

On 9 June 2010 01:28, phoebe ayers <phoebe.wiki@gmail.com> wrote:

> On Tue, Jun 8, 2010 at 3:31 PM, Erik Moeller <erik@wikimedia.org> wrote:
>
> <a long, thoughtful, and really important post>
>
> > 7) Further experimentation with tools like IdeaTorrent for large-scale
> > brainstorming and ranking purposes (we have a prototype running at
> > http://prototype.wikimedia.org/en-idea/ideatorrent/ ).
>
> I was super excited to see this go up the other day. Can we move it
> to, say, strategywiki to try out? Any thoughts on scope? Not having
> used this much I don't know if you'd want separate torrents by broad
> topic (ideas for the wikimedia foundation, ideas for mediawiki, ideas
> for the projects, ideas for outreach) or one big one that could be
> sorted by topic.
>
> -- phoebe
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/06/2010 03:28, Aryeh Gregor wrote:

> I recall reading that IBM improved its
> participation in the Linux kernel community by getting rid of all
> internal communications among its kernel developers, meaning they had
> to use the public project lists to bounce ideas off anyone.

I think this idea is key.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJMD083AAoJEHCAuDvx9Z6L4bIIAON6OAXBQTDd8xYycxCX84JV
yhRCJBJM76mGCePuKnY7CoGdOi8tPnweLRCDjn2xBBE6N4rbkfjwf/FbeQv2a+YK
JO6jg1CHq23QtidNMsJyexufnWuIG+Rjf0AoDFBlWOCW46Fk4GjcAb+gt50EQeL8
POqXJ8AJ2t2UcBJX1CD+ZAuGVU4Nw1IxK1sbSJNjHRE6SJqyRVy4YnJ6Eqiammzk
sV7h0Z0EY750etIYErpE7zTShCTlLFdxYzlzAKMlfalIL/BZgYhCsIKSe5AWcVXM
99/40Jx15t0HKcGoleN5oYzZd+hVTlgS3C/NrHlpRGb5A6f1xsF6Dh/+sl7UbhM=
=Flfm
-----END PGP SIGNATURE-----

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Wed, Jun 9, 2010 at 12:55 AM, Aryeh Gregor
<Simetrical+wikilist@gmail.com> wrote:
> 2) Make sure that every paid developer spends time dealing with the
> community.  This can include giving support to end users, discussing
> things with volunteers, reviewing patches, etc.  They should be doing
> this on paid time, and they should be discussing their personal
> opinions without consulting with anyone else (i.e., not summarizing
> official positions).  Paid developers and volunteers have to get to
> know each other and have to be able to discuss MediaWiki together.

I like the "discussing their personal opinions without consulting with
anyone else" bit, and you bring up a very good point.

I don't think (and I don't mean to imply that anyone else does) that
anyone's conspiring to keep the community out, or saying "leave this
to the professionals, we know better." When you're hired onto a team,
though, you're wary of saying anything that would cause strife or
confusion. This isn't necessarily out of fear of retribution from
your employer—it's simply conventional professional ethics, and it's
usually not even a conscious thing. (It's also not limited to paid
staff—the people we put on the Board specifically for their vocal
opinions on things often fall into this, for understandable reasons.)

This "united front," however, results in the "us vs. them" mentality
that we're all now lamenting. Volunteers are now "giving feedback"
rather than "making decisions," as Greg put very well, and we wind up
with questionable UI decisions becoming surrogate arguments for the
roles of community and staff.

I don't think that there's a magic fix for this—it's simply a matter
of culture, and making sure everyone involved understands it and can
work effectively in it. We can point to the little things, but the
systemic problem needs to be addressed.

Austin

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
I think the idea that Aryeh Gregor brought up is incredible. We should
follow the strategy use by IBM in helping develop Linux. Open all
discussion to the Wikimedia community will bring the power of Wikipedia's
collaborative process to the operations of of Wikimedia. Volunteers would
get involved in all aspects of Wikimedia from advertising to programing. We
have build the greatest encyclopedia in the world now we can build the
greatest non profit.

> I recall reading that IBM improved its
> participation in the Linux kernel community by getting rid of all
> internal communications among its kernel developers, meaning they had
> to use the public project lists to bounce ideas off anyone.

James Heilman, MD
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 9:28 PM, Aryeh Gregor
<Simetrical+wikilist@gmail.com> wrote:
> It's not specific to Wikimedia, it's practically universal in
> open-source development.  To get it to happen, you need pushing from
> the top: formally stating it as part of people's job duties (so they
> don't feel they have to do "real work" instead), and forcing them to
> engage by only giving them public media to discuss things in with
> their co-workers.  I recall reading that IBM improved its
> participation in the Linux kernel community by getting rid of all
> internal communications among its kernel developers, meaning they had
> to use the public project lists to bounce ideas off anyone.

Here's the reference for that:

"""
Dan Frye's keynote reflecting on IBM's 10+ years of experience with
Linux was easily one of the best of the day. IBM's experience has
certainly not been 100% smooth sailing; there were a lot of mistakes
made along the way. As Dan put it, it is relatively easy for a company
to form a community around itself, but it's much harder - and more
valuable - to join an established community under somebody else's
control.

A number of lessons learned were offered, starting with an
encouragement to get projects out into the community early and to
avoid closed-door communications. IBM discovered the hard way that
dumping large blocks of completed code into the kernel community was
not going to be successful. The community must be involved earlier
than that. To help in that direction, IBM prohibited the use of
internal communications for many projects, forcing developers to have
their discussions in public forums.
"""
http://lwn.net/Articles/383945/

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
IBMs decision to get rid of all internal communication sounds to me as a
very good practice for us.
It also fits in well with the wikipedia culture of consensus in decision
making.

Following this comm. strategy involves the large volunteer community, and
taps on the vast knowledge of our community.

thank you, Aryeh, for bringing this up.
teun


> It's not specific to Wikimedia, it's practically universal in
> open-source development. To get it to happen, you need pushing from
> the top: formally stating it as part of people's job duties (so they
> don't feel they have to do "real work" instead), and forcing them to
> engage by only giving them public media to discuss things in with
> their co-workers. I recall reading that IBM improved its
> participation in the Linux kernel community by getting rid of all
> internal communications among its kernel developers, meaning they had
> to use the public project lists to bounce ideas off anyone.
>
> It's also worth pointing out that a good way *not* to engage with the
> community is to not touch preexisting code that volunteers are
> familiar with. All the Usability Initiative stuff was created
> separately: a new skin, and all other functionality in extensions.
> There's mostly no technical reason for this; at least some could have
> been integrated with the existing code. Putting most of your code in
> a directory called "UsabilityInitiative" is a good way of signaling
> "this is ours, not yours", whether that was the intent or not. If it
> had touched code that volunteers were familiar with, they would have
> been more engaged from the start, because they'd have stronger
> opinions on the changes and no presumption that they shouldn't touch
> it.
>
> _______________________________________________
> foundation-l mailing list
> foundation-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Tue, Jun 8, 2010 at 6:28 PM, Aryeh Gregor
<Simetrical+wikilist@gmail.com<Simetrical%2Bwikilist@gmail.com>
> wrote:

> It's not specific to Wikimedia, it's practically universal in
> open-source development. To get it to happen, you need pushing from
> the top: formally stating it as part of people's job duties (so they
> don't feel they have to do "real work" instead), and forcing them to
> engage by only giving them public media to discuss things in with
> their co-workers.


Hi Aryeh,

First, let me state from the outset: I think it's great to work out in the
open, and I find that the people I'm working with at WMF are at the leading
edge of community collaboration on a number of fronts (compared to peers at
typical tech companies or even non-profits). Feel Free to ping me on IRC,
email about anything I'm working on right now (that goes for others as
well). Note also: in the spirit of this conversation, I didn't run this by
anyone at WMF, and I'm still using my @wikimedia.org address anyway (and I'm
only a contractor). We'll see if I get in any hot water for that ;-) Just
so you know, part of my job here (besides work on Pending Changes) is to
work on development process at WMF, so this thread is pretty relevant to my
day job here.

As you know, any time you want to compel someone to do something, there's
always the carrot and the stick. One thing I don't like about the way
you've phrased that is that is that you seem to be advocating the stick. Am
I reading that right?

One undertone that I've witnessed everywhere is that people in open source
communities that have a clear organizational "owner" is that there is a very
uneven distribution of people who want a peer-to-peer relationship versus a
customer-vendor relationship. This makes it really difficult to work out in
the public, because some people seem to prefer the trappings of a
peer-to-peer relationship (let me in on your early thinking, publish your
roadmaps, work in the fishbowl), where others prefer the trappings of the
customer-vendor relationship (the customer is always right, the customer is
the boss). Some will even go so far as to want a customer-to-peer
relationship, which is clearly not sustainable. To be really clear here,
most of my impressions on this topic come from my previous work experience
(been doing the corporate open source thing for a while), and only in a
limited way with this community, but I've seen hints that the
WMF<=>community relationship has some of the same traits.

From the vantage point of the "vendor" in this case, the problem is
compounded by the cognitive bias Erik pointed to (belief that the group
you're a member of is diverse, whereas other groups are not). The net
result of different expectations in the community is that, from the vendor
point of viewer, it looks like the community is demanding a customer-to-peer
relationship, since that is the "average" opinion of a pretty large and
diverse group. That's why I'm generally pretty careful about using the term
"the community", because for those not used to working out in the open, it's
really scary to get mixed up in public conversations.

One thing to consider about the IBM example is that IBM is a company of
about 400,000 employees, and was probably in the middle of their "we're
spending $1 billion/year on Linux" year when they instituted that policy.
They could probably stand to be a little inefficient in the name of
insinuating themselves in the community. We're not working with that sort
of cushion.

As someone who currently works from Seattle (and worked on a distributed
team in my last job), I also know that long distance collaboration (even in
the same timezone as SF) has its disadvantages from an efficiency
perspective. Most people have a strong preference for face-to-face
communication for collaboration for good reason...it's high bandwidth. Even
people who are really good at doing it take some time to figure out how to
be effective using only email and IRC; forcing people who aren't good at it
is really a productivity hit.

My recommendation is to strive to make it incredibly compelling for WMF
staff to work out in the community. That means adhering to WP:BITE and
WP:GOODFAITH in spades, and reminding each other that we're all on the same
team here. It means making sure that it actually feels like it's increasing
our productivity to do it, rather than feeling like a drag. That's not to
say the burden needs to be solely on you all, but I think "forcing"
employees to work in the community is some customer-vendor thinking at play.

Don't get me wrong: I think it's an incredibly good idea for us to figure
out how to all work together better, and clearly a big part of that is going
to be strengthening our working relationship with non-employees. It wasn't
that long ago I was a non-employee Wikipedian, and may be one again soon. I
share your goal. We have an amazingly diverse community with (very
importantly) a fantastic volunteer work ethic, and I think we should be able
to figure this out.

So, I'll start chipping in my work at the page Erik has started:
http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas

...and I encourage you to, too. I probably won't start in earnest until
after the Pending Changes launch next week, but I will get out there.

Rob
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
Rob Lanphier wrote:
> So, I'll start chipping in my work at the page Erik has started:
> http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas

You all really just don't get it, do you? Part of the problem is that the
usability wiki is viewed as a walled garden. Your "solution" is to create a
page there for others to add comments?

Meta-Wiki is the Wikimedia community's wiki. Go there and create a dialogue.

MZMcBride



_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Wed, Jun 9, 2010 at 7:08 PM, MZMcBride <z@mzmcbride.com> wrote:

> Rob Lanphier wrote:
> > So, I'll start chipping in my work at the page Erik has started:
> > http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas
>
> You all really just don't get it, do you? Part of the problem is that the
> usability wiki is viewed as a walled garden. Your "solution" is to create a
> page there for others to add comments?
>
> Meta-Wiki is the Wikimedia community's wiki. Go there and create a
> dialogue.
>
>
Um, ok:
http://meta.wikimedia.org/wiki/Product_Development_Process_Ideas
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
--- On Wed, 6/9/10, Rob Lanphier <robla@wikimedia.org> wrote:


>
> One undertone that I've witnessed everywhere is that people
> in open source
> communities that have a clear organizational "owner" is
> that there is a very
> uneven distribution of people who want a peer-to-peer
> relationship versus a
> customer-vendor relationship.  This makes it really
> difficult to work out in
> the public, because some people seem to prefer the
> trappings of a
> peer-to-peer relationship (let me in on your early
> thinking, publish your
> roadmaps, work in the fishbowl), where others prefer the
> trappings of the
> customer-vendor relationship (the customer is always right,
> the customer is
> the boss).  Some will even go so far as to want a
> customer-to-peer
> relationship, which is clearly not sustainable.  To be
> really clear here,
> most of my impressions on this topic come from my previous
> work experience
> (been doing the corporate open source thing for a while),
> and only in a
> limited way with this community, but I've seen hints that
> the
> WMF<=>community relationship has some of the same
> traits.
>
> From the vantage point of the "vendor" in this case, the
> problem is
> compounded by the cognitive bias Erik pointed to (belief
> that the group
> you're a member of is diverse, whereas other groups are
> not).  The net
> result of different expectations in the community is that,
> from the vendor
> point of viewer, it looks like the community is demanding a
> customer-to-peer
> relationship, since that is the "average" opinion of a
> pretty large and
> diverse group.  That's why I'm generally pretty
> careful about using the term
> "the community", because for those not used to working out
> in the open, it's
> really scary to get mixed up in public conversations.
>
> One thing to consider about the IBM example is that IBM is
> a company of
> about 400,000 employees, and was probably in the middle of
> their "we're
> spending $1 billion/year on Linux" year when they
> instituted that policy.
> They could probably stand to be a little inefficient in
> the name of
> insinuating themselves in the community.  We're not
> working with that sort
> of cushion.
>
> As someone who currently works from Seattle (and worked on
> a distributed
> team in my last job), I also know that long distance
> collaboration (even in
> the same timezone as SF) has its disadvantages from an
> efficiency
> perspective.  Most people have a strong preference for
> face-to-face
> communication for collaboration for good reason...it's high
> bandwidth.  Even
> people who are really good at doing it take some time to
> figure out how to
> be effective using only email and IRC; forcing people who
> aren't good at it
> is really a productivity hit.
>
> My recommendation is to strive to make it incredibly
> compelling for WMF
> staff to work out in the community.  That means
> adhering to WP:BITE and
> WP:GOODFAITH in spades, and reminding each other that we're
> all on the same
> team here.  It means making sure that it actually
> feels like it's increasing
> our productivity to do it, rather than feeling like a
> drag.  That's not to
> say the burden needs to be solely on you all, but I think
> "forcing"
> employees to work in the community is some customer-vendor
> thinking at play.
>
> Don't get me wrong: I think it's an incredibly good idea
> for us to figure
> out how to all work together better, and clearly a big part
> of that is going
> to be strengthening our working relationship with
> non-employees.  It wasn't
> that long ago I was a non-employee Wikipedian, and may be
> one again soon.  I
> share your goal.  We have an amazingly diverse
> community with (very
> importantly) a fantastic volunteer work ethic, and I think
> we should be able
> to figure this out.

I think you are conflating two very seperate issues here. There is a peer-to-peer relationship between developers (staff and volunteer) and a customer-vendor relationship between the larger non-technical consensus that is formed and developers (both staff and volunteer). Although I don't think I would describe it as "the customer is always right"; technical vetos by developers are common. The suggestion here is to eliminate the barriers between two groups of developers. There will always be some kind of barrier between the largely non-technical community and developers. There are a alot of rough edges to that customer-vendor relationship, but the volunteer developers have had alot of experience with the pitfalls there and can help staff developers navigate them.

Birgitte SB





_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Thu, Jun 10, 2010 at 8:59 AM, Birgitte SB <birgitte_sb@yahoo.com> wrote:

> --- On Wed, 6/9/10, Rob Lanphier <robla@wikimedia.org> wrote:
> > From the vantage point of the "vendor" in this case, the
> > problem is compounded by the cognitive bias Erik pointed to (belief
> > that the group you're a member of is diverse, whereas other groups are
> > not). The net result of different expectations in the community is that,
> > from the vendor point of viewer, it looks like the community is demanding
> a
> > customer-to-peer relationship, since that is the "average" opinion of a
> > pretty large and diverse group. That's why I'm generally pretty
> > careful about using the term "the community", because for those not used
> to working out
> > in the open, it's really scary to get mixed up in public conversations.
>
> I think you are conflating two very seperate issues here. There is a
> peer-to-peer relationship between developers (staff and volunteer) and a
> customer-vendor relationship between the larger non-technical consensus that
> is formed and developers (both staff and volunteer). Although I don't think
> I would describe it as "the customer is always right"; technical vetos by
> developers are common. The suggestion here is to eliminate the barriers
> between two groups of developers. There will always be some kind of barrier
> between the largely non-technical community and developers. There are a
> alot of rough edges to that customer-vendor relationship, but the volunteer
> developers have had alot of experience with the pitfalls there and can help
> staff developers navigate them.


My point is that many people conflate these two relationships, and that
furthermore, the two are inextricably bound in a world of total
transparency, since its impossible to communicate to one group without the
other overhearing. Furthermore, a "customer-vendor"-style relationship
between the larger non-technical consensus and the development community is
probably not sustainable in the world of completely transparent open source.
It still needs to be more collaborative in nature. The latter point is
fairly well understood when the development community is almost completely
decentralized (think Linux or Apache), but becomes muddier (but no less
true) when there is a central organization involved.

Rob
_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Wed, Jun 9, 2010 at 8:46 PM, Rob Lanphier <robla@wikimedia.org> wrote:
> As you know, any time you want to compel someone to do something, there's
> always the carrot and the stick.  One thing I don't like about the way
> you've phrased that is that is that you seem to be advocating the stick.  Am
> I reading that right?

I'm not clear what you're asking in practice. I don't think Wikimedia
should be penalizing people for not making enough wikitech-l posts per
week or anything, no. I do think it's reasonable for it to make
organizational decisions, like what sort of communication fora are
used for developing its software. I also think it's reasonable for it
to ask its paid employees to try to treat volunteers similarly to paid
people: ask that they try to respond when intelligent questions are
raised, maybe review patches, that kind of thing. Some people don't
do well in community-oriented jobs, and that's fine; there are an
enormous number of non-community-based development jobs available at
other organizations.

> One undertone that I've witnessed everywhere is that people in open source
> communities that have a clear organizational "owner" is that there is a very
> uneven distribution of people who want a peer-to-peer relationship versus a
> customer-vendor relationship.  This makes it really difficult to work out in
> the public, because some people seem to prefer the trappings of a
> peer-to-peer relationship (let me in on your early thinking, publish your
> roadmaps, work in the fishbowl), where others prefer the trappings of the
> customer-vendor relationship (the customer is always right, the customer is
> the boss).

Are you talking about people paid by the organization, or volunteers,
or users? As far as I've seen, the users and volunteers here
overwhelmingly prefer a more peer-to-peer approach. If someone
applies for a job at the Wikimedia Foundation, on the other hand, they
should expect to be dealing with the community on a day-to-day basis,
and shouldn't apply if they don't want to do that. Every development
job I can find at
<http://wikimediafoundation.org/wiki/Special:PrefixIndex/Job_openings>
has "You must be comfortable in a highly collaborative,
consensus-oriented environment" as a requirement.

> One thing to consider about the IBM example is that IBM is a company of
> about 400,000 employees, and was probably in the middle of their "we're
> spending $1 billion/year on Linux" year when they instituted that policy.
>  They could probably stand to be a little inefficient in the name of
> insinuating themselves in the community.  We're not working with that sort
> of cushion.

Is vendor-to-customer development more efficient than peer-to-peer in
the end? A lot of open-source projects (e.g., Firefox) have as many
features as their closed-source counterparts, on a much smaller
budget. And of course, Wikipedia managed to become awfully large on a
tiny budget, based almost exclusively on community (i.e.,
non-Wikimedia) contributions. Paid developers get less work done, but
you encourage an effective development community. Volunteers don't
only do development themselves, they also provide a pool of people to
recruit who won't need to spend time getting up to speed when hired.

> As someone who currently works from Seattle (and worked on a distributed
> team in my last job), I also know that long distance collaboration (even in
> the same timezone as SF) has its disadvantages from an efficiency
> perspective.  Most people have a strong preference for face-to-face
> communication for collaboration for good reason...it's high bandwidth.  Even
> people who are really good at doing it take some time to figure out how to
> be effective using only email and IRC; forcing people who aren't good at it
> is really a productivity hit.

On the other hand, by involving volunteers as much as possible, you
get a great deal of free labor. Empirically, this seems like a very
effective approach for organizations that don't have much money, like
Wikimedia or Mozilla. Open-source software is turned out on far lower
budgets than typical commercial software.

> My recommendation is to strive to make it incredibly compelling for WMF
> staff to work out in the community.  That means adhering to WP:BITE and
> WP:GOODFAITH in spades, and reminding each other that we're all on the same
> team here.

This is exactly what's *un*important. Many of the most
community-oriented projects are notoriously vitriolic, while
cathedral-style developers virtually always treat even completely
braindead users as though they were made of glass. Being nice is a
good thing (to a point), but it's not at all essential.

The essential thing is to give developers the ability and the will to
contribute. For them to have the ability to contribute, development
must all be public and easily followed, even if that results in
superficial efficiency hits. How can anyone contribute usefully to
the Usability Initiative (say) if they have no idea what the rationale
was behind any of the design decisions, because the decision-making
was done face-to-face or by phone? That's more efficient for the
employees, but sure as heck less efficient for the volunteers.

For volunteer developers to have the will to contribute, they need to
be rewarded, and one of the surest rewards is respect. Respect means
that they're treated according to their history of contribution. It
means that they can object to something and be listened to even if
their objection contradicts what some paid people agreed to in a
closed room. It means that an established volunteer contributor has
more say in practice than a new hiree. It means they make decisions,
they don't just give feedback.

> That's not to
> say the burden needs to be solely on you all, but I think "forcing"
> employees to work in the community is some customer-vendor thinking at play.

I don't understand what you mean by this at all. Neither an employee
or an employer plays the role of a customer in any sense here.

On Thu, Jun 10, 2010 at 12:37 AM, Rob Lanphier <robla@robla.net> wrote:
> Um, ok:
> http://meta.wikimedia.org/wiki/Product_Development_Process_Ideas

Wiki pages can be used for collecting ideas, but I don't think that's
important right now. We have ideas, we just need to discuss them to
figure out which ones are best. Mailing lists are a good way to do
that -- wikis are bad for discussion.

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Fri, Jun 11, 2010 at 7:00 AM, Aryeh Gregor
<Simetrical+wikilist@gmail.com> wrote:
>....
>
> Is vendor-to-customer development more efficient than peer-to-peer in
> the end?  A lot of open-source projects (e.g., Firefox) have as many
> features as their closed-source counterparts, on a much smaller
> budget.
> ...
>...  Empirically, this seems like a very
> effective approach for organizations that don't have much money, like
> Wikimedia or Mozilla.  Open-source software is turned out on far lower
> budgets than typical commercial software.

Mozilla is an excellent example of a non-profit organisation who also
have a similar policy of expecting their staff to do their work out in
the open, and they regularly recruit from within their community.

Read more about their governance, etc here

http://www.mozilla.org/about/governance.html

See point 8 of the Mozilla Manifesto

http://www.mozilla.org/about/manifesto

"8. Transparent community-based processes promote participation,
accountability, and trust."

You can see a list of their mailing lists here:

http://www.mozilla.org/community/forums/

.. including a vibrant list about governance ;-)

http://groups.google.com/group/mozilla.governance

They do have private lists as well, and (had?) a private bug database
for in-the-wild security issues, however these are subject to review

http://wiki.mozilla.org/GovernanceIssues#Shouldn.27t-Be-Private_Mailing_Lists

--
John Vandenberg

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On 6/9/2010 2:01 AM, Austin Hair wrote:
> On Wed, Jun 9, 2010 at 12:55 AM, Aryeh Gregor
> <Simetrical+wikilist@gmail.com> wrote:
>
>> 2) Make sure that every paid developer spends time dealing with the
>> community. This can include giving support to end users, discussing
>> things with volunteers, reviewing patches, etc. They should be doing
>> this on paid time, and they should be discussing their personal
>> opinions without consulting with anyone else (i.e., not summarizing
>> official positions). Paid developers and volunteers have to get to
>> know each other and have to be able to discuss MediaWiki together.
>>
> I like the "discussing their personal opinions without consulting with
> anyone else" bit, and you bring up a very good point.
>
> I don't think (and I don't mean to imply that anyone else does) that
> anyone's conspiring to keep the community out, or saying "leave this
> to the professionals, we know better." When you're hired onto a team,
> though, you're wary of saying anything that would cause strife or
> confusion. This isn't necessarily out of fear of retribution from
> your employer—it's simply conventional professional ethics, and it's
> usually not even a conscious thing. (It's also not limited to paid
> staff—the people we put on the Board specifically for their vocal
> opinions on things often fall into this, for understandable reasons.)
>
When it comes to the board, along with others who have oversight
responsibilities like management staff, there's an additional factor in
this. It's not generally appropriate, or good for staff morale, to
publicly go through the work of employees or contractors when you're in
such a position. There are good reasons that work evaluations and other
personnel matters are considered confidential. I don't mean to say that
staff shouldn't be discussing code, roadmaps, or rationales as widely
and openly as possible, but if for example I was qualified to review a
staff member's patch (which I'm not), I might want to think twice about
what audience gets that feedback.

--Michael Snow


_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Fri, Jun 11, 2010 at 4:49 PM, Michael Snow <wikipedia@verizon.net> wrote:
> ... if for example I was qualified to review a
> staff member's patch (which I'm not), I might want to think twice about
> what audience gets that feedback.

Ugh.

You are implicitly expecting that the patch submitter and the code
reviewer are staff members, and that the patch submitted is not very
good! ;-)

Most open source developers have been lambasted in a public code
review; they either pick up their game and look back at their previous
code and cringe, or they go work for a company that has ten layers of
bureaucracy to protect their feelings and prevent their crappy code
from making it into the wild.

The content community is constantly putting up their work for public
review, and working collaboratively with the reviewers.

The simple solution is: Don't employ bad coders! ;-)

Or said another way, require that applicants have existing open
source/content contributions under their belt (preferably on
mediawiki/wikimedia), so that you can inspect the caliber of their
work before employing them.

With Mozilla, patches are uploaded onto bugzilla for review by *the
community*. Code review comments go on the *public* bug page. etc.
Staff members talk privately among themselves, but then so do
community members. No special process for staff vs community; the
only difference should be that the priorities of the staff are
strategically set by the organisation.

--
John Vandenberg

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: Community, collaboration, and cognitive biases [ In reply to ]
On Fri, Jun 11, 2010 at 2:49 AM, Michael Snow <wikipedia@verizon.net> wrote:
> ...if for example I was qualified to review a
> staff member's patch (which I'm not), I might want to think twice about
> what audience gets that feedback.
>
> --Michael Snow
>

Why? If they're contributing a patch to MediaWiki, they should go
through the same public patch/feedback -> commit/feedback cycle
as everyone else. The only acceptable time to develop in private is
when we're looking at active security vulnerabilities, and even then
once a patch has been written the code is committed and the issue
becomes public knowledge.

Can we be a bit harsh sometimes? Sure. But we're equal
opportunity offenders here. Anyone who submits code--staff or
volunteer--is subject to the same treatment on Bugzilla and Code
Review. If your patch sucks, we're going to tell you about it, and
there's absolutely no reason to sugarcoat it.

If someone can't take public criticism, then quite frankly they
probably shouldn't be working on open source software.

-Chad

_______________________________________________
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

1 2  View All