Mailing List Archive

Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives
Hi Pythonistas,

I've had the pleasure of speaking with Guido at PyCon and it became evident
that some of Python's included batteries are significantly lagging behind
the rapidly-evolving defacto standards of the community—specifically in
cases like urllib and urllib2, which lack important features provided by
alternatives like httplib2, requests, and my own urllib3.


Part 1:
I propose we add a snippet to the top of the documentation of specific
archaic standard modules which encourages users to investigate third-party
alternatives if they find the standard module frustrating or otherwise
lacking. These notes would target new users, including those coming from
other languages where the third-party library choices are not nearly as
amazing as they are in Python.

(For what it's worth, Guido has verbally agreed to a proposal of this
nature.)

What such a snippet might look like:

"Batteries are included with Python but sometimes they are old and
leaky—this is one of those cases. Please have a look in PyPI for more
modern alternatives provided by the Python community."

Additionally, I would like for us as a community to identify which other
standard libraries (cgi? ssl? others?) are candidates for this kind of
messaging in the official Python documentation.


Part 2:
I propose we add a new category of package identifiers such as "Topic ::
Standard Library Alternative :: {stdlib_package_name}" which authors of
libraries can tag themselves under. The documentation warning snippet will
provide a link to the appropriate PyPI query to list packages claiming to
be alternatives to the stdlib package in question.


Objections? Concerns? Improvements? What is the next step to making this
happen?

Pythonically yours,
- Andrey (on behalf of Ori Livneh, Kenneth Reitz, Brandon Rhodes, David
Wolever, and everyone else who contributed to this letter during our PyCon
sprint.)



P.S. Appendix: Here are some additional snippet alternatives that were
proposed:

> Batteries are included with Python but sometimes they are old and
leaky—this is one of those cases. Please have a look in PyPI for more
modern alternatives provided by the Python community.

> While this module has served Python programmers faithfully for many
years, there are now many powerful alternatives available as third-party
modules. To learn more about them, view the Python Package Index results
for the category "Topic :: Standard Library Alternative :: asyncore." —
[With the topic name as a hyperlink]

> This module has been identified by the community as crusty, a signal that
better alternatives exist outside the standard library. Because the Python
standard library is constained to maintain backward-compatibility, it does
not always reflect what is current or common in the Python community. If
you are not constrained to support legacy code, you may wish to browse the
Python Package Index for alternatives to this module.
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
I don't like any of the suggested wordings. I have no problem with
us recommending other modules, but most of the Python libraries are
perfectly functional (not "leaky" or some other pejorative), they just
aren't as capable as the wiz-bang new stuff that's available on PyPI.

--David
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, Mar 13, 2012 at 05:07, R. David Murray <rdmurray@bitdance.com> wrote:
> I don't like any of the suggested wordings.  I have no problem with
> us recommending other modules, but most of the Python libraries are
> perfectly functional (not "leaky" or some other pejorative), they just
> aren't as capable as the wiz-bang new stuff that's available on PyPI.
>

+1 to David's comment, and -0 on the proposal as a whole.

The suggested wordings are simply offensive to those modules & their
maintainers specifically, and to Python generally.

Personally, I think an intelligent user should realize that a
language's standard library won't provide all the latest and shiniest
gadgets. Rather, it will focus on providing stable tools that have
withstood the test of time and can serve as a basis for building more
advanced tools. That intelligent user should also be aware of PyPI
(and the main Python page makes it prominent enough), so I see no
reason explicitly pointing to it in the documentation of several
modules.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 19:23, Andrey Petrov <shazow@gmail.com> wrote:
> What such a snippet might look like:
>
> "Batteries are included with Python but sometimes they are old and
> leaky—this is one of those cases. Please have a look in PyPI for more modern
> alternatives provided by the Python community."

What does "leaky" mean here? Someone's going to see that, think it has
memory leaks, then rant on the internet about how we ship crap and
just document it as so.


> Part 2:
> I propose we add a new category of package identifiers such as "Topic ::
> Standard Library Alternative :: {stdlib_package_name}" which authors of
> libraries can tag themselves under. The documentation warning snippet will
> provide a link to the appropriate PyPI query to list packages claiming to be
> alternatives to the stdlib package in question.

Automating it to something on PyPI is the not the right answer. People
will use it incorrectly, either in that they'll add it to packages for
which it isn't accurate, and people just flat out won't use it or know
about it. It won't be accurate this way, and anything that we're
documenting needs to be vetted.

It's not often that a great alternative comes up, so I don't see the
manual burden being too great.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 8:58 PM, Brian Curtin <brian@python.org> wrote:
> On Mon, Mar 12, 2012 at 19:23, Andrey Petrov <shazow@gmail.com> wrote:
>> What such a snippet might look like:
>>
>> "Batteries are included with Python but sometimes they are old and
>> leaky—this is one of those cases. Please have a look in PyPI for more modern
>> alternatives provided by the Python community."
>
> What does "leaky" mean here? Someone's going to see that, think it has
> memory leaks, then rant on the internet about how we ship crap and
> just document it as so.

I agree Brian and David, the choice of "leaky" in the wording is poor.
It was supposed to be maintaining the "batteries" metaphor but it's
clearly ambiguous.

Perhaps something along the lines of...

"Batteries are included with Python but for stability and backwards
compatibility, some of the standard library is not always as modern as
alternatives provided by the Python community—this is one of those
cases. Please have a look at PyPI for more cutting-edge alternatives."

>> Part 2:
>> I propose we add a new category of package identifiers such as "Topic ::
>> Standard Library Alternative :: {stdlib_package_name}" which authors of
>> libraries can tag themselves under. The documentation warning snippet will
>> provide a link to the appropriate PyPI query to list packages claiming to be
>> alternatives to the stdlib package in question.
>
> Automating it to something on PyPI is the not the right answer. People
> will use it incorrectly, either in that they'll add it to packages for
> which it isn't accurate, and people just flat out won't use it or know
> about it. It won't be accurate this way, and anything that we're
> documenting needs to be vetted.
>
> It's not often that a great alternative comes up, so I don't see the
> manual burden being too great.

There are a dozen or more urllib/httplib/pycurl competitors on PyPI,
and new ones spring up all the time. I'm not sure how we would go
about objectively blessing the best "official" option at any given
moment, or how frequently we would have to do this.

With self-identifying, we could sort by some sort metric (monthly
downloads? magical score?) and create a somewhat-actionable list.

- Andrey
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On 3/12/2012 10:23 PM, Andrey Petrov wrote:
> I've had the pleasure of speaking with Guido at PyCon and it became
> evident that some of Python's included batteries are significantly
> lagging behind the rapidly-evolving defacto standards of the
> community—specifically in cases like urllib and urllib2, which lack
> important features provided by alternatives like httplib2, requests, and
> my own urllib3.
>
>
> Part 1:
> I propose we add a snippet to the top of the documentation of specific
> archaic standard modules which encourages users to investigate
> third-party alternatives if they find the standard module frustrating or
> otherwise lacking. These notes would target new users, including those
> coming from other languages where the third-party library choices are
> not nearly as amazing as they are in Python.

I would rather we figure out how to encourage authors of advancing
packages to contribute better implementations of existing features and
well-tested new features back to the stdlib module.

For instance, are you the same 'Andrey Petrov' who is'darkprokoba' on
the tracker? As near as I can tell, that user has posted on one issue
about free threading and nothing else, in particular, nothing about
web protocols. If that is you, why not?

> What such a snippet might look like:
> "Batteries are included with Python but sometimes they are old and
> leaky—this is one of those cases. Please have a look in PyPI for more
> modern alternatives provided by the Python community."

You have every right to work independently. develop alternative modules,
and promote them. But suggesting that we denigrate our work to promote
yours strikes me as inappropriate. If nothing else, it would discourage
rather than encourage more contributions from more people.

> Additionally, I would like for us as a community to identify which
> other standard libraries (cgi? ssl? others?) are candidates for this
> kind of messaging in the official Python documentation.

To the degree feasible, stdlib modules should be the best possible in
the area they cover. Then all who install Python would benefit. Do you
disagree? I would like more of the community to help make that happen.

Any messages in the stdlib doc should be about modules that do things
intentionally not covered in the stdlib.

--
Terry Jan Reedy


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 21:14, Andrey Petrov <shazow@gmail.com> wrote:
> On Mon, Mar 12, 2012 at 8:58 PM, Brian Curtin <brian@python.org> wrote:
>> On Mon, Mar 12, 2012 at 19:23, Andrey Petrov <shazow@gmail.com> wrote:
>>> What such a snippet might look like:
>>>
>>> "Batteries are included with Python but sometimes they are old and
>>> leaky—this is one of those cases. Please have a look in PyPI for more modern
>>> alternatives provided by the Python community."
>>
>> What does "leaky" mean here? Someone's going to see that, think it has
>> memory leaks, then rant on the internet about how we ship crap and
>> just document it as so.
>
> I agree Brian and David, the choice of "leaky" in the wording is poor.
> It was supposed to be maintaining the "batteries" metaphor but it's
> clearly ambiguous.
>
> Perhaps something along the lines of...
>
> "Batteries are included with Python but for stability and backwards
> compatibility, some of the standard library is not always as modern as
> alternatives provided by the Python community—this is one of those
> cases. Please have a look at PyPI for more cutting-edge alternatives."

Sorry for another color choice on the bikeshed, but I would drop the
word or references to "batteries". *We* know what "batteries included"
means, but there are undoubtedly people who won't get it. It's just
code - let's call it code.

>>> Part 2:
>>> I propose we add a new category of package identifiers such as "Topic ::
>>> Standard Library Alternative :: {stdlib_package_name}" which authors of
>>> libraries can tag themselves under. The documentation warning snippet will
>>> provide a link to the appropriate PyPI query to list packages claiming to be
>>> alternatives to the stdlib package in question.
>>
>> Automating it to something on PyPI is the not the right answer. People
>> will use it incorrectly, either in that they'll add it to packages for
>> which it isn't accurate, and people just flat out won't use it or know
>> about it. It won't be accurate this way, and anything that we're
>> documenting needs to be vetted.
>>
>> It's not often that a great alternative comes up, so I don't see the
>> manual burden being too great.
>
> There are a dozen or more urllib/httplib/pycurl competitors on PyPI,
> and new ones spring up all the time. I'm not sure how we would go
> about objectively blessing the best "official" option at any given
> moment, or how frequently we would have to do this.

The same way we choose to accept libraries into the standard library.
New ones spring up all the time - mature, proven, and widely used ones
do not. If someone thinks libfoo is ready, they suggest it. If we
haven't heard of it, the conversation ends. If we have people who know
it, maybe we have them look deeper and figure out if it's something we
can put our stamp on just like we might with the recent talk of
"experimental package" inclusion.

> With self-identifying, we could sort by some sort metric (monthly
> downloads? magical score?) and create a somewhat-actionable list.

Downloads don't mean the code is good. Voting is gamed. I really don't
think there's a good automated solution to tell us what the
high-quality replacement projects are.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 9:22 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> I would rather we figure out how to encourage authors of advancing packages
> to contribute better implementations of existing features and well-tested
> new features back to the stdlib module.

I would not. There are many excellent packages out there that should
not be made into stdlib packages simply because their authors are not
done adding new features. If you contribute something to the stdlib
and also maintain a non-stdlib version of the same code to which you
regularly add features, your code is not ready for inclusion into the
stdlib. Not only should you be willing to wait 18 months (until the
next feature release) before your features are released, but you
should also accept that only the latest version of Python will see
those features.

This obviously makes it very unattractive for many authors to ever
contribute to the stdlib. That's fine. There's a healthy ecosystem of
3rd party modules. For some areas (web stuff especially) there's just
no way that the stdlib can keep up. Yes, the stdlib offerings work.
But no, they are not very convenient and may not support popular
idioms very well. For these types of modules I think it is a good idea
to place some sort of pointer in the stdlib docs to an external page
(maybe a wiki page) that collects a currently popular set of
alternatives, or perhaps a few pointers and wiki pages. We should
still be conservative with this, and we should word it to avoid
implying that the stdlib code is buggy -- it just isn't as spiffy or
featureful as the 3rd party options.

(Agreed that the "leaky" wording was unfortunate. I may have
inadvertently suggested this, taking the analogy with "batteries
included". But I didn't mean it to be literally included into the
stdlib.)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 9:23 PM, Brian Curtin <brian@python.org> wrote:
> Downloads don't mean the code is good. Voting is gamed. I really don't
> think there's a good automated solution to tell us what the
> high-quality replacement projects are.

Sure, these are imperfect metrics. But not having any metrics at all
is flawed too. Despite the huge flamewar we had 1-2 years ago about
PyPI comments, I think we should follow the lead of the many app
stores that pop up on the web -- users will recognize the pattern and
will tune their skepticism sensors as needed.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, Mar 13, 2012 at 06:43, Guido van Rossum <guido@python.org> wrote:
> On Mon, Mar 12, 2012 at 9:23 PM, Brian Curtin <brian@python.org> wrote:
>> Downloads don't mean the code is good. Voting is gamed. I really don't
>> think there's a good automated solution to tell us what the
>> high-quality replacement projects are.
>
> Sure, these are imperfect metrics. But not having any metrics at all
> is flawed too. Despite the huge flamewar we had 1-2 years ago about
> PyPI comments, I think we should follow the lead of the many app
> stores that pop up on the web -- users will recognize the pattern and
> will tune their skepticism sensors as needed.
>

An additional bonus of such a system is that we won't have to maintain
a separate Wiki page with "popular" choices. Pointing to the PyPI
"rating" page, which can presumably be filtered by tags (i.e. web,
scientific, XML, etc.) should be sufficient, given that such a rating
page exists.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
Definitely think some library vetting needs to occur. Superior alternatives
do exist and are difficult to find and choose from. Stuff like LXML,
Requests, Tornado are clear winners.

The more of this done externally (ie PyPI the better). I still think a set
of requirements for "official approval" would be good. This could outline
things like requiring that certain stable Python versions are supported,
interface stability, demonstrated user base, documentation etc.
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 07:23:11PM -0700, Andrey Petrov wrote:
> I've had the pleasure of speaking with Guido at PyCon and it became evident
> that some of Python's included batteries are significantly lagging behind the
> rapidly-evolving defacto standards of the community specifically in cases like
> urllib and urllib2, which lack important features provided by alternatives like
> httplib2, requests, and my own urllib3.

Well, I think I have address this because I am the maintainer of those
modules in standard lib.

First things first, it looks to me that trashing something gives good
motivation to you (and others working on related modules). I don't
have a problem with that.

But on the other hand, if you think things can be improved in stdlib,
you are welcome to contribute. Just remember that new features,
refactoring with backwards compatibility, 'cool api' for new features
should go in 3.3+ onwards. Bug fixes, confusions on what's RFC
supported vs what's defacto standards, fine line between bugs and
features, those can be considered for 2.7.

I am personally in favor of constantly improving the standard library
modules along with mention of any good libraries which can be useful
for the purposes of the user.

We already have lots of such references in standard library
documentation. If there is a well maintained package, as long as the
external package is active and maintained, we can have it as link in
the docs. Sometimes those external packages become inactive too, in
those cases, those should pruned. Its' all about maintaining libraries
and docs and being helpful.

--
Senthil

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
Dear authors of Python's standard library: Please accept my deepest
apologies. We didn't mean for the messaging to come off as
unappreciative for you work, and I am very sorry for that!

Without the aforementioned urllib/httplib/etc I would have never made
it as far as to build my own libraries which have suited my needs
better than the foundations that originally allowed me to get off the
ground. I am very grateful for all of those responsible for building
the Python standard library and I appreciate your continued efforts.

Some specific replies:

@Senthil: I originally asked Guido for guidance on improving the
standard library and perhaps including some of my favourite projects,
but he pointed out that in a couple of years we might end up again in
the same position as before but with one extra library people will
complain about for being obsoleted yet remains impossible to
deprecate.

I agreed with Guido that embracing and even encouraging users to use the
rapidly-evolving community-built packages alongside the tried-and-true
standard library is the best move here.

@Terry: I don't know who 'darkprokoba' is. Unfortunately 'Andrey
Petrov' is a very common name. I go under the handle 'shazow' but I
haven't participated in core Python discussions until today.

I did not suggest for Python to endorse a specific module, even if it
is my own. In fact, I generally oppose doing this as I feel that when
Django was announced as the blessed 'winner' of the Python MVC
frameworks was a harmful event to all other competing frameworks at
the time.

My suggestion is to inform the users when there are other potentially
better suited alternatives available from the community PyPI and to
educate them how to find these alternatives.

- Andrey
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 10:48:04PM -0700, Andrey Petrov wrote:
> @Senthil: I originally asked Guido for guidance on improving the
> standard library and perhaps including some of my favourite projects,
> but he pointed out that in a couple of years we might end up again in
> the same position as before but with one extra library people will
> complain about for being obsoleted yet remains impossible to
> deprecate.
>
> I agreed with Guido that embracing and even encouraging users to use the
> rapidly-evolving community-built packages alongside the tried-and-true
> standard library is the best move here.

I agree with that too.

I think, any improvements which can made by external libraries, if
they can be considered as can be good improvement in the stdlib, It's
a good idea to push them in.

I looked at the requests, I think that apis for various HTTP verbs is
something interesting, I think web applications authors see some
advantage there explicit GET / POST and sending specific options to
those. That facility, if it can be brought into urllib.request (3.3
onwards) it could be nice. Also you could have noticed an addition of
method= parameter in 3.3

I have not had a chance to look at urllib3. Should do. I have been
following httplib2 and use it.

Thanks,
Senthil


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tuesday, March 13, 2012 at 9:31 AM, Paul Moore wrote:
> On 13 March 2012 03:48, C. Titus Brown <ctb@msu.edu (mailto:ctb@msu.edu)> wrote:
> > I feel like there's a middle ground where stable, long-term go-to modules could
> > be mentioned, though. I don't spend a lot of time browsing PyPI, but I suspect
> > almost everyone spends a certain amount of time in the Python docs (which is a
> > testimony to their quality IMO). So I'm in favor of conservative link-outs
> > but without any deprecating language.
> >
>
>
> I applaud the idea of promoting the many excellent packages available.
> It can be very hard to separate the good from the indifferent (or even
> bad) when browsing PyPI. I've found some very good packages recently
> which I'd never have known about without some random comment on a
> mailing list.
>
> However, I'm not keen on having the stdlib documentation suggest that
> I should be using something else. No code should ever be documenting
> "don't use me, there are better alternatives" unless it is deprecated
> or obsolete.
>
> On the other hand, I would love to see a community-maintained document
> that described packages that are acknowledged as "best of breed". That
> applies whether or not those packages replace something in the stdlib.
> Things like pywin32, lxml, and requests would be examples in my
> experience. There's no reason this *has* to be in the core
> documentation - it may be relevant that nothing has sprung up
> independently yet...
>
>

http://python-guide.org ?
>
> Maybe a separate item in the Python documentation, "External Modules",
> could be created and maintained by the community? By being in the
> documentation, it has a level of "official recommendation" status, and
> by being a top-level document it's visible (more so than, for example,
> a HOWTO document would be). Because it's in the released
> documentation, it is relatively stable, which implies that external
> modules would need to have a genuine track record to get in there, but
> because it's community maintained it should reflect a wider consensus
> than just the core developers' views.
>
> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org (mailto:Python-Dev@python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
>
>
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Mon, Mar 12, 2012 at 10:10 PM, Senthil Kumaran <senthil@uthcode.com> wrote:
> On Mon, Mar 12, 2012 at 07:23:11PM -0700, Andrey Petrov wrote:
>> I've had the pleasure of speaking with Guido at PyCon and it became evident
>> that some of Python's included batteries are significantly lagging behind the
>> rapidly-evolving defacto standards of the community specifically in cases like
>> urllib and urllib2, which lack important features provided by alternatives like
>> httplib2, requests, and my own urllib3.
>
> Well, I think I have address this because I am the maintainer of those
> modules in standard lib.
>
> First things first, it looks to me that trashing something gives good
> motivation to you (and others working on related modules). I don't
> have a problem with that.
>
> But on the other hand, if you think things can be improved in stdlib,
> you are welcome to contribute. Just remember that new features,
> refactoring with backwards compatibility, 'cool api' for new features
> should go in 3.3+ onwards. Bug fixes, confusions on what's RFC
> supported vs what's defacto standards, fine line between bugs and
> features, those can be considered for 2.7.
>
> I am personally in favor of constantly improving the standard library
> modules along with mention of any good libraries which can be useful
> for the purposes of the user.
>
> We already have lots of such references in standard library
> documentation. If there is a well maintained package, as long as the
> external package is active and maintained, we can have it as link in
> the docs. Sometimes those external packages become inactive too, in
> those cases, those should pruned. Its' all about maintaining libraries
> and docs and being helpful.

Well said. Improving existing stdlib modules is always welcome of
course! (And the bar is much lower than for adding new modules.)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On 13 March 2012 13:34, Donald Stufft <donald.stufft@gmail.com> wrote:

> http://python-guide.org ?

Hmm, yes maybe. I had seen this before (it's where I found out about
requests, IIRC).

As it says, it "is mostly a skeleton at the moment". With some
fleshing out, then it's probably a good start. I have some problems
with its recommendations (notably "If you’re starting work on a new
Python module, I recommend you write it for Python 2.5 or 2.6, and add
support for Python3 in a later iteration." which is probably not
appropriate for something that would be officially sanctioned by the
core developers). Also, I don't think we want something advertised as
"opinionated".

And it covers a much wider area than the original suggestion. I'd
envisaged something more like a simple list of "obvious" modules:

"""
Requests - URL

Requests is a module designed to make getting resources from the web
as easy as possible. It is a simpler and more powerful alternative to
the stdlib urllib and urllib2 modules. ...

Some code samples here giving basic usage.
"""

My ideal would be something I could scan in a few spare moments, and
pick up pointers to particular modules that I'd find useful.
Basically, take the "Scenario Guide" section of python-guide, flesh it
out, and turn it into a flat list. (I don't like the "Scenario"
approach as it tends to force people into a particular view of the
world, but maybe that's just me, my applications tend to be more
eclectic than any of the "normal" categories).

Paul.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On 3/13/2012 6:31 AM, Paul Moore wrote:
> It can be very hard to separate the good from the indifferent (or even
> bad) when browsing PyPI. I've found some very good packages recently
> which I'd never have known about without some random comment on a
> mailing list.

+1

> However, I'm not keen on having the stdlib documentation suggest that
> I should be using something else. No code should ever be documenting
> "don't use me, there are better alternatives" unless it is deprecated
> or obsolete.

+0

> On the other hand, I would love to see a community-maintained document
> that described packages that are acknowledged as "best of breed". That
> applies whether or not those packages replace something in the stdlib.
> Things like pywin32, lxml, and requests would be examples in my
> experience.

+1

> There's no reason this*has* to be in the core
> documentation - it may be relevant that nothing has sprung up
> independently yet...

Hmm.

> Maybe a separate item in the Python documentation, "External Modules",
> could be created and maintained by the community? By being in the
> documentation, it has a level of "official recommendation" status, and
> by being a top-level document it's visible (more so than, for example,
> a HOWTO document would be). Because it's in the released
> documentation, it is relatively stable, which implies that external
> modules would need to have a genuine track record to get in there, but
> because it's community maintained it should reflect a wider consensus
> than just the core developers' views.

+1 This is the best proposal I've seen for including references to
external modules. It gets it in the core documentation, hopefully with
enough keywords that search would typically find external modules that
are supersets of stdlib modules in the same result set that the stdlib
module would be found. Yet it doesn't intrude on the documentation for
the stdlib module. And beyond a 1-2 paragraph description, would not be
fully documented, except by referencing the external module's documentation.
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
I think the cheesehop trove classifiers would be the ideal way to agnostically link to a page of packages related to the standard package in question. No need for sort order.

The beauty of this solution is that packages that aren't maintained won't add the appropriate classifier to their package, and therefore not show up in the list.

--
Kenneth Reitz


On Monday, March 12, 2012 at 9:23 PM, Brian Curtin wrote:

> On Mon, Mar 12, 2012 at 21:14, Andrey Petrov <shazow@gmail.com (mailto:shazow@gmail.com)> wrote:
> > On Mon, Mar 12, 2012 at 8:58 PM, Brian Curtin <brian@python.org (mailto:brian@python.org)> wrote:
> > > On Mon, Mar 12, 2012 at 19:23, Andrey Petrov <shazow@gmail.com (mailto:shazow@gmail.com)> wrote:
> > > > What such a snippet might look like:
> > > >
> > > > "Batteries are included with Python but sometimes they are old and
> > > > leaky—this is one of those cases. Please have a look in PyPI for more modern
> > > > alternatives provided by the Python community."
> > > >
> > >
> > >
> > > What does "leaky" mean here? Someone's going to see that, think it has
> > > memory leaks, then rant on the internet about how we ship crap and
> > > just document it as so.
> > >
> >
> >
> > I agree Brian and David, the choice of "leaky" in the wording is poor.
> > It was supposed to be maintaining the "batteries" metaphor but it's
> > clearly ambiguous.
> >
> > Perhaps something along the lines of...
> >
> > "Batteries are included with Python but for stability and backwards
> > compatibility, some of the standard library is not always as modern as
> > alternatives provided by the Python community—this is one of those
> > cases. Please have a look at PyPI for more cutting-edge alternatives."
> >
>
>
> Sorry for another color choice on the bikeshed, but I would drop the
> word or references to "batteries". *We* know what "batteries included"
> means, but there are undoubtedly people who won't get it. It's just
> code - let's call it code.
>
> > > > Part 2:
> > > > I propose we add a new category of package identifiers such as "Topic ::
> > > > Standard Library Alternative :: {stdlib_package_name}" which authors of
> > > > libraries can tag themselves under. The documentation warning snippet will
> > > > provide a link to the appropriate PyPI query to list packages claiming to be
> > > > alternatives to the stdlib package in question.
> > > >
> > >
> > >
> > > Automating it to something on PyPI is the not the right answer. People
> > > will use it incorrectly, either in that they'll add it to packages for
> > > which it isn't accurate, and people just flat out won't use it or know
> > > about it. It won't be accurate this way, and anything that we're
> > > documenting needs to be vetted.
> > >
> > > It's not often that a great alternative comes up, so I don't see the
> > > manual burden being too great.
> > >
> >
> >
> > There are a dozen or more urllib/httplib/pycurl competitors on PyPI,
> > and new ones spring up all the time. I'm not sure how we would go
> > about objectively blessing the best "official" option at any given
> > moment, or how frequently we would have to do this.
> >
>
>
> The same way we choose to accept libraries into the standard library.
> New ones spring up all the time - mature, proven, and widely used ones
> do not. If someone thinks libfoo is ready, they suggest it. If we
> haven't heard of it, the conversation ends. If we have people who know
> it, maybe we have them look deeper and figure out if it's something we
> can put our stamp on just like we might with the recent talk of
> "experimental package" inclusion.
>
> > With self-identifying, we could sort by some sort metric (monthly
> > downloads? magical score?) and create a somewhat-actionable list.
> >
>
>
> Downloads don't mean the code is good. Voting is gamed. I really don't
> think there's a good automated solution to tell us what the
> high-quality replacement projects are.
>
>
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, Mar 13, 2012 at 14:13, Kenneth Reitz <me@kennethreitz.com> wrote:
> I think the cheesehop trove classifiers would be the ideal way to
> agnostically link to a page of packages related to the standard package in
> question. No need for sort order.

Randomize the order for all I care. We still need to ensure we're
suggesting quality projects. It doesn't make sense for us to suggest
alternatives that we wouldn't want to use ourselves by just polling
some list that anyone can get on.

This is documentation that receives hundreds of thousands of views a
month*. We need to be picky about what goes in it.

> The beauty of this solution is that packages that aren't maintained won't
> add the appropriate classifier to their package, and therefore not show up
> in the list.

Just because it's maintained doesn't mean it's not garbage. I think we
really need to start every project off with a 0 and make them prove
that they're a 10. Just being active means nothing.



* http://www.python.org/webstats/usage_201202.html#TOPURLS - I don't
know what page "Documentation" means since it doesn't have a specific
link, but whatever page that is got hit 960K times in February.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On 3/13/2012 12:40 AM, Guido van Rossum wrote:
> On Mon, Mar 12, 2012 at 9:22 PM, Terry Reedy<tjreedy@udel.edu> wrote:
>> I would rather we figure out how to encourage authors of advancing packages
>> to contribute better implementations of existing features and well-tested
>> new features back to the stdlib module.
>
> I would not.

I think you misunderstood me and are talking about something other than
what I meant. There are about 3250 open issues (this slowly but steadily
grows). Of them, 1450 are behavior (bug) issues. We need more people,
especially people with specialized expertise, writing and reviewing
patches. As you said in response to Senthil

> Improving existing stdlib modules is always welcome

Exactly. So I would like to figure out how to encourage more such
improvements.

> There are many excellent packages out there that should
> not be made into stdlib packages simply because their authors are not
> done adding new features.

Or because the package is outside the reasonable scope of the stdlib, or
requires a different type of expertise than most core development, or
for other reasons.

Authors of separately maintained packages are, from our viewpoint, as
eligible to help with tracker issues as anyone else, even while they
continue work on their external package. Some of them are more likely
than most contributors to have the knowledge needed for some particular
issues.

--
Terry Jan Reedy

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, Mar 13, 2012 at 12:38, Brian Curtin <brian@python.org> wrote:

> On Tue, Mar 13, 2012 at 14:13, Kenneth Reitz <me@kennethreitz.com> wrote:
> > I think the cheesehop trove classifiers would be the ideal way to
> > agnostically link to a page of packages related to the standard package
> in
> > question. No need for sort order.
>
> Randomize the order for all I care. We still need to ensure we're
> suggesting quality projects. It doesn't make sense for us to suggest
> alternatives that we wouldn't want to use ourselves by just polling
> some list that anyone can get on.
>
> This is documentation that receives hundreds of thousands of views a
> month*. We need to be picky about what goes in it.
>
> > The beauty of this solution is that packages that aren't maintained won't
> > add the appropriate classifier to their package, and therefore not show
> up
> > in the list.
>
> Just because it's maintained doesn't mean it's not garbage. I think we
> really need to start every project off with a 0 and make them prove
> that they're a 10. Just being active means nothing.
>
>
>
> * http://www.python.org/webstats/usage_201202.html#TOPURLS - I don't
> know what page "Documentation" means since it doesn't have a specific
> link, but whatever page that is got hit 960K times in February.
>

GroupURL /doc/* Documentation

So it's anything that's in www.python.org/doc/. I don't believe it counts
doc.python.org and docs.python.org.

--
Thomas Wouters <thomas@python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, Mar 13, 2012 at 12:49 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> Authors of separately maintained packages are, from our viewpoint, as
> eligible to help with tracker issues as anyone else, even while they
> continue work on their external package. Some of them are more likely than
> most contributors to have the knowledge needed for some particular issues.

This is a good idea. I was chatting w. Senthil this morning about
adding improvements to urllib/request.py based upon ideas from
urllib3, requests, httplib2 (?), and we came to the conclusion that it
might be a good idea to let those packages' authors review the
proposed stdlib improvements.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, 13 Mar 2012 14:16:40 -0700
Guido van Rossum <guido@python.org> wrote:

> On Tue, Mar 13, 2012 at 12:49 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> > Authors of separately maintained packages are, from our viewpoint, as
> > eligible to help with tracker issues as anyone else, even while they
> > continue work on their external package. Some of them are more likely than
> > most contributors to have the knowledge needed for some particular issues.
>
> This is a good idea. I was chatting w. Senthil this morning about
> adding improvements to urllib/request.py based upon ideas from
> urllib3, requests, httplib2 (?), and we came to the conclusion that it
> might be a good idea to let those packages' authors review the
> proposed stdlib improvements.

We don't have any provisions against reviewal by third-party
developers already. I think the main problem (for us, of course) is that
these people generally aren't interested enough to really dive in
stdlib patches and proposals.

For example, for the ssl module, I have sometimes tried to involve
authors of third-party packages such as pyOpenSSL (or, IIRC, M2Crypto),
but I got very little or no reviewing.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Docs of weak stdlib modules should encourage exploration of 3rd-party alternatives [ In reply to ]
On Tue, Mar 13, 2012 at 2:21 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
> On Tue, 13 Mar 2012 14:16:40 -0700
> Guido van Rossum <guido@python.org> wrote:
>
>> On Tue, Mar 13, 2012 at 12:49 PM, Terry Reedy <tjreedy@udel.edu> wrote:
>> > Authors of separately maintained packages are, from our viewpoint, as
>> > eligible to help with tracker issues as anyone else, even while they
>> > continue work on their external package. Some of them are more likely than
>> > most contributors to have the knowledge needed for some particular issues.
>>
>> This is a good idea. I was chatting w. Senthil this morning about
>> adding improvements to urllib/request.py based upon ideas from
>> urllib3, requests, httplib2 (?), and we came to the conclusion that it
>> might be a good idea to let those packages' authors review the
>> proposed stdlib improvements.
>
> We don't have any provisions against reviewal by third-party
> developers already. I think the main problem (for us, of course) is that
> these people generally aren't interested enough to really dive in
> stdlib patches and proposals.
>
> For example, for the ssl module, I have sometimes tried to involve
> authors of third-party packages such as pyOpenSSL (or, IIRC, M2Crypto),
> but I got very little or no reviewing.

IIRC M2Crypto is currently unmaintained, so that doesn't surprise me.
(In general it seems most crypto wrappers seem unmaintained -- it must
be a thankless job.)

Still, AFAICT both requests and urllib3 are very actively maintained
by people who know what they are doing, and it would be nice if we
could build bridges instead of competition. So let's at least try.
(But I'm not asking you, Antoine, to try and approach them personally.
:-)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

1 2  View All