Mailing List Archive

On Wed, Apr 4, 2012 at 10:33 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> Python has worked pretty well without high res and monotonic clocks for 20
> years. Let's not rush into a suboptimal design based on who can outlast
> everyone else in this discussion.

+1

FWIW, I'd be fine with underscore prefixes on *any* additions to the
relevant module APIs for 3.3.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Le 04/04/2012 02:33, Steven D'Aprano a écrit :
> Judging by the hundreds of emails regarding PEP 418, the disagreements
> about APIs, namings, and even what characteristics clocks should have, I
> believe that the suggestion is too divisive (and confusing!) to be
> accepted or rejected at this time.

Oh, I just "rewrote" the PEP before reading your email. Sorry for the
noise with this PEP :-) I just read again all emails related to this PEP
to complete the PEP. The PEP should now list all proposed API designs. I
hope that I did not forget anything.

I failed to propose a consistent and clear API because I (and Guido!)
wanted to fallback to the system clock. Falling back to the system clock
is a problem when you have to define the function in the documentation
or if you don't want to use the system clock (but do something else) if
no monotonic clock is available.

So I rewrote the PEP to simplify it:

* Don't fallback to system clock: time.monotonic() is always monotonic
(cannot go backward), but it is not always available. You have to write
a classic try/except ImportError which has a nice advantage: your
program will work on Python older than 3.3 ;-)

* Remove the time.perf_counter() function (it was called
time.highres() before). "highres" notion was confusing. I only wrote the
function to expose QueryPerformanceCounter (while it was already
accessible using os.clock()). The function was not well defined. Another
PEP should be written or at least the subject should be discussed after
the PEP 418 (monotonic clock).

* Rename time.steady() to time.monotonic(), again :-)

> Everyone has a different opinion, everyone believes their opinion holds
> for the majority, and it isn't practical for anyone to read the entire
> discussion.

I read most emails and I can say that:

* There is a need for a monotonic clock
* Most people prefer to handle explicitly the fallback if no monotonic
clock is available
* Most people don't want to call the new function "steady" because it
stands for something different

> I propose for Python 3.3:
>
> 1) the os module should expose lightweight wrappers around whatever
> clocks the operating system provides;

Python 3.3 has already time.clock_gettime() and time.clock_getres() with
CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES.

mach_absolute_time() and GetTickCount/GetTick64 are not available yet.

> 3) we postpone PEP 418 until there is some real-world experience with
> using the os clocks from Python and we can develop a consensus of what
> is actually needed rather than what people think we need (i.e. probably
> in 3.4);

Many applications already implement their own "monotonic" clock". Some
libraries provide also such clock for Python. On UNIX, it's always using
clock_gettime(MONOTONIC). On Windows, it's sometimes GetTickCount,
sometimes QueryPerformanceCounter. On Mac OS X, it's always
mach_absolute_time(). I didn't find a library supporting Solaris.

> 4) if the standard library has need for a "use the best clock available,
> for some definition of best, and fall back to time() if not" clock, then
> the time module should do the simplest thing that could possible work,
> flagged as a private function:

In the last version of my PEP, time.monotonic() is simply defined as "a
monotonic clock (cannot go backward)". There is no more "... best ..."
in its definition.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 2012-04-03, at 8:33 PM, Steven D'Aprano wrote:

> 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides;

+1. That should make it flexible enough to those who really need it.

> 2) the time module should NOT provide any further clocks other than the existing time() and clock() functions (but see point 4 below);
>
> 3) we postpone PEP 418 until there is some real-world experience with using the os clocks from Python and we can develop a consensus of what is actually needed rather than what people think we need (i.e. probably in 3.4);
>
> 4) if the standard library has need for a "use the best clock available, for some definition of best, and fall back to time() if not" clock, then the time module should do the simplest thing that could possible work, flagged as a private function:

+1 on overall idea too.

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Finally! We've come full circle.

+1 for monotonic as just described by Victor.

Lock it in before the paint dries.
On Apr 4, 2012 10:05 AM, "Matt Joiner" <anacrolix@gmail.com> wrote:

> Finally! We've come full circle.
>
> +1 for monotonic as just described by Victor.
>

On Wed, Apr 04, 2012 at 03:28:34AM +0200, Victor Stinner wrote:
> Le 04/04/2012 02:33, Steven D'Aprano a écrit :
> >Judging by the hundreds of emails regarding PEP 418, the disagreements
> >about APIs, namings, and even what characteristics clocks should have, I
> >believe that the suggestion is too divisive (and confusing!) to be
> >accepted or rejected at this time.
>
> Oh, I just "rewrote" the PEP before reading your email. Sorry for the
> noise with this PEP :-) I just read again all emails related to this PEP
> to complete the PEP. The PEP should now list all proposed API designs. I
> hope that I did not forget anything.

I think the PEP is a good, important PEP, and thank you for being the
PEP's champion. But in my opinion, this is too big to rush it and risk
locking in a sub-standard API for the next decade or two.


> >I propose for Python 3.3:
> >
> >1) the os module should expose lightweight wrappers around whatever
> >clocks the operating system provides;
>
> Python 3.3 has already time.clock_gettime() and time.clock_getres() with
> CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES.

Why does it already have these things when the PEP is not accepted?

(This is not a rhetorical question, perhaps there is a good reason why
these have been added independently of the PEP.)

If I remember correctly, Guido earlier mentioned that he only wanted to
see one or two (I forget which) new clocks, and I see in 3.3.0a1 there
are already at least five new clocks:

monotonic or clock_gettime(CLOCK_MONOTONIC) # Are these the same thing?
wallclock
clock_gettime(CLOCK_PROCESS_CPUTIME_ID)
clock_gettime(CLOCK_REALTIME)
clock_gettime(CLOCK_THREAD_CPUTIME_ID)

plus the old ways, time.time and time.clock. (Neither of which seems
to have a clock-id.)


> mach_absolute_time() and GetTickCount/GetTick64 are not available yet.

That will make potentially 10 different clocks in the time module.


It may be that, eventually, Python should support all these ten
different clocks. (Personally, I doubt that the average Python
programmer cares about the difference between time() and clock(), let
alone another eight more.) But there's no rush. I think we should start
by supporting OS-specific clocks in the os module, and then once we have
some best-practice idioms, we can promote some of them to the time
module.


--
Steven
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Wed, 4 Apr 2012 18:09:40 +1000
Steven D'Aprano <steve@pearwood.info> wrote:
> > Python 3.3 has already time.clock_gettime() and time.clock_getres() with
> > CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES.
>
> Why does it already have these things when the PEP is not accepted?
>
> (This is not a rhetorical question, perhaps there is a good reason why
> these have been added independently of the PEP.)

Because these are thin (low-level) wrappers around the corresponding
POSIX APIs, so there is no reason not to add them.

I know you were asking for such wrappers to be in the "os" module, but
my understanding is that time-related functions should preferably go
into the "time" module. "os" is already full of very diverse stuff, and
documentation-wise it is better if time-related functions end up in a
time-related module. Otherwise we'll end up having to cross-link
manually, which is always cumbersome (for us) and less practical (for
the reader).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

>> Why does it already have these things when the PEP is not accepted?
>> ...
>> (This is not a rhetorical question, perhaps there is a good reason why
>> these have been added independently of the PEP.)

time.clock_gettime() & friends were added by the issue #10278. The
function was added before someone asked (me) to write a PEP. The need
of a PEP came later, when time.wallclock() and time.monotonic()
functions were added.

> Because these are thin (low-level) wrappers around the corresponding
> POSIX APIs, so there is no reason not to add them.

time.clock_gettime() can be used for other purpose than a monotonic
clock. For example, CLOCK_THREAD_CPUTIME_ID is the only available
function to get the "Thread-specific CPU-time clock". It also gives
access to CLOCK_MONOTONIC_RAW which is not used by the
time.monotonic() function proposed in the PEP.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

> I failed to propose a consistent and clear API because I (and Guido!) wanted
> to fallback to the system clock. Falling back to the system clock is a
> problem when you have to define the function in the documentation or if you
> don't want to use the system clock (but do something else) if no monotonic
> clock is available.

Well, it was not only Guido and me.

Nick Coghlan wrote:
"However, I think Victor's right to point out that the *standard
library* needs to have a fallback to maintain backwards compatibility
if time.monotonic() isn't available, and it seems silly to implement
the same fallback logic in every module where we manipulate timeouts."
and
"Since duplicating that logic in every module that handles timeouts
would be silly, it makes sense to provide an obvious way to do it in
the time module."

Michael Foord wrote:
"It is this always-having-to-manually-fallback-depending-on-os that I
was hoping your new functionality would avoid. Is time.try_monotonic()
suitable for this usecase?"

The following functions / libraries fall back to the system clock if
no monotonic clock is available:
- QElapsedTimer class of the Qt library
- g_get_monotonic_time() of the glib library
- monotonic_clock library
- AbsoluteTime.now (third-party Ruby library),
"AbsoluteTime.monotonic?" tells if AbsoluteTime.now is monotonic

Extract of the glib doc: "Otherwise, we make a best effort that
probably involves returning the wall clock time (with at least
microsecond accuracy, subject to the limitations of the OS kernel)."

--

Only the python-monotonic-time fails with an OSError if no monotonic
clock is available.

System.nanoTime() of Java has few garantee: "Returns the current value
of the most precise available system timer, in nanoseconds. This
method can only be used to measure elapsed time and is not related to
any other notion of system or wall-clock time. The value returned
represents nanoseconds since some fixed but arbitrary time (perhaps in
the future, so values may be negative)." I don't even know if it is
monotonic, steady or has an high resolution.

Note: Boost.Chrono.high_resolution_clock falls back to the system
clock if no steady clock is available. (But the high-resolution clock
idea was deferred, it's something different than a monotonic or steady
clock.)

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

I am fine with the PEP as it is now (2012-04-04 15:34 GMT).

A question:

Since the only monotonic clock that can be adjusted by NTP is Linux'
CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always
give a clock that isn't adjusted by NTP. That would however mean we
wouldn't support monotonic clocks on systems that run a Linux that is
older than mid-2008. Is this generally seen as a problem?

//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 2012-04-03, at 9:28 PM, Victor Stinner wrote:

> In the last version of my PEP, time.monotonic() is simply defined as "a monotonic clock (cannot go backward)". There is no more "... best ..." in its definition.

I like the last version of the PEP ;)

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Lennart Regebro wrote:
> Since the only monotonic clock that can be adjusted by NTP is Linux'
> CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always
> give a clock that isn't adjusted by NTP.

I thought we decided that NTP adjustment isn't an issue, because
it's always gradual.

--
Greg

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

> I failed to propose a consistent and clear API because I (and Guido!) wanted
> to fallback to the system clock. Falling back to the system clock is a
> problem when you have to define the function in the documentation or if you
> don't want to use the system clock (but do something else) if no monotonic
> clock is available.

More details why it's hard to define such function and why I dropped
it from the PEP.

If someone wants to propose again such function ("monotonic or
fallback to system" clock), two issues should be solved:

- name of the function
- description of the function

At least, "monotonic" and "steady" names are not acceptable names for
such function, even if the function has an optional "strict=False" or
"fallback=True" parameter. By the way, someone complained that having
a boolean parameter requires to create a new function if you want to
call it without an argument (use a lambda function, functools.partial,
or anything else). Example:

get_time = lambda: try_monotonic(fallback=True)
t = get_time()

The description should give the least guarantees.

If the function doesn't promise anything (or only a few weak
properties), it is harder to decide which clock do you need for your
use case: time.clock(), time.time(), time.monotonic(), time.<name of
the monotonic-of-fallback function>, ...

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Apr 4, 2012 7:28 PM, "Victor Stinner" <victor.stinner@gmail.com> wrote:
>
> More details why it's hard to define such function and why I dropped
> it from the PEP.
>
> If someone wants to propose again such function ("monotonic or
> fallback to system" clock), two issues should be solved:
>
> - name of the function
> - description of the function

Maybe I missed it, but did anyone ever give a reason why the fallback
couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
system clock? (Given a suitable minimum delta.) That function appeared to
me to provide a sufficiently monotonic clock for timeout purposes, if
nothing else.

On 04Apr2012 22:23, PJ Eby <pje@telecommunity.com> wrote:
| On Apr 4, 2012 7:28 PM, "Victor Stinner" <victor.stinner@gmail.com> wrote:
| > More details why it's hard to define such function and why I dropped
| > it from the PEP.
| >
| > If someone wants to propose again such function ("monotonic or
| > fallback to system" clock), two issues should be solved:
| >
| > - name of the function
| > - description of the function
|
| Maybe I missed it, but did anyone ever give a reason why the fallback
| couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
| system clock? (Given a suitable minimum delta.) That function appeared to
| me to provide a sufficiently monotonic clock for timeout purposes, if
| nothing else.

It was pointed out (by Nick Coglan I think?) that if the system clock
stepped backwards then a timeout would be extended by at least that
long. For example, code that waited (by polling the synthetic clock)
for 1s could easily wait an hour if the system clock stepped back that
far. Probaby undesirable.

I think synthetic clocks are an extra task; they will all have side
effects of one kind of another.

A system monotonic clock, by contrast, may have access to some clock
hardware that doesn't step when the "main" system clock gets adjusted,
and can stay monotonic. A synthetic clock without such access can't
behave as nicely.

If synthetic clocks get handed out as fallback there should be some way
for the user to know, or a big glaring negative guarrentee in the docs
and on platforms without a system monotonic clock you might get a clock
with weird (but monotonic!) behaviours if you use the fallback.

Cheers,
--
Cameron Simpson <cs@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Tachyon: A gluon that's not completely dry.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Thu, Apr 5, 2012 at 1:41 PM, Cameron Simpson <cs@zip.com.au> wrote:
> It was pointed out (by Nick Coglan I think?) that if the system clock
> stepped backwards then a timeout would be extended by at least that
> long.

Guido pointed it out (it was in a reply to me, though).

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Thu, Apr 5, 2012 at 00:45, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
> Lennart Regebro wrote:
>>
>> Since the only monotonic clock that can be adjusted by NTP is Linux'
>> CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always
>> give a clock that isn't adjusted by NTP.
>
> I thought we decided that NTP adjustment isn't an issue, because
> it's always gradual.

Well, in timings it is an issue, but perhaps not a big one. :-)
In any case, which one we use will not change the API, so if it is
decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in
the future, once Linux < 2.6.26 (or whatever it was) is deemed
unsupported.

//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

>>> Since the only monotonic clock that can be adjusted by NTP is Linux'
>>> CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always
>>> give a clock that isn't adjusted by NTP.
>>
>> I thought we decided that NTP adjustment isn't an issue, because
>> it's always gradual.
>
> Well, in timings it is an issue, but perhaps not a big one. :-)
> In any case, which one we use will not change the API, so if it is
> decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in
> the future, once Linux < 2.6.26 (or whatever it was) is deemed
> unsupported.

I prefer to use CLOCK_MONOTONIC, not because it is also available for
older Linux kernels, but because it is more reliable. Even if the
underlying clock source is unstable (unstable frequency), a delta of
two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
smaller than a second. time.monotonic() unit is the second, as written
in its documentation.

Linux is the OS providing the most reliable monotonic clock, why
should you use a not reliable monotonic clock instead?

NTP doesn't step CLOCK_MONOTONIC, it only slew it.
http://www.python.org/dev/peps/pep-0418/#ntp-adjustment

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

2012/4/5 PJ Eby <pje@telecommunity.com>:
>> More details why it's hard to define such function and why I dropped
>> it from the PEP.
>>
>> If someone wants to propose again such function ("monotonic or
>> fallback to system" clock), two issues should be solved:
>>
>>  - name of the function
>>  - description of the function
>
> Maybe I missed it, but did anyone ever give a reason why the fallback
> couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the system
> clock?  (Given a suitable minimum delta.)  That function appeared to me to
> provide a sufficiently monotonic clock for timeout purposes, if nothing
> else.


Did you read the following section of the PEP?
http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs

Did I miss something? If yes, could you write a patch for the PEP please?

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Cameron Simpson wrote:
> On 04Apr2012 22:23, PJ Eby <pje@telecommunity.com> wrote:
> | On Apr 4, 2012 7:28 PM, "Victor Stinner" <victor.stinner@gmail.com> wrote:
> | > More details why it's hard to define such function and why I dropped
> | > it from the PEP.
> | >
> | > If someone wants to propose again such function ("monotonic or
> | > fallback to system" clock), two issues should be solved:
> | >
> | > - name of the function
> | > - description of the function
> |
> | Maybe I missed it, but did anyone ever give a reason why the fallback
> | couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
> | system clock? (Given a suitable minimum delta.) That function appeared to
> | me to provide a sufficiently monotonic clock for timeout purposes, if
> | nothing else.
>
> It was pointed out (by Nick Coglan I think?) that if the system clock
> stepped backwards then a timeout would be extended by at least that
> long. For example, code that waited (by polling the synthetic clock)
> for 1s could easily wait an hour if the system clock stepped back that
> far. Probaby undesirable.

Steven D'Aprano's synthetic clock is able to partially avoid that
situation -- worst case is a timeout of double what you asked for -- so
10 seconds instead of 5 (which is much better than 3600!).

~Ethan~
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Thu, 05 Apr 2012 08:32:22 -0700
Ethan Furman <ethan@stoneleaf.us> wrote:
>
> Steven D'Aprano's synthetic clock is able to partially avoid that
> situation -- worst case is a timeout of double what you asked for -- so
> 10 seconds instead of 5 (which is much better than 3600!).

The remaining issue is that the clock is not system-wide, it's
interpreter-specific.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Wed, Apr 4, 2012 at 11:41 PM, Cameron Simpson <cs@zip.com.au> wrote:

> On 04Apr2012 22:23, PJ Eby <pje@telecommunity.com> wrote:
> | On Apr 4, 2012 7:28 PM, "Victor Stinner" <victor.stinner@gmail.com>
> wrote:
> | > More details why it's hard to define such function and why I dropped
> | > it from the PEP.
> | >
> | > If someone wants to propose again such function ("monotonic or
> | > fallback to system" clock), two issues should be solved:
> | >
> | > - name of the function
> | > - description of the function
> |
> | Maybe I missed it, but did anyone ever give a reason why the fallback
> | couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
> | system clock? (Given a suitable minimum delta.) That function appeared
> to
> | me to provide a sufficiently monotonic clock for timeout purposes, if
> | nothing else.
>
> It was pointed out (by Nick Coglan I think?) that if the system clock
> stepped backwards then a timeout would be extended by at least that
> long. For example, code that waited (by polling the synthetic clock)
> for 1s could easily wait an hour if the system clock stepped back that
> far. Probaby undesirable.
>

Steven D'Aprano's algorithm doesn't do that. If the system clock steps
backwards, it still stepped forward by a specified minimum delta. The
amount of time that a timeout was extended would be a function of the
polling frequency, not the presence of absence of backward steps in the
underlying clock.

On Thu, Apr 5, 2012 at 6:34 AM, Victor Stinner <victor.stinner@gmail.com>wrote:

> 2012/4/5 PJ Eby <pje@telecommunity.com>:
> >> More details why it's hard to define such function and why I dropped
> >> it from the PEP.
> >>
> >> If someone wants to propose again such function ("monotonic or
> >> fallback to system" clock), two issues should be solved:
> >>
> >> - name of the function
> >> - description of the function
> >
> > Maybe I missed it, but did anyone ever give a reason why the fallback
> > couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
> system
> > clock? (Given a suitable minimum delta.) That function appeared to me
> to
> > provide a sufficiently monotonic clock for timeout purposes, if nothing
> > else.
>
>
> Did you read the following section of the PEP?
>
> http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs
>
> Did I miss something? If yes, could you write a patch for the PEP please?
>

What's missing is that if you're using a monotonic clock for timeouts, then
a monotonically-adjusted system clock can do that, subject to the polling
frequency -- it does not break just because the system clock is set
backwards; it simply loses time proportional to the frequency with which it
is polled.

For timeout purposes in a single process, such a clock is useful. It just
isn't suitable for benchmarks, or for interprocess coordination.

On Thu, Apr 5, 2012 at 9:48 AM, PJ Eby <pje@telecommunity.com> wrote:
>
>
> On Thu, Apr 5, 2012 at 6:34 AM, Victor Stinner <victor.stinner@gmail.com>
> wrote:
>>
>> 2012/4/5 PJ Eby <pje@telecommunity.com>:
>> >> More details why it's hard to define such function and why I dropped
>> >> it from the PEP.
>> >>
>> >> If someone wants to propose again such function ("monotonic or
>> >> fallback to system" clock), two issues should be solved:
>> >>
>> >>  - name of the function
>> >>  - description of the function
>> >
>> > Maybe I missed it, but did anyone ever give a reason why the fallback
>> > couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
>> > system
>> > clock?  (Given a suitable minimum delta.)  That function appeared to me
>> > to
>> > provide a sufficiently monotonic clock for timeout purposes, if nothing
>> > else.
>>
>>
>> Did you read the following section of the PEP?
>>
>> http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs
>>
>> Did I miss something? If yes, could you write a patch for the PEP please?
>
>
> What's missing is that if you're using a monotonic clock for timeouts, then
> a monotonically-adjusted system clock can do that, subject to the polling
> frequency -- it does not break just because the system clock is set
> backwards; it simply loses time proportional to the frequency with which it
> is polled.

Depending on the polling frequency sounds like a bad idea, since you
can't know that you're the only user of the clock. Also depending on
the use case, too short a timeout may be worse than too long a
timeout. E.g. imagine hitting a website that usually takes 2 seconds
to respond, and setting a timeout to e.g. 4 seconds to bail. If the
timeout somehow gets reduced to 1 second it will appear as if the
website died, whereas if the timeout was increased to 1 hour, nothing
bad would happen unless the website *actually* started having truly
bad response times.

> For timeout purposes in a single process, such a clock is useful.  It just
> isn't suitable for benchmarks, or for interprocess coordination.

I think it would be better if the proposed algorithm (or whatever
algorithm to "fix" timeouts) was implemented by the
application/library code using the timeout (or provided as a separate
library function), rather than by the clock, since the clock can't
know what fallback behavior the app/lib needs.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com