Mailing List Archive

On Thu, 5 Apr 2012 09:56:19 -0700
Guido van Rossum <guido@python.org> wrote:
>
> > For timeout purposes in a single process, such a clock is useful.  It just
> > isn't suitable for benchmarks, or for interprocess coordination.
>
> I think it would be better if the proposed algorithm (or whatever
> algorithm to "fix" timeouts) was implemented by the
> application/library code using the timeout (or provided as a separate
> library function), rather than by the clock, since the clock can't
> know what fallback behavior the app/lib needs.

Agreed with providing it as a separate library function.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

>> > For timeout purposes in a single process, such a clock is useful.  It just
>> > isn't suitable for benchmarks, or for interprocess coordination.
>>
>> I think it would be better if the proposed algorithm (or whatever
>> algorithm to "fix" timeouts) was implemented by the
>> application/library code using the timeout (or provided as a separate
>> library function), rather than by the clock, since the clock can't
>> know what fallback behavior the app/lib needs.
>
> Agreed with providing it as a separate library function.

I changed time.monotonic() to not fallback to the system clock exactly
for this reason: Python cannot guess what the developer expects, or
how the developer will use the clock.

Instead of implementing your own clock in your application, it's maybe
easier to patch your OS? I suppose that you are running on GNU/Hurd,
because I didn't find yet other OS not providing a monotonic clock :-)

If you are using an OS that doesn't provide a monotonic clock, do you
really need to implement your own in your application?

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Victor Stinner wrote:
> I changed time.monotonic() to not fallback to the system clock exactly
> for this reason: Python cannot guess what the developer expects, or
> how the developer will use the clock.

Which is exactly why I like Cameron Simpson's approach to selecting a
clock -- let the developer/user decide what kind of clock they need, and
ask for one that matches their criteria.

~Ethan~
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Thu, Apr 5, 2012 at 12:56 PM, Guido van Rossum <guido@python.org> wrote:

> Depending on the polling frequency sounds like a bad idea, since you
> can't know that you're the only user of the clock. Also depending on
> the use case, too short a timeout may be worse than too long a
> timeout.


Given a small enough delta, the timeout won't be too short. (Steven's
original code sample I believed used either 0 or 1 as a delta, but it could
be as small a fraction as will add correctly in the datatype used.) And
the worst case polling frequency is the length of the timeout, meaning you
can't end up with more than double your intended timeout.

In the opposite scenario, where the time is polled in a tight loop, then as
long as Python doesn't sample the raw clock so often that the summed
fractional deltas exceeds the real clock speed, the timeout won't be
shortened by any appreciable amount. In fact, this can be guaranteed by
measuring time as a (raw, increment) tuple, where the increment can be an
arbitrarily-large integer. Each new time value is greater than the one
before, yet the real component remains untouched. With this approach, the
timeout can only be delayed for however long the system clock *stops*, and
the timeout can only be shortened by the system clock skipping ahead.

Okay, having thought that out, I now agree that there are too many fine
points to make this cover enough of the use cases without needing
parameters.

Or more to the point, "If the implementation is hard to explain, it's a bad
idea." ;-)

Folks:

Good job, Victor Stinner on baking the accumulated knowledge of this
thread into PEP 418. Even though I'm very interested in the topic, I
haven't been able to digest the whole thread(s) on the list and
understand what the current collective understanding is. The detailed
PEP document helps a lot.

I think there are still some mistakes, either in our collective
understanding as reflected by the PEP, or in my own head.

For starters, I still don't understand the first, most basic thing:
what do people mean when they say "monotonic clock"? I don't
understand the current text of PEP 418 with regard to the definition
of that word.

Allow me to resort to an analogy. There is an infinitely long,
perfectly straight and flat racetrack. There is a flag that gets
dragged along it at a constant rate, with the label "REAL TIME" on the
flag. There are some runners, each with a different label on their
chest:

Runner A: a helicopter hovers over Runner A. Occasionally it picks him
up and plops him down right next to the flag. Also, he wears a headset
and listens to instructions from his coach to run a little faster or
slower, as necessary, to remain abreast of the flag.

Runner B: a helicopter hovers over Runner B. If he is behind the flag,
it will pick him up and plop him down right next to the flag. However,
if he is ahead of the flag it will not pick him up.

Runner C: no helicopter ever picks up Runner C, but he does wear a
headset and listens to instructions from his coach to run a little
faster or a little slower. His coach tells him to run a little faster
if he is behind the flag or run a little slower if he is in front of
the flag, with the goal of eventually having him right next to the
flag.

Runner D: like Runner C, he never gets picked up, but he listens to
instructions to run a little faster or a little slower. However,
instead of telling him to run faster in order to catch up to the flag,
or to run slower in order to "catch down" to the flag, his coach
instead tells him to run a little faster if he is moving slower than
the flag is moving, and to run a little slower if he is moving faster
than the flag is moving. Note that this is very different from Runner
C, in that it is not intended to cause him to eventually be right next
to the flag, and indeed if it is done right it guarantees that he will
*never* be right next to the flag, although he will be moving just as
fast as the flag is moving.

Runner E: no helicopter, no headset. He just proceeds at his own pace,
blissfully unaware of the exhortations of others.

Now: which ones of these five runners do you call "monotonic"? Which
ones do you call "steady"?

Regards,

Zooko
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 05Apr2012 09:56, Guido van Rossum <guido@python.org> wrote:
| On Thu, Apr 5, 2012 at 9:48 AM, PJ Eby <pje@telecommunity.com> wrote:
| > What's missing is that if you're using a monotonic clock for timeouts, then
| > a monotonically-adjusted system clock can do that, subject to the polling
| > frequency -- it does not break just because the system clock is set
| > backwards; it simply loses time proportional to the frequency with which it
| > is polled.
|
| Depending on the polling frequency sounds like a bad idea, since you
| can't know that you're the only user of the clock.

You can if you're handed a shiny new "clock" object in some way, with a
not-a-singleton guarrentee. Of course, such a clock is immediately
_less_ reliable to synchornisation with other clock users:-)

| Also depending on
| the use case, too short a timeout may be worse than too long a
| timeout. [...]
|
| > For timeout purposes in a single process, such a clock is useful.  It just
| > isn't suitable for benchmarks, or for interprocess coordination.
|
| I think it would be better if the proposed algorithm (or whatever
| algorithm to "fix" timeouts) was implemented by the
| application/library code using the timeout (or provided as a separate
| library function), rather than by the clock, since the clock can't
| know what fallback behavior the app/lib needs.

Absolutely. I for one would be happy with a clocktools module or
something offering a bunch of synthetic clocks. Especially if they were
compatible in API with whatever clock objects the core time module
clocks used, so that a user _could_ add them into the pick-a-clock
decision easily.

Cheers,
--
Cameron Simpson <cs@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I'd be careful who you call smart or not smart. Smart isn't knowing how to
save six bytes. Smart is knowing WHEN. - Peter Cherna, Amiga O.S. Development
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 05Apr2012 13:39, Zooko Wilcox-O'Hearn <zooko@zooko.com> wrote:
| Good job, Victor Stinner on baking the accumulated knowledge of this
| thread into PEP 418. Even though I'm very interested in the topic, I
| haven't been able to digest the whole thread(s) on the list and
| understand what the current collective understanding is.

There isn't a collective understanding :-) That's why all the noise!

| The detailed
| PEP document helps a lot.

Yes indeed, though like all of us I think it could (always) use more
detail on my pet concerns.

| I think there are still some mistakes, either in our collective
| understanding as reflected by the PEP, or in my own head.
|
| For starters, I still don't understand the first, most basic thing:
| what do people mean when they say "monotonic clock"? I don't
| understand the current text of PEP 418 with regard to the definition
| of that word.

A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
polls of the clock. On its own it says nothing about steadiness or
correlation with real world time.

_Quality of implementation_ says that the montonic() call should try to
return a close that is monotonic and _also_ steady and preferably
precise. How these things are balancer is a matter of policy.

| Allow me to resort to an analogy. There is an infinitely long,
| perfectly straight and flat racetrack. There is a flag that gets
| dragged along it at a constant rate, with the label "REAL TIME" on the
| flag. There are some runners, each with a different label on their
| chest:
|
| Runner A: a helicopter hovers over Runner A. Occasionally it picks him
| up and plops him down right next to the flag. Also, he wears a headset
| and listens to instructions from his coach to run a little faster or
| slower, as necessary, to remain abreast of the flag.

If he always runs forwards, it is montonic. Not very steady when the
helicopter comes to play.

| Runner B: a helicopter hovers over Runner B. If he is behind the flag,
| it will pick him up and plop him down right next to the flag. However,
| if he is ahead of the flag it will not pick him up.

Seems like runner A without instruction. Monotonic. Not very steady.

| Runner C: no helicopter ever picks up Runner C, but he does wear a
| headset and listens to instructions from his coach to run a little
| faster or a little slower. His coach tells him to run a little faster
| if he is behind the flag or run a little slower if he is in front of
| the flag, with the goal of eventually having him right next to the
| flag.

If he always runs forward, monotonic. And steady.

| Runner D: like Runner C, he never gets picked up, but he listens to
| instructions to run a little faster or a little slower. However,
| instead of telling him to run faster in order to catch up to the flag,
| or to run slower in order to "catch down" to the flag, his coach
| instead tells him to run a little faster if he is moving slower than
| the flag is moving, and to run a little slower if he is moving faster
| than the flag is moving. Note that this is very different from Runner
| C, in that it is not intended to cause him to eventually be right next
| to the flag, and indeed if it is done right it guarantees that he will
| *never* be right next to the flag, although he will be moving just as
| fast as the flag is moving.
|
| Runner E: no helicopter, no headset. He just proceeds at his own pace,
| blissfully unaware of the exhortations of others.
|
| Now: which ones of these five runners do you call "monotonic"? Which
| ones do you call "steady"?

If they all run forwards, they're all monotonic.

If their coach or helicopter can move then _backwards_ they're not
monotonic. If the helicopter can move them an arbitrary (but matching
the game plan) distance, they're not steady. Otherwise they are steady,
if the runner's speed is always sufficiently close to the flag speed
(this threshold and the criteria for measuring it as subject to debate,
forming policy).

And "high resolution" has its own flavours, though generally less
contentious.

Cheers,
--
Cameron Simpson <cs@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

If you don't shoot the fish in your barrel, your barrel will soon be
full of fish. - Tim Mefford
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Apr 5, 2012 11:01 AM, "Guido van Rossum" <guido@python.org> wrote:
> I think it would be better if the proposed algorithm (or whatever
> algorithm to "fix" timeouts) was implemented by the
> application/library code using the timeout (or provided as a separate
> library function), rather than by the clock, since the clock can't
> know what fallback behavior the app/lib needs.

+1

-eric

Cameron Simpson wrote:

> A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
> polls of the clock. On its own it says nothing about steadiness or
> correlation with real world time.

No, no, no.

This is the strict mathematical meaning of the word "monotonic",
but the way it's used in relation to OS clocks, it seems to
mean rather more than that.

A clock whose only guarantee is that it never goes backwards
is next to useless. For things like benchmarks and timeouts,
the important thing about a clock that it *keeps going forward*
at a reasonably constant rate. On the other hand, it can have
an arbitrary starting point and doesn't have to be related
to any external time standard.

I'm assuming this is what Linux et al mean when they talk
about a "monotonic clock", because anything else doesn't make
sense.

So if we're going to use the term "monotonic" at all, I think we
should explicitly define it as having this meaning, i.e.
both mathematically monotonic and steady. Failure to be clear
about this has caused a huge amount of confusion in this thead
so far.

--
Greg
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 06Apr2012 13:14, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
| Cameron Simpson wrote:
| > A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
| > polls of the clock. On its own it says nothing about steadiness or
| > correlation with real world time.
|
| No, no, no.
| This is the strict mathematical meaning of the word "monotonic",
| but the way it's used in relation to OS clocks, it seems to
| mean rather more than that.

It's like my next paragraph didn't exist...

_Quality of implementation_ says that the montonic() call should try to
return a close that is monotonic and _also_ steady and preferably precise.
How these things are balancer is a matter of policy.

To forstall things, right at the bottom of this I'm going to say i agree
with you about objectives, but not about terminology. I maintain that
"monotonic" still means what I said, and that it is the combination of
the word with "clock" that brings in your other criteria.

| A clock whose only guarantee is that it never goes backwards
| is next to useless. For things like benchmarks and timeouts,
| the important thing about a clock that it *keeps going forward*
| at a reasonably constant rate. [...]
| I'm assuming this is what Linux et al mean when they talk
| about a "monotonic clock", because anything else doesn't make
| sense.

This is why I say there's no global understanding. Why assume?
On a Linux box, "man 3 clock_getres" says (sorry, this is a bit wordy):

All implementations support the system-wide real-time clock, which
is identified by CLOCK_REALTIME. Its time represents seconds and
nanoseconds since the Epoch. When its time is changed, timers for
a relative interval are unaffected, but timers for an absolute point
in time are affected.

More clocks may be implemented. The interpretation of the
corresponding time values and the effect on timers is unspecified.

Sufficiently recent versions of glibc and the Linux kernel support
the following clocks:

CLOCK_REALTIME
System-wide real-time clock. Setting this clock requires
appropriate privileges.

CLOCK_MONOTONIC
Clock that cannot be set and represents monotonic time since
some unspecified starting point.

CLOCK_MONOTONIC_RAW (since Linux 2.6.28; Linux-specific)
Similar to CLOCK_MONOTONIC, but provides access to a raw hard-
ware-based time that is not subject to NTP adjustments.

CLOCK_PROCESS_CPUTIME_ID
High-resolution per-process timer from the CPU.

CLOCK_THREAD_CPUTIME_ID
Thread-specific CPU-time clock.

The first paragraph is very clear about real time (wall clock, and what
time.time() does, being "seconds since the epoch"). The CLOCK_MONOTONIC*
modes clearly imply steadiness.

"man 3p clock_getres" (POSIX clock_getres) is even more verbose and general.

| So if we're going to use the term "monotonic" at all, I think we
| should explicitly define it as having this meaning, i.e.
| both mathematically monotonic and steady.

I think if it is too "unsteady" then it is not "time".

So I actually side with you in the requirement for a "clock", but
monotonic alone does not mean that. Quality of implementation _may_ mean
we don't offer something abjectly erratic.

| Failure to be clear
| about this has caused a huge amount of confusion in this thead
| so far.

And burdening the word "monotonic" itself with exciting new meanings
doesn't help. "monotonic clock", sure, _that_ has additional
connotations, but not just the word monotonic.

Cheers,
--
Cameron Simpson <cs@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

We had the experience, but missed the meaning. - T.S. Eliot
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Greg Ewing wrote:
> Cameron Simpson wrote:
>
>> A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
>> polls of the clock. On its own it says nothing about steadiness or
>> correlation with real world time.
>
> No, no, no.
>
> This is the strict mathematical meaning of the word "monotonic",
> but the way it's used in relation to OS clocks, it seems to
> mean rather more than that.
>
> A clock whose only guarantee is that it never goes backwards
> is next to useless. For things like benchmarks and timeouts,
> the important thing about a clock that it *keeps going forward*

That would be a *strictly* monotonic clock, as opposed to merely monotonic.

And yes, a merely monotonic clock could just return a constant value, forever:

9, 9, 9, 9, 9, ...

and yes, such a thing would be useless.

Various people have suggested caching the last value of time() and re-using it
if the new value is in the past. This will give a monotonic clock, but since
it can give constant timestamps for an indefinite period, it's usefulness is
limited.

I earlier put forward an alternate implementation which gives no more than one
such constant tick in a row. If you know the hardware resolution of the clock,
you can even avoid that single constant tick by always advancing the timestamp
by that minimum resolution:

_prev = _prev_raw = 0
_res = 1e-9 # nanosecond resolution
def monotonic():
global _prev, _prev_raw
raw = time()
delta = max(_res, raw - _prev_raw)
_prev_raw = raw
_prev += delta
return _prev

Even if time() jumps backwards, or stays constant, monotonic() here will be
strictly monotonic.



--
Steven
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Cameron Simpson wrote:

> I maintain that
> "monotonic" still means what I said, and that it is the combination of
> the word with "clock" that brings in your other criteria.

I'm not trying to redefine the word "monotonic" in general.
All I'm saying is that *if* the PEP is going to talk about
a "monotonic clock" or "monotonic time", it should clearly
define what this means, because it's not obvious to everyone
that it implies something more than the mathematical meaning.

Alternatively, don't use the word "monotonic" at all, and
find a better term.

> CLOCK_MONOTONIC
> Clock that cannot be set and represents monotonic time since
> some unspecified starting point.

Which doesn't help very much, because it talks about "monotonic
time" without saying what that means. Googling for that phrase
doesn't seem turn up anything very useful. Apparently we're
supposed to just know.

--
Greg
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Steven D'Aprano wrote:
> Greg Ewing wrote:
>
>> the important thing about a clock that it *keeps going forward*
>
> That would be a *strictly* monotonic clock, as opposed to merely monotonic.

Well, yes, but even that's not enough -- it needs to go forward
at a reasonably constant rate, otherwise it's just as useless.
If it had enough resolution, it could go forward by one femtosecond
every hour for a while and still call itself strictly monotonic...

--
Greg
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 6 April 2012 02:50, Cameron Simpson <cs@zip.com.au> wrote:
(Quoted from the Linux manpage)

> All implementations support the system-wide real-time clock, which
> is identified by CLOCK_REALTIME. Its time represents seconds and
> nanoseconds since the Epoch. When its time is changed, timers for
> a relative interval are unaffected, but timers for an absolute point
> in time are affected.
>

This made me think. They make a distinction between "timers for a relative
interval" and "timers for an absolute point in time".

But that's not right - *all* of the clock calls we are talking about here
return a *single* number. Interpreting that as an absolute time needs an
epoch. Or to put it another way, clock values are always meaningless
without context - whereas clock *differences* are what actually carry
meaning (in terms of a duration).

On that basis, I'd say that

- A clock that doesn't get adjusted or slewed is a tick counter (which
technically doesn't have any relationship to time, although the tick
frequency can be used to convert to seconds, but see the next entry)
- A clock that gets slewed but not adjusted is a seconds counter (or at
least, the nearest approximation the system can provide - presumably better
than a tick counter)
- A clock that gets adjusted is not an interval timer at all, but an
absolute timer (which therefore shouldn't really be used for benchmarking
or timeouts)

It seems to me that what *I* would most often need are the second two of
these (to at least as high a precision as my app needs, which may vary but
"to the highest precision possible" would do :-)) I'd be happy for a
seconds counter to fallback to a tick counter converted to seconds using
its frequency - slewing is simply an accuracy improvement process, as far
as I can see.

It seems to me that the current time.time() and time.wallclock() are the
right names for my "absolute timer" and "seconds timer" above. Whether
their implementations match my definitions I'm not sure, but that's what
I'd hope. One thing I would expect is that time.wallclock() would never go
backwards (so differences are always positive). The various other debates
about monotonic, steady, etc, seem to me to be only relevant for specialist
uses that I don't care about.

As regards suspension, if I'm timing intervals and the system suspends, I'd
be happy to say all bets are off. Similarly with timeouts. If I cared, I'd
simply make sure the system didn't suspend :-)

As far as comparability between different threads or processes are
concerned, I would expect absolute time (time.time) to be the same across
threads or processes (but wouldn't generally write apps that were affected
if it weren't - at least by small amounts), but I wouldn't expect
time.wallclock values obtained in different threads or processes to be
comparable (mostly because I can't think of a case where I'd compare them).
Where VMs or multiple machines are involved, I wouldn't even expect
absolute time to match (but that's the job of NTP, and if time.time follows
NTP, there's no reason why there would be an issue even there).

Summary: I'm happy with time.time and time.wallclock. The rest of this
debate doesn't matter for my usecases (and I suspect many other people's in
practice).

[Update, after I downloaded and installed 3.3a2] Bah, looks like
time.wallclock is gone. (Actually, looks like it was documented but not
implemented in 3.3a1!). Actually, the docs and the implementation don't
match - clock_gettime is documented as available, but it's not (at least on
Windows). I still prefer time.wallclock() as described above and in the
3.3a1 documentation. I thought I knew what was going on, but now I'm
confused. My comments above still stand, though.

Paul.

On Thu, Apr 5, 2012 at 12:32, Victor Stinner <victor.stinner@gmail.com> wrote:
> I prefer to use CLOCK_MONOTONIC, not because it is also available for
> older Linux kernels, but because it is more reliable. Even if the
> underlying clock source is unstable (unstable frequency), a delta of
> two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
> whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
> smaller than a second.

Aha. OK, CLOCK_MONOTONIC it is then.

//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 4 April 2012 10:33, Steven D'Aprano <steve@pearwood.info> wrote:

> try:
> from os import bestclock as _bestclock
> except ImportError:
> _bestclock = time
>

My problem here is that "best clock" means different things to different
people (as the number of emails shows).

I think exposing specific clocks is also useful (sometimes people may need
a steady clock, and early failure is better than clock skew). However, I
propose a loosely classified set of clocks built on top of the specific
clocks, all of which can fall back to the lowest precision/non-monotonic
clock if needed.

1. The "steadiest" clock on the system. Ideally this would be a steady
clock, but may not be.

2. The "most precise" clock on the system. This would have the finest-grain
tick available on the system.

3. The "highest performance" (or maybe "lowest latency") clock. This would
be whichever clock on the system returned its results fastest.

I'm not sure if there are more that would be needed ("most accurate" comes
to mind, but feels like it's arbitrarily choosing between steadiest and
most precise, so I don't think it's valid).

By choosing relative terms, it caters to people's desire to have the "best"
clock, but doesn't set the expectation that the behaviour is guaranteed.

Tim Delaney

> 1. The "steadiest" clock on the system. Ideally this would be a steady
> clock, but may not be.

time.monotonic() as proposed in the PEP 418 *is* the steadiest
available clock, but we cannot say that it is steady :-)

> 2. The "most precise" clock on the system. This would have the finest-grain
> tick available on the system.

It's discussed in the "Deferred API: time.perf_counter()" section. It
would be nice to provide such clock, but I don't feel able right now
to propose ab API for such requirement. It's unclear to me if it must
be monotonic, steady, count elapsed time during a sleep or not, etc.

It is already very hard to propose one single time function
(time.monotonic), so I chose to simplify the PEP and not propose two
functions but only one :-)

> 3. The "highest performance" (or maybe "lowest latency") clock. This would
> be whichever clock on the system returned its results fastest.

Linux provides CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE
clocks, reading the ACPI Power Management clock is known to be slow.
But should the clock be monotonic or not? Return seconds or CPU ticks?
If the clock is not well defined, it's useless or at least, not
portable. Exposing CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE
constants should be enough.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 4/5/2012 6:32 AM, Victor Stinner wrote:
>>>> Since the only monotonic clock that can be adjusted by NTP is Linux'
>>>> CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always
>>>> give a clock that isn't adjusted by NTP.
>>>
>>> I thought we decided that NTP adjustment isn't an issue, because
>>> it's always gradual.
>>
>> Well, in timings it is an issue, but perhaps not a big one. :-)
>> In any case, which one we use will not change the API, so if it is
>> decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in
>> the future, once Linux< 2.6.26 (or whatever it was) is deemed
>> unsupported.
>
> I prefer to use CLOCK_MONOTONIC, not because it is also available for
> older Linux kernels, but because it is more reliable. Even if the
> underlying clock source is unstable (unstable frequency), a delta of
> two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
> whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
> smaller than a second. time.monotonic() unit is the second, as written
> in its documentation.
>

I believe the above is only true for sufficiently large time deltas. One
of the major purposes of NTP slewing is to give up some short term
accuracy in order to achieve long term accuracy (e.g. whenever the clock
is found to be ahead of real time it is purposefully ticked slower than
real time).

So for benchmarking it would not be surprising to be better off with the
non-adjusted clock. Ideally there would be a clock that was slewed "just
enough" to try and achieve short term accuracy, but I don't know of
anything providing that.

Janzert

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

2012/4/7 Janzert <janzert@janzert.com>:
> On 4/5/2012 6:32 AM, Victor Stinner wrote:
>> I prefer to use CLOCK_MONOTONIC, not because it is also available for
>> older Linux kernels, but because it is more reliable. Even if the
>> underlying clock source is unstable (unstable frequency), a delta of
>> two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
>> whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
>> smaller than a second. time.monotonic() unit is the second, as written
>> in its documentation.
>
> I believe the above is only true for sufficiently large time deltas. One of
> the major purposes of NTP slewing is to give up some short term accuracy in
> order to achieve long term accuracy (e.g. whenever the clock is found to be
> ahead of real time it is purposefully ticked slower than real time).

I don't think that NTP works like that. NTP only uses very smooth adjustements:

""slewing": change the clock frequency to be slightly faster or slower
(which is done with adjtime()). Since the slew rate is limited to 0.5
ms/s, each second of adjustment requires an amortization interval of
2000 s. Thus, an adjustment of many seconds can take hours or days to
amortize."
http://www.python.org/dev/peps/pep-0418/#ntp-adjustment

> So for benchmarking it would not be surprising to be better off with the
> non-adjusted clock. Ideally there would be a clock that was slewed "just
> enough" to try and achieve short term accuracy, but I don't know of anything
> providing that.

time.monotonic() is not written for benchmarks. It does not have the
highest frequecency, it's primary property is that is monotonic. A
side effect is that it is usually the steadiest clock.

For example, on Windows time.monotonic() has only an accuracy of 15 ms
(15 milliseconds not 15 microseconds).

If you consider that the PEP should also solve the issue of
benchmarking clock, we should continue the work on this section:
http://www.python.org/dev/peps/pep-0418/#deferred-api-time-perf-counter

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Victor Stinner wrote:
> 2012/4/7 Janzert <janzert@janzert.com>:
>> On 4/5/2012 6:32 AM, Victor Stinner wrote:
>>> I prefer to use CLOCK_MONOTONIC, not because it is also available for
>>> older Linux kernels, but because it is more reliable. Even if the
>>> underlying clock source is unstable (unstable frequency), a delta of
>>> two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
>>> whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
>>> smaller than a second. time.monotonic() unit is the second, as written
>>> in its documentation.
>> I believe the above is only true for sufficiently large time deltas. One of
>> the major purposes of NTP slewing is to give up some short term accuracy in
>> order to achieve long term accuracy (e.g. whenever the clock is found to be
>> ahead of real time it is purposefully ticked slower than real time).
>
> I don't think that NTP works like that. NTP only uses very smooth adjustements:
>
> ""slewing": change the clock frequency to be slightly faster or slower
> (which is done with adjtime()). Since the slew rate is limited to 0.5
> ms/s, each second of adjustment requires an amortization interval of
> 2000 s. Thus, an adjustment of many seconds can take hours or days to
> amortize."
> http://www.python.org/dev/peps/pep-0418/#ntp-adjustment


That is incorrect. NTP by default will only slew the clock for small
discrepancies. For large discrepancies, it will step the clock, causing the
time to jump. By default, "large" here means more than 128 milliseconds.

Yes, milliseconds.

http://www.ntp.org/ntpfaq/NTP-s-config-tricks.htm#AEN4249


In any case, NTP is not the only thing that adjusts the clock, e.g. the
operating system will adjust the time for daylight savings.


--
Steven

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

2012/4/7 Steven D'Aprano <steve@pearwood.info>:
> Victor Stinner wrote:
>>
>> 2012/4/7 Janzert <janzert@janzert.com>:
>>>
>>> On 4/5/2012 6:32 AM, Victor Stinner wrote:
>>>>
>>>> I prefer to use CLOCK_MONOTONIC, not because it is also available for
>>>> older Linux kernels, but because it is more reliable. Even if the
>>>> underlying clock source is unstable (unstable frequency), a delta of
>>>> two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
>>>> whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
>>>> smaller than a second. time.monotonic() unit is the second, as written
>>>> in its documentation.
>>>
>>> I believe the above is only true for sufficiently large time deltas. One
>>> of
>>> the major purposes of NTP slewing is to give up some short term accuracy
>>> in
>>> order to achieve long term accuracy (e.g. whenever the clock is found to
>>> be
>>> ahead of real time it is purposefully ticked slower than real time).
>>
>> I don't think that NTP works like that. NTP only uses very smooth
>> adjustements:
>>
>> ""slewing": change the clock frequency to be slightly faster or slower
>> (which is done with adjtime()). Since the slew rate is limited to 0.5
>> ms/s, each second of adjustment requires an amortization interval of
>> 2000 s. Thus, an adjustment of many seconds can take hours or days to
>> amortize."
>> http://www.python.org/dev/peps/pep-0418/#ntp-adjustment
>

> That is incorrect. NTP by default will only slew the clock for small
> discrepancies. For large discrepancies, it will step the clock, causing the
> time to jump. By default, "large" here means more than 128 milliseconds.
>
> Yes, milliseconds.
>
> http://www.ntp.org/ntpfaq/NTP-s-config-tricks.htm#AEN4249

We are talking about CLOCK_MONOTONIC. Steping is disabled on this clock.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 07Apr2012 20:40, Steven D'Aprano <steve@pearwood.info> wrote:
| Victor Stinner wrote:
| > I don't think that NTP works like that. NTP only uses very smooth adjustements:
[...]
| > http://www.python.org/dev/peps/pep-0418/#ntp-adjustment
|
| That is incorrect. NTP by default will only slew the clock for small
| discrepancies. For large discrepancies, it will step the clock, causing the
| time to jump. By default, "large" here means more than 128 milliseconds.
| Yes, milliseconds.
| http://www.ntp.org/ntpfaq/NTP-s-config-tricks.htm#AEN4249
|
| In any case, NTP is not the only thing that adjusts the clock, e.g. the
| operating system will adjust the time for daylight savings.

Ignoring the discussion of NTP, daylight saving is a _presentation_
issue. It is _display_. The OS clock does not change for daylight
saving! Think: "seconds since the epoch". This is a continuous function.
Daylight saving presentation occurs turning a seconds-since-epoch into a
human decomposed time (hours, etc).

Now, AFAIR, Windows used to run its system clock in "local time"
i.e. it did have to jump its clock for daylight saving. Hopefully that
is long gone.

UNIX never did this. It ran in seconds since the epoch (in its case, start of
01jan1970 GMT). Printing dates and timestamps to humans needed daylight
saving knowledge etc.

Cheers,
--
Cameron Simpson <cs@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Stan Malyshev <stas@netcom.com> wrote:
| You're dragging a peg in a blind hairpin when you see a patch of coolant
| up ahead, and you hear the airbrakes of an oncoming 18-wheeler.
| What do you do? WHAT DO YOU DO? ("Speed", anyone?)
Shoot my pillion? - Vociferous Mole <stevegr@Starbase.NeoSoft.COM>
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote:

> In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings.

Daylight savings time is not a clock adjustment, at least not in the sense this thread has mostly been talking about the word "clock". It doesn't affect the "seconds from epoch" measurement, it affects the way in which the clock is formatted to the user.

-glyph

On Sat, Apr 7, 2012 at 4:56 PM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:

> On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote:
>
> In any case, NTP is not the only thing that adjusts the clock, e.g. the
> operating system will adjust the time for daylight savings.
>
>
> Daylight savings time is not a clock adjustment, at least not in the sense
> this thread has mostly been talking about the word "clock". It doesn't
> affect the "seconds from epoch" measurement, it affects the way in which
> the clock is formatted to the user.
>
> -glyph
>

even on windows where the system hardware clock is maintained in local time?

Is it still? I thought they fixed that ages ago?

On Mon, Apr 9, 2012 at 4:42 PM, Gregory P. Smith <greg@krypto.org> wrote:
>
> On Sat, Apr 7, 2012 at 4:56 PM, Glyph Lefkowitz <glyph@twistedmatrix.com>
> wrote:
>>
>> On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote:
>>
>> In any case, NTP is not the only thing that adjusts the clock, e.g. the
>> operating system will adjust the time for daylight savings.
>>
>>
>> Daylight savings time is not a clock adjustment, at least not in the sense
>> this thread has mostly been talking about the word "clock".  It doesn't
>> affect the "seconds from epoch" measurement, it affects the way in which the
>> clock is formatted to the user.
>>
>> -glyph
>
>
> even on windows where the system hardware clock is maintained in local time?
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com