a key in a dict:

>>> {float("nan"), float("nan")}

{nan, nan}

except that sometimes it can't:

>>> nan = float("nan")

>>> {nan, nan}

{nan}

--

http://mail.python.org/mailman/listinfo/python-list

float("nan") in set or as key

a key in a dict:

>>> {float("nan"), float("nan")}

{nan, nan}

except that sometimes it can't:

>>> nan = float("nan")

>>> {nan, nan}

{nan}

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Here's a curiosity. float("nan") can occur multiple times in a set or as

> a key in a dict:

>

> >>> {float("nan"), float("nan")}

> {nan, nan}

>

> except that sometimes it can't:

>

> >>> nan = float("nan")

> >>> {nan, nan}

> {nan}

It's fundamentally because NaN is not equal to itself, by design.

Dictionaries and sets rely on equality to test for uniqueness of keys or

elements.

>>> nan = float("nan")

>>> nan == nan

False

In short, don't do that.

--

Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/

San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis

There was never a good war or a bad peace.

-- Benjamin Franklin, 1706-1790

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Here's a curiosity. float("nan") can occur multiple times in a set or as

> a key in a dict:

>

> >>> {float("nan"), float("nan")}

> {nan, nan}

That's an implementation detail. Python is free to reuse the same object

when you create an immutable object twice on the same line, but in this

case doesn't. (I don't actually know if it ever does, but it could.)

And since NAN != NAN always, you can get two NANs in the one set, since

they're unequal.

> when you write float('nan')

>

> except that sometimes it can't:

>

> >>> nan = float("nan")

> >>> {nan, nan}

> {nan}

But in this case, you try to put the same NAN in the set twice. Since

sets optimize element testing by checking for identity before equality,

the NAN only goes in once.

--

Steven

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Here's a curiosity. float("nan") can occur multiple times in a set or as

> a key in a dict:

>

> >>> {float("nan"), float("nan")}

> {nan, nan}

>

These two nans are not equal (they are two different nans)

> except that sometimes it can't:

>

> >>> nan = float("nan")

> >>> {nan, nan}

> {nan}

This is the same nan, so it is equal to itself.

Two "nan"s are not equal in the manner that 1.0 and 1.0 are equal:

>>> 1.0 == 1.0

True

>>> float("nan") == float("nan")

False

I can't cite this in a spec, but it makes sense (to me) that two things

which are nan are not necessarily the same nan.

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> MRAB wrote:

>

>> Here's a curiosity. float("nan") can occur multiple times in a set or as a

>> key in a dict:

>>

>> >>> {float("nan"), float("nan")}

>> {nan, nan}

>>

>> except that sometimes it can't:

>>

>> >>> nan = float("nan")

>> >>> {nan, nan}

>> {nan}

>>

>

> It's fundamentally because NaN is not equal to itself, by design.

> Dictionaries and sets rely on equality to test for uniqueness of keys or

> elements.

>

>

> >>> nan = float("nan")

> >>> nan == nan

> False

>

> In short, don't do that.

There's a second part the mystery - sets and dictionaries (and I think

lists) assume that identify implies equality (hence the second result). This

was recently discussed on python-dev, and the decision was to leave things

as-is.

Tim Delaney

Re: float("nan") in set or as key
[ In reply to ]

> This is the same nan, so it is equal to itself.

>

Actually, they're not. But it's possible the dictionary uses an 'is'

check to save computation, and if one thing 'is' another, it is

assumed to equal it. That's true of most well-behaved objects, but nan

is not well-behaved :)

Chris Angelico

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> On Sun, 2011-05-29 at 00:41 +0100, MRAB wrote:

>>>> 1.0 == 1.0

> True

>>>> float("nan") == float("nan")

> False

>

> I can't cite this in a spec, but it makes sense (to me) that two things

> which are nan are not necessarily the same nan.

It's part of the IEEE standard.

--

Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/

San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis

There was never a good war or a bad peace.

-- Benjamin Franklin, 1706-1790

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> float("nan") can occur multiple times in a set or as

> a key in a dict:

>

> >>> {float("nan"), float("nan")}

> {nan, nan}

>

> except that sometimes it can't:

>

> >>> nan = float("nan")

> >>> {nan, nan}

> {nan}

NaNs are weird. They're not equal to themselves:

Python 2.7 (r27:82500, Oct 15 2010, 21:14:33)

[GCC 4.2.1 (Apple Inc. build 5664)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> nan = float("nan")

>>> nan == nan

False

This confuses the daylights out of Python's dict lookup machinery,

which assumes that two references to the same object can't possibly

compare unequal, so it doesn't bother calling __eq__ on them.

--

Greg

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> On Sun, 2011-05-29 at 00:41 +0100, MRAB wrote:

>> Here's a curiosity. float("nan") can occur multiple times in a set or as

>> a key in a dict:

>>

>> >>> {float("nan"), float("nan")}

>> {nan, nan}

>>

> These two nans are not equal (they are two different nans)

>

>> except that sometimes it can't:

>>

>> >>> nan = float("nan")

>> >>> {nan, nan}

>> {nan}

>

> This is the same nan, so it is equal to itself.

No, it's not.

>>> x = float("nan")

>>> y = x

>>> x is y

True

>>> x == y

False

> I can't cite this in a spec, but it makes sense (to me) that two things

> which are nan are not necessarily the same nan.

Even if they _are_ the same nan, it's still not equal to itself.

--

Grant

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> MRAB wrote:

>> float("nan") can occur multiple times in a set or as a key in a dict:

>>

>> >>> {float("nan"), float("nan")}

>> {nan, nan}

>>

>> except that sometimes it can't:

>>

>> >>> nan = float("nan")

>> >>> {nan, nan}

>> {nan}

>

> NaNs are weird. They're not equal to themselves:

>

> Python 2.7 (r27:82500, Oct 15 2010, 21:14:33)

> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin

> Type "help", "copyright", "credits" or "license" for more information.

> >>> nan = float("nan")

> >>> nan == nan

> False

>

> This confuses the daylights out of Python's dict lookup machinery,

> which assumes that two references to the same object can't possibly

> compare unequal, so it doesn't bother calling __eq__ on them.

Right.

The correct answer to "nan == nan" is to raise an exception, because

you have asked a question for which the answer is nether True nor False.

The correct semantics for IEEE floating point look something like

this:

1/0 INF

INF + 1 INF

INF - INF NaN

INF == INF unordered

NaN == NaN unordered

INF and NaN both have comparison semantics which return

"unordered". The FPU sets a bit for this, which most language

implementations ignore. But you can turn on floating point

exception traps, and on x86 machines, they're exact - the

exception will occur exactly at the instruction which

triggered the error. In superscalar CPUs, a sizable part of

the CPU handles the unwinding necessary to do that. x86 does

it, because it's carefully emulating non-superscalar machines.

Most RISC machines don't bother.

Python should raise an exception on unordered comparisons.

Given that the language handles integer overflow by going to

arbitrary-precision integers, checking the FPU status bits is

cheap.

The advantage of raising an exception is that the logical operations

still work. For example,

not (a == b)

a != b

will always return the same results if exceptions are raised for

unordered comparison results. Also, exactly one of

a = b

a < b

a > b

is always true - something sorts tend to assume.

If you get an unordered comparison exception, your program

almost certainly was getting wrong answers.

(I used to do dynamics simulation engines, where this mattered.)

John Nagle

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> There's a second part the mystery - sets and dictionaries (and

> I think lists) assume that identify implies equality (hence

> the second result). This was recently discussed on

> python-dev, and the decision was to leave things as-is.

On Sonntag 29 Mai 2011, Grant Edwards wrote:

> Even if they are the same nan, it's still not equal to itself.

if I understand this thread correctly, they are not equal to

itself as specified by IEEE but Python treats them equal in

sets and dictionaries for performance reasons

--

Wolfgang

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> On Sun, May 29, 2011 at 10:28 AM, Albert Hopkins

> <marduk@letterboxes.org> wrote:

>> This is the same nan, so it is equal to itself.

>>

>>

> Actually, they're not. But it's possible the dictionary uses an 'is'

> check to save computation, and if one thing 'is' another, it is assumed

> to equal it. That's true of most well-behaved objects, but nan is not

> well-behaved :)

*Exactly* correct.

NAN != NAN even if they are the same NAN, by design. This makes NANs ill-

behaved, but usefully so. Most (all?) Python built-ins assume that any

object X is equal to itself, so they behave strangely with NANs.

--

Steven

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> The correct answer to "nan == nan" is to raise an exception, because

> you have asked a question for which the answer is nether True nor False.

Wrong.

The correct answer to "nan == nan" is False, they are not equal. Just as

None != "none", and 42 != [42], or a teacup is not equal to a box of

hammers.

Asking whether NAN < 0 could arguably either return "unordered" (raise an

exception) or return False ("no, NAN is not less than zero; neither is it

greater than zero"). The PowerPC Macintishes back in the 1990s supported

both behaviours. But that's different to equality tests.

> The correct semantics for IEEE floating point look something like

> this:

>

> 1/0 INF

> INF + 1 INF

> INF - INF NaN

> INF == INF unordered

Wrong. Equality is not an order comparison.

--

Steven

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> On Sonntag 29 Mai 2011, Tim Delaney wrote:

>> There's a second part the mystery - sets and dictionaries (and

>> I think lists) assume that identify implies equality (hence

>> the second result). This was recently discussed on

>> python-dev, and the decision was to leave things as-is.

>

> On Sonntag 29 Mai 2011, Grant Edwards wrote:

>> Even if they are the same nan, it's still not equal to itself.

>

> if I understand this thread correctly, they are not equal to itself

> as specified by IEEE

And Python follows that convention.

> but Python treats them equal in sets and dictionaries for performance

> reasons

It treats them as identical (not sure if that's the right word). The

implementation is checking for ( A is B or A == B ). Presumably, the

assumpting being that all objects are equal to themselves. That

assumption is not true for NaN objects, so the buggy behavior is

observed.

--

Grant

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> On 2011-05-29, Wolfgang Rohdewald<wolfgang@rohdewald.de> wrote:

>> On Sonntag 29 Mai 2011, Tim Delaney wrote:

>>> There's a second part the mystery - sets and dictionaries (and

>>> I think lists) assume that identify implies equality (hence

>>> the second result). This was recently discussed on

>>> python-dev, and the decision was to leave things as-is.

>>

>> On Sonntag 29 Mai 2011, Grant Edwards wrote:

>>> Even if they are the same nan, it's still not equal to itself.

>>

>> if I understand this thread correctly, they are not equal to itself

>> as specified by IEEE

>

> And Python follows that convention.

>

>> but Python treats them equal in sets and dictionaries for performance

>> reasons

>

> It treats them as identical (not sure if that's the right word). The

> implementation is checking for ( A is B or A == B ). Presumably, the

> assumpting being that all objects are equal to themselves. That

> assumption is not true for NaN objects, so the buggy behavior is

> observed.

>

Would there be any advantage to making NaN a singleton? I'm thinking

that it could make checking for it cheaper in the implementation of

sets and dicts. Or making NaN unhashable?

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Would there be any advantage to making NaN a singleton? I'm thinking

> that it could make checking for it cheaper in the implementation of

> sets and dicts. Or making NaN unhashable?

Doesn't matter. It still wouldn't be equal to itself, even though it

'is' itself, which will greatly confuse anything that optimizes that

away. Numbers are well-behaved; NaN is not a number; NaN is not

well-behaved. It makes sense... in a way.

Chris Angelico

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Would there be any advantage to making NaN a singleton? I'm thinking

> that it could make checking for it cheaper in the implementation of

> sets and dicts. Or making NaN unhashable?

It can't be a singleton, because IEEE 754 specifies millions of millions

of different NaN values. There are positive and negative NaNs, quiet

NaNs and signaling NaNs. 50 of 52 mantissa bits can vary freely, one bit

makes the difference between signaling and quiet NaNs and at least one

bit must be non-zero.

Christian

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Would there be any advantage to making NaN a singleton?

Absolutely not. That would be a step backwards.

NANs can carry payload (a code indicating what sort of NAN it represents

-- log(-1) and 1/INF are not the same). So although Python currently has

no easy way to access that payload (you can do it with the struct

module), it does exist and for serious work you would want to be able to

set and get it.

> I'm thinking

> that it could make checking for it cheaper in the implementation of sets

> and dicts.

I don't see how would it be cheaper, but even if it were, talk about a

micro-optimization! I'd really *love* to see the code where the time it

takes to insert a NAN in a set was the bottleneck!

> Or making NaN unhashable?

I could live with that, although I don't think it is necessary. What

actual problem are you hoping to solve here?

--

Steven

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> Am 29.05.2011 19:44, schrieb MRAB:

>> Would there be any advantage to making NaN a singleton? I'm thinking

>> that it could make checking for it cheaper in the implementation of

>> sets and dicts. Or making NaN unhashable?

>

> It can't be a singleton, because IEEE 754 specifies millions of millions

> of different NaN values.

A million-millioneton then? *wink*

> There are positive and negative NaNs,

I've never quite understood that. NANs are unordered, and therefore

cannot be said to be larger than zero (positive) or less than zero

(negative). So even if a NAN has the sign bit set, surely the right way

to think about that is to treat the sign bit as part of the payload?

It seems to me that talking about signed NANs is inaccurate and adds

confusion. NANs cause enough confusion as it is, without adding to it...

(I would expect the copysign function to honour the sign bit, so I

suppose in that sense one might describe NANs as signed.)

--

Steven

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

>> The correct answer to "nan == nan" is to raise an exception, because

>> you have asked a question for which the answer is nether True nor False.

>

> Wrong.

That's overstating it. There's a good argument to be made for raising an

exception. Bear in mind that an exception is not necessarily an error,

just an "exceptional" condition.

> The correct answer to "nan == nan" is False, they are not equal.

There is no correct answer to "nan == nan". Defining it to be false is

just the "least wrong" answer. Arguably, "nan != nan" should also be

false, but that would violate the invariant "(x != y) == !(x == y)".

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> On Sun, 29 May 2011 10:29:28 +0000, Steven D'Aprano wrote:

>

>>> The correct answer to "nan == nan" is to raise an exception,

>>> because

>>> you have asked a question for which the answer is nether True nor

>>> False.

>>

>> Wrong.

>

> That's overstating it. There's a good argument to be made for raising an

> exception.

If so, I've never heard it, and I cannot imagine what such a good

argument would be. Please give it.

(I can think of *bad* arguments, like "NANs confuse me and I don't

understand the reason for their existence, therefore I'll give them

behaviours that make no sense and aren't useful". But you did state there

is a *good* argument.)

> Bear in mind that an exception is not necessarily an error,

> just an "exceptional" condition.

True, but what's your point? Testing two floats for equality is not an

exceptional condition.

>> The correct answer to "nan == nan" is False, they are not equal.

>

> There is no correct answer to "nan == nan".

Why on earth not?

> Defining it to be false is just the "least wrong" answer.

So you say, but I think you are incorrect.

> Arguably, "nan != nan" should also be false,

> but that would violate the invariant "(x != y) == !(x == y)".

I cannot imagine what that argument would be. Please explain.

--

Steven

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

$ python

...

>>> nan = float("nan")

>>> nan

nan

>>> nan is nan

True

>>> nan == nan

False

In article <4de1e3e7$0$2195$742ec2ed@news.sonic.net>

John Nagle <nagle@animats.com> wrote:

> The correct answer to "nan == nan" is to raise an exception, because

>you have asked a question for which the answer is nether True nor False.

Well, in some sense, the "correct answer" depends on which question

you *meant* to ask. :-) Seriously, some (many?) instruction sets

have two kinds of comparison instructions: one that raises an

exception here, and one that does not.

> The correct semantics for IEEE floating point look something like

>this:

>

> 1/0 INF

> INF + 1 INF

> INF - INF NaN

> INF == INF unordered

> NaN == NaN unordered

>

>INF and NaN both have comparison semantics which return

>"unordered". The FPU sets a bit for this, which most language

>implementations ignore.

Again, this depends on the implementation.

This is similar to (e.g.) the fact that on the MIPS, there are two

different integer add instructions ("addi" and "addiu"): one

raises an overflow exception, the other performs C "unsigned"

style arithmetic (where, e.g., 0xffffffff + 1 = 0, in 32 bits).

>Python should raise an exception on unordered comparisons.

>Given that the language handles integer overflow by going to

>arbitrary-precision integers, checking the FPU status bits is

>cheap.

I could go for that myself. But then you also need a "don't raise

exception but give me an equality test result" operator (for various

special-case purposes at least) too. Of course a simple "classify

this float as one of normal, subnormal, zero, infinity, or NaN"

operator would suffice here (along with the usual "extract sign"

and "differentiate between quiet and signalling NaN" operations).

--

In-Real-Life: Chris Torek, Wind River Systems

Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603

email: gmail (figure it out) http://web.torek.net/torek/index.html

Re: float("nan") in set or as key
[ In reply to ]

> On Sun, 29 May 2011 22:19:49 +0100, Nobody wrote:

>

> > On Sun, 29 May 2011 10:29:28 +0000, Steven D'Aprano wrote:

> >

> >>> The correct answer to "nan == nan" is to raise an exception,

> >>> because

> >>> you have asked a question for which the answer is nether True nor

> >>> False.

> >>

> >> Wrong.

> >

> > That's overstating it. There's a good argument to be made for raising an

> > exception.

>

> If so, I've never heard it, and I cannot imagine what such a good

> argument would be. Please give it.

Floating point arithmetic evolved more or less on languages like Fortran where things like exceptions were unheard of, and defining NaN != NaN was a bad trick they chose for testing against NaN for lack of a better way.

If exceptions had commonly existed in that environment there's no chance they would have chosen that behavior; comparison against NaN (or any operation with NaN) would have signaled a floating point exception. That is the correct way to handle exceptional conditions.

The only reason to keep NaN's current behavior is to adhere to IEEE, but given that Python has trailblazed a path of correcting arcane mathematical behavior, I definitely see an argument that Python should do the same for NaN, and if it were done Python would be a better language.

Carl Banks

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> It treats them as identical (not sure if that's the right word). The

> implementation is checking for ( A is B or A == B ). Presumably, the

> assumpting being that all objects are equal to themselves. That

> assumption is not true for NaN objects, so the buggy behavior is

> observed.

Python makes this assumption in lots of common situations (apparently in an implementation-defined manner):

>>> nan = float("nan")

>>> nan == nan

False

>>> [nan] == [nan]

True

Therefore, I'd recommend never to rely on NaN != NaN except in casual throwaway code. It's too easy to forget that it will stop working when you throw an item into a list or tuple. There's a function, math.isnan(), that should be the One Obvious Way to test for NaN. NaN should also never be used as a dictionary key or in a set (of course).

If it weren't for compatibility with IEEE, there would be no sane argument that defining an object that is not equal to itself isn't a bug. But because there's a lot of code out there that depends on NaN != NaN, Python has to tolerate it.

Carl Banks

--

http://mail.python.org/mailman/listinfo/python-list

Re: float("nan") in set or as key
[ In reply to ]

> If exceptions had commonly existed in that environment there's no chance they would have chosen that behavior; comparison against NaN (or any operation with NaN) would have signaled a floating point exception. That is the correct way to handle exceptional conditions.

>

> The only reason to keep NaN's current behavior is to adhere to IEEE, but given that Python has trailblazed a path of correcting arcane mathematical behavior, I definitely see an argument that Python should do the same for NaN, and if it were done Python would be a better language.

If you're going to change behaviour, why have a floating point value

called "nan" at all? Other than being a title for one's grandmother,

what meaning does that string have, and why should it be able to be

cast as floating point?

Lifting from http://en.wikipedia.org/wiki/NaN a list of things that

can return a NaN (I've removed non-ASCII characters from this

snippet):

* Operations with a NaN as at least one operand.

(you need to bootstrap that somehow, so we can ignore this - it just

means that nan+1 = nan)

* The divisions 0/0 and infinity/infinity

* The multiplications 0*infinity and infinity*0

* The additions +inf + (-inf), (-inf) + +inf and equivalent subtractions

* The standard pow function and the integer exponent pown function

define 0**0, 1**inf, and inf**0 as 1.

* The powr function define all three indeterminate forms as invalid

operations and so returns NaN.

* The square root of a negative number.

* The logarithm of a negative number

* The inverse sine or cosine of a number that is less than -1 or

greater than +1.

Rather than having comparisons with NaN trigger exceptions, wouldn't

it be much cleaner to have all these operations trigger exceptions?

And, I would guess that they probably already do.

NaN has an additional use in that it can be used like a "null

pointer"; a floating-point variable can store 1.0, or 0.000000000005,

or "no there's no value that I'm storing in this variable". Since a

Python variable can contain None instead of a float, this use is

unnecessary too.

So, apart from float("nan"), are there actually any places where real

production code has to handle NaN? I was unable to get a nan by any of

the above methods, except for operations involving inf; for instance,

float("inf")-float("inf") == nan. All the others raised an exception

rather than return nan.

Chris Angelico

--

http://mail.python.org/mailman/listinfo/python-list