Mailing List Archive

Google peering in LAX
Anyone know why Google announces only aggregates via peering and
disaggregate prefixes over transit?

For example, I had a customer complaining about a path that was taking
the long way instead of via peering and when I looked I saw:

Only 172.217.0.0/16 over Any2 LAX

That plus 172.217.14.0/24 over transit

Any inquiries to Google just get a generic "we're not setting up any new
peering but we're on route servers" response for almost a year now. Or
is it because they don't send the /24's to route servers and I'm stuck
until they finish their forever improvement project to turn up a direct
neighbor?
Re: Google peering in LAX [ In reply to ]
In part, it might be because people you’re not paying may be less tolerant of anti-social behavior than people you are paying.

It does seem rather odd that Google would prefer to receive their traffic over transit, but I’m not going to try and second guess that decision.

Owen


> On Mar 2, 2020, at 12:40 PM, Seth Mattinen <sethm@rollernet.us> wrote:
>
> Anyone know why Google announces only aggregates via peering and disaggregate prefixes over transit?
>
> For example, I had a customer complaining about a path that was taking the long way instead of via peering and when I looked I saw:
>
> Only 172.217.0.0/16 over Any2 LAX
>
> That plus 172.217.14.0/24 over transit
>
> Any inquiries to Google just get a generic "we're not setting up any new peering but we're on route servers" response for almost a year now. Or is it because they don't send the /24's to route servers and I'm stuck until they finish their forever improvement project to turn up a direct neighbor?
Re: Google peering in LAX [ In reply to ]
On 3/2/20 12:44 PM, Owen DeLong wrote:
> In part, it might be because people you’re not paying may be less tolerant of anti-social behavior than people you are paying.
>

I'm not sure how I was being offensive but OK.
Re: Google peering in LAX [ In reply to ]
I believe Owen was referring here to Google's actions: that the disagg is
the antisocial behaviour and that transit providers (the people they are
paying) would be more tolerant of that antisocial behaviour than would be
peers (the people they are not paying).

On Mon., Mar. 2, 2020, 13:19 Seth Mattinen <sethm@rollernet.us> wrote:

> On 3/2/20 12:44 PM, Owen DeLong wrote:
> > In part, it might be because people you’re not paying may be less
> tolerant of anti-social behavior than people you are paying.
> >
>
> I'm not sure how I was being offensive but OK.
>
Re: Google peering in LAX [ In reply to ]
Yes… That’s correct.

Owen


> On Mar 2, 2020, at 2:20 PM, Hugo Slabbert <hugo@slabnet.com> wrote:
>
> I believe Owen was referring here to Google's actions: that the disagg is the antisocial behaviour and that transit providers (the people they are paying) would be more tolerant of that antisocial behaviour than would be peers (the people they are not paying).
>
> On Mon., Mar. 2, 2020, 13:19 Seth Mattinen <sethm@rollernet.us <mailto:sethm@rollernet.us>> wrote:
> On 3/2/20 12:44 PM, Owen DeLong wrote:
> > In part, it might be because people you’re not paying may be less tolerant of anti-social behavior than people you are paying.
> >
>
> I'm not sure how I was being offensive but OK.
Re: Google peering in LAX [ In reply to ]
On 3/2/20 2:20 PM, Hugo Slabbert wrote:
> I believe Owen was referring here to Google's actions: that the disagg
> is the antisocial behaviour and that transit providers (the people they
> are paying) would be more tolerant of that antisocial behaviour than
> would be peers (the people they are not paying).


I suppose that one went over my head.

To clarify I am the one with peering in LAX and I'm only seeing the big
aggregates via the Any2 Easy servers. At the moment I can only infer
that Google announces aggregates to the route servers and maybe one only
gets the /24's after you turn up a direct neighbor or PNI, but there's
no way to do that since Google isn't accepting new peering requests and
steers such requests back to what's available on route servers.

I suppose what I could do is filter /24's from 15169$ in the absence of
being able to see if a direct/PNI peering would include them where route
servers do not.
Re: Google peering in LAX [ In reply to ]
----- On Mar 2, 2020, at 5:37 PM, Seth Mattinen sethm@rollernet.us wrote:

> I suppose that one went over my head.
>
> To clarify I am the one with peering in LAX and I'm only seeing the big
> aggregates via the Any2 Easy servers. At the moment I can only infer
> that Google announces aggregates to the route servers and maybe one only
> gets the /24's after you turn up a direct neighbor or PNI, but there's
> no way to do that since Google isn't accepting new peering requests and
> steers such requests back to what's available on route servers.
>
> I suppose what I could do is filter /24's from 15169$ in the absence of
> being able to see if a direct/PNI peering would include them where route
> servers do not.

I would say it would be best to see if you can get a direct peer with Google via the IX. I have done this with some of the ISPs I work with. It was no additional cost since the physical connections are already in place and actually was highly recommended when first turning up the IX circuits.


-Randy
Re: Google peering in LAX [ In reply to ]
On Mar 2, 2020, at 17:38, Seth Mattinen <sethm@rollernet.us> wrote:
> ?On 3/2/20 2:20 PM, Hugo Slabbert wrote:
>> I believe Owen was referring here to Google's actions: that the disagg is the antisocial behaviour and that transit providers (the people they are paying) would be more tolerant of that antisocial behaviour than would be peers (the people they are not paying).
>
>
> I suppose that one went over my head.
>
> To clarify I am the one with peering in LAX and I'm only seeing the big aggregates via the Any2 Easy servers. At the moment I can only infer that Google announces aggregates to the route servers and maybe one only gets the /24's after you turn up a direct neighbor or PNI, but there's no way to do that since Google isn't accepting new peering requests and steers such requests back to what's available on route servers.
>
> I suppose what I could do is filter /24's from 15169$ in the absence of being able to see if a direct/PNI peering would include them where route servers do not.

Your routers, your decision.

But how much traffic are you sending TO Google? Most people get the vast majority of traffic FROM Google. They send you videos, you send them ACKs. Does it matter where the ACKs go?

--
TTFN,
patrick
Re: Google peering in LAX [ In reply to ]
On 3/2/20 3:02 PM, Randy Carpenter wrote:
> I would say it would be best to see if you can get a direct peer with Google via the IX. I have done this with some of the ISPs I work with. It was no additional cost since the physical connections are already in place and actually was highly recommended when first turning up the IX circuits.


They won't; I just get a canned message that says they aren't doing any
new IX peering "as we improve our automation systems".
Re: Google peering in LAX [ In reply to ]
You hit the nail on the head. Google only seems to announce a subset of their routes to the route servers, but does announce all routes (for some definition of “all”) to direct peers. I notice this every time I turn up a new IX and traffic heads off onto my backbone instead of the local IX.

I did a spot check and I get that /24 via my direct peering (along with the /16).

Justin Seabrook-Rocha
--
Xenith || xenith@xenith.org || http://xenith.org/



> On Mar 2, 2020, at 12:40, Seth Mattinen <sethm@rollernet.us> wrote:
>
> Anyone know why Google announces only aggregates via peering and disaggregate prefixes over transit?
>
> For example, I had a customer complaining about a path that was taking the long way instead of via peering and when I looked I saw:
>
> Only 172.217.0.0/16 over Any2 LAX
>
> That plus 172.217.14.0/24 over transit
>
> Any inquiries to Google just get a generic "we're not setting up any new peering but we're on route servers" response for almost a year now. Or is it because they don't send the /24's to route servers and I'm stuck until they finish their forever improvement project to turn up a direct neighbor?
Re: Google peering in LAX [ In reply to ]
On 3/2/20 3:09 PM, Patrick W. Gilmore wrote:
>
> Your routers, your decision.
>
> But how much traffic are you sending TO Google? Most people get the vast
> majority of traffic FROM Google. They send you videos, you send them
> ACKs. Does it matter where the ACKs go?


A customer is complaining that data they're sending is going over a
higher latency (longer) path. I don't know what they're doing I don't
generally ask why, but they claim it's a problem for whatever they're
doing and I don't have a reason to doubt them. It's not youtube.

I agree that it's an undesirable long term solution but if filtering
select transit-only /24's shifts the path to peering and reduces
latency, if the customer is happy then I'm happy and if/when Google
starts accepting peering requests again I'll revisit it.
Re: Google peering in LAX [ In reply to ]
On Mar 2, 2020, at 6:30 PM, Seth Mattinen <sethm@rollernet.us> wrote:
> On 3/2/20 3:09 PM, Patrick W. Gilmore wrote:
>> Your routers, your decision.
>> But how much traffic are you sending TO Google? Most people get the vast majority of traffic FROM Google. They send you videos, you send them ACKs. Does it matter where the ACKs go?
>
>
> A customer is complaining that data they're sending is going over a higher latency (longer) path. I don't know what they're doing I don't generally ask why, but they claim it's a problem for whatever they're doing and I don't have a reason to doubt them. It's not youtube.
>
> I agree that it's an undesirable long term solution but if filtering select transit-only /24's shifts the path to peering and reduces latency, if the customer is happy then I'm happy and if/when Google starts accepting peering requests again I'll revisit it.

Again, your routers, your decision. But if I had a customer who was complaining, I would take steps to fix it.

Google is sending you prefixes over the IX. You have every right to send them traffic over the IX to those prefixes.

That said, I fear this is going to be a problem long term. A blind “no /24s” filter is dangerous, plus it might solve all traffic issues. It is going to take effort to be sure you don’t get bitten by the Law Of Unintended Consequences.

Good luck.

--
TTFN,
patrick
Re: Google peering in LAX [ In reply to ]
On 3/2/20 4:32 PM, Patrick W. Gilmore wrote:
> That said, I fear this is going to be a problem long term. A blind “no /24s” filter is dangerous, plus it might solve all traffic issues. It is going to take effort to be sure you don’t get bitten by the Law Of Unintended Consequences.


As soon as Google un-freezes new peering requests so I can get a direct
peering that includes appropriate /24's I've been told offlist I should
get (instead of the route server subset) I'll happily remove the transit
filters. But I can only work with what I'm given.
Re: Google peering in LAX [ In reply to ]
It may be worthwhile for you to consider adding 15169 to your "Don't accept
$tier1 prefixes from other peers" policy in your inbound policy chain.

I've found that there's a set of $LARGE_ENOUGH networks that, even though
they're not literal $tier1 providers, benefit from that same level of
filtering. You wouldn't want to try sending Level3 traffic through a
random peer, as the results would likely be catastrophic; so, make use of
that same filter rule in your inbound policy to filter out hearing 15169
prefixes from other peering sessions.

The caveat to that, of course, is that successful failover will mean
carrying traffic across your backbone when your 15169 prefixes in one
location disappear during an outage/maintenance window, so make sure your
backbone is correctly sized to handle those reroute situations. It also
means that multi-homed downstream customers are likely to send less
upstream traffic through you to reach Google.

But that *will* mean that no amount of leaking more specific prefixes
through other paths will unexpectedly cause your traffic to shift.

Matt



On Mon, Mar 2, 2020 at 5:39 PM Seth Mattinen <sethm@rollernet.us> wrote:

> On 3/2/20 4:32 PM, Patrick W. Gilmore wrote:
> > That said, I fear this is going to be a problem long term. A blind “no
> /24s” filter is dangerous, plus it might solve all traffic issues. It is
> going to take effort to be sure you don’t get bitten by the Law Of
> Unintended Consequences.
>
>
> As soon as Google un-freezes new peering requests so I can get a direct
> peering that includes appropriate /24's I've been told offlist I should
> get (instead of the route server subset) I'll happily remove the transit
> filters. But I can only work with what I'm given.
>
>
Re: Google peering in LAX [ In reply to ]
> Your routers, your decision.
>
> But how much traffic are you sending TO Google? Most people get the
> vast majority of traffic FROM Google. They send you videos, you send
> them ACKs. Does it matter where the ACKs go?
>
Lot's of DNS traffic, now.  All of the dns or https and all those
clients pointing to 8.8.8.8.  Google's DNS servers are slow and extra
latency makes it worse.

--
Best Regards
Curtis Maurand
mailto:cmaurand@xyonet.com
Re: Google peering in LAX [ In reply to ]
On Wed, Mar 4, 2020 at 4:52 PM Curtis Maurand <cmaurand@xyonet.com> wrote:
>
>Google's DNS servers are slow and extra
> latency makes it worse.

odd, I don't think that's the intent of those dns servers though...
did you have a pointer/graph/note about how/where/when you see slowness?
there are folk who care about this metric, deeply, and if the service
isn't what
they expect they can toggle robot arms to get better results.