Mailing List Archive

cDOT cluster replication aka peering via specific LIFs
Hi All,

I have two production cDOT clusters out of 8 nodes in two DCs, running
OnTAP 8.3.2.

I ended up having a bit of a weird network topology where 6 nodes (3
pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.

I wonder how can I enforce cluster peering traffic via those 10Gb enabled nodes?

I know that I have to have ICL LIFs on every node in the clusters, and
that's fine, I can do that, but would like to set something like a
preference for ICL traffic to flow via 10Gb enabled LIFs.

Cheers,
Vladimir
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
Simply choose your 10 gb interface for your ICL. You could also put your ICL on a vlan off that 10gb link.

.

> On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com> wrote:
>
> Hi All,
>
> I have two production cDOT clusters out of 8 nodes in two DCs, running
> OnTAP 8.3.2.
>
> I ended up having a bit of a weird network topology where 6 nodes (3
> pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
>
> I wonder how can I enforce cluster peering traffic via those 10Gb enabled nodes?
>
> I know that I have to have ICL LIFs on every node in the clusters, and
> that's fine, I can do that, but would like to set something like a
> preference for ICL traffic to flow via 10Gb enabled LIFs.
>
> Cheers,
> Vladimir
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
A bit more in depth in response to your question about "preference". WIth ICL you have to create a LIF specifically for IC traffic, as opposed to selecting among existing LIFs for that traffic.

.

> On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com> wrote:
>
> Hi All,
>
> I have two production cDOT clusters out of 8 nodes in two DCs, running
> OnTAP 8.3.2.
>
> I ended up having a bit of a weird network topology where 6 nodes (3
> pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
>
> I wonder how can I enforce cluster peering traffic via those 10Gb enabled nodes?
>
> I know that I have to have ICL LIFs on every node in the clusters, and
> that's fine, I can do that, but would like to set something like a
> preference for ICL traffic to flow via 10Gb enabled LIFs.
>
> Cheers,
> Vladimir
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
This is exactly what I did. What this approach doesn't guarantee is to
always stick to 10Gb LIFs SM traffic, therefore there is my initial
question.
On Jun 27, 2016 17:49, "Francis Kim" <fkim@berkcom.com> wrote:

> Simply choose your 10 gb interface for your ICL. You could also put your
> ICL on a vlan off that 10gb link.
>
> .
>
> > On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com> wrote:
> >
> > Hi All,
> >
> > I have two production cDOT clusters out of 8 nodes in two DCs, running
> > OnTAP 8.3.2.
> >
> > I ended up having a bit of a weird network topology where 6 nodes (3
> > pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
> >
> > I wonder how can I enforce cluster peering traffic via those 10Gb
> enabled nodes?
> >
> > I know that I have to have ICL LIFs on every node in the clusters, and
> > that's fine, I can do that, but would like to set something like a
> > preference for ICL traffic to flow via 10Gb enabled LIFs.
> >
> > Cheers,
> > Vladimir
> > _______________________________________________
> > Toasters mailing list
> > Toasters@teaparty.net
> > http://www.teaparty.net/mailman/listinfo/toasters
>
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
On the CLI, start by looking at "net int show -role intercluster" on each
cluster.
Make sure the LIFs for interclusters are on the interfaces you are
expecting.

Next, run "cluster peer show -instance" from each cluster.
-> look at the local and remote interfaces. These should only be
intercluster interfaces (LIFs)

SnapMirror traffic should only flow to/from those interfaces listed.



--tmac

*Tim McCarthy, **Principal Consultant*


On Mon, Jun 27, 2016 at 2:32 PM, Momonth <momonth@gmail.com> wrote:

> This is exactly what I did. What this approach doesn't guarantee is to
> always stick to 10Gb LIFs SM traffic, therefore there is my initial
> question.
> On Jun 27, 2016 17:49, "Francis Kim" <fkim@berkcom.com> wrote:
>
>> Simply choose your 10 gb interface for your ICL. You could also put your
>> ICL on a vlan off that 10gb link.
>>
>> .
>>
>> > On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com> wrote:
>> >
>> > Hi All,
>> >
>> > I have two production cDOT clusters out of 8 nodes in two DCs, running
>> > OnTAP 8.3.2.
>> >
>> > I ended up having a bit of a weird network topology where 6 nodes (3
>> > pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
>> >
>> > I wonder how can I enforce cluster peering traffic via those 10Gb
>> enabled nodes?
>> >
>> > I know that I have to have ICL LIFs on every node in the clusters, and
>> > that's fine, I can do that, but would like to set something like a
>> > preference for ICL traffic to flow via 10Gb enabled LIFs.
>> >
>> > Cheers,
>> > Vladimir
>> > _______________________________________________
>> > Toasters mailing list
>> > Toasters@teaparty.net
>> > http://www.teaparty.net/mailman/listinfo/toasters
>>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
Wonder if your broadcast domains and failover groups should be reviewed.

.

On Jun 27, 2016, at 11:32 AM, Momonth <momonth@gmail.com<mailto:momonth@gmail.com>> wrote:


This is exactly what I did. What this approach doesn't guarantee is to always stick to 10Gb LIFs SM traffic, therefore there is my initial question.

On Jun 27, 2016 17:49, "Francis Kim" <fkim@berkcom.com<mailto:fkim@berkcom.com>> wrote:
Simply choose your 10 gb interface for your ICL. You could also put your ICL on a vlan off that 10gb link.

.

> On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com<mailto:momonth@gmail.com>> wrote:
>
> Hi All,
>
> I have two production cDOT clusters out of 8 nodes in two DCs, running
> OnTAP 8.3.2.
>
> I ended up having a bit of a weird network topology where 6 nodes (3
> pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
>
> I wonder how can I enforce cluster peering traffic via those 10Gb enabled nodes?
>
> I know that I have to have ICL LIFs on every node in the clusters, and
> that's fine, I can do that, but would like to set something like a
> preference for ICL traffic to flow via 10Gb enabled LIFs.
>
> Cheers,
> Vladimir
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net<mailto:Toasters@teaparty.net>
> http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
That is a possibility

--tmac

*Tim McCarthy, **Principal Consultant*


On Mon, Jun 27, 2016 at 3:06 PM, Francis Kim <fkim@berkcom.com> wrote:

> Wonder if your broadcast domains and failover groups should be reviewed.
>
> .
>
> On Jun 27, 2016, at 11:32 AM, Momonth <momonth@gmail.com> wrote:
>
> This is exactly what I did. What this approach doesn't guarantee is to
> always stick to 10Gb LIFs SM traffic, therefore there is my initial
> question.
> On Jun 27, 2016 17:49, "Francis Kim" <fkim@berkcom.com> wrote:
>
>> Simply choose your 10 gb interface for your ICL. You could also put your
>> ICL on a vlan off that 10gb link.
>>
>> .
>>
>> > On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com> wrote:
>> >
>> > Hi All,
>> >
>> > I have two production cDOT clusters out of 8 nodes in two DCs, running
>> > OnTAP 8.3.2.
>> >
>> > I ended up having a bit of a weird network topology where 6 nodes (3
>> > pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
>> >
>> > I wonder how can I enforce cluster peering traffic via those 10Gb
>> enabled nodes?
>> >
>> > I know that I have to have ICL LIFs on every node in the clusters, and
>> > that's fine, I can do that, but would like to set something like a
>> > preference for ICL traffic to flow via 10Gb enabled LIFs.
>> >
>> > Cheers,
>> > Vladimir
>> > _______________________________________________
>> > Toasters mailing list
>> > Toasters@teaparty.net
>> > http://www.teaparty.net/mailman/listinfo/toasters
>>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
Sorry, if I was not clear enough in my first message. I meant to say that
the 6 nodes on 1Gb network is my infrastructure limitation, ie those node
can not be connected to 10Gb network, but only 1Gb.
On Jun 27, 2016 20:50, "tmac" <tmacmd@gmail.com> wrote:

> On the CLI, start by looking at "net int show -role intercluster" on each
> cluster.
> Make sure the LIFs for interclusters are on the interfaces you are
> expecting.
>
> Next, run "cluster peer show -instance" from each cluster.
> -> look at the local and remote interfaces. These should only be
> intercluster interfaces (LIFs)
>
> SnapMirror traffic should only flow to/from those interfaces listed.
>
>
>
> --tmac
>
> *Tim McCarthy, **Principal Consultant*
>
>
> On Mon, Jun 27, 2016 at 2:32 PM, Momonth <momonth@gmail.com> wrote:
>
>> This is exactly what I did. What this approach doesn't guarantee is to
>> always stick to 10Gb LIFs SM traffic, therefore there is my initial
>> question.
>> On Jun 27, 2016 17:49, "Francis Kim" <fkim@berkcom.com> wrote:
>>
>>> Simply choose your 10 gb interface for your ICL. You could also put
>>> your ICL on a vlan off that 10gb link.
>>>
>>> .
>>>
>>> > On Jun 27, 2016, at 7:59 AM, Momonth <momonth@gmail.com> wrote:
>>> >
>>> > Hi All,
>>> >
>>> > I have two production cDOT clusters out of 8 nodes in two DCs, running
>>> > OnTAP 8.3.2.
>>> >
>>> > I ended up having a bit of a weird network topology where 6 nodes (3
>>> > pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled.
>>> >
>>> > I wonder how can I enforce cluster peering traffic via those 10Gb
>>> enabled nodes?
>>> >
>>> > I know that I have to have ICL LIFs on every node in the clusters, and
>>> > that's fine, I can do that, but would like to set something like a
>>> > preference for ICL traffic to flow via 10Gb enabled LIFs.
>>> >
>>> > Cheers,
>>> > Vladimir
>>> > _______________________________________________
>>> > Toasters mailing list
>>> > Toasters@teaparty.net
>>> > http://www.teaparty.net/mailman/listinfo/toasters
>>>
>>
>> _______________________________________________
>> Toasters mailing list
>> Toasters@teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>>
>>
>
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
Momonth> Sorry, if I was not clear enough in my first message. I meant
Momonth> to say that the 6 nodes on 1Gb network is my infrastructure
Momonth> limitation, ie those node can not be connected to 10Gb
Momonth> network, but only 1Gb.

It would be nice if you could show the output of 'show int' for one of
your 10g nodes and your 1g nodes. But I also find it surprising that
you don't have a set of 10g switches for the InterClusterLinks, since
that's the standard for Netapp installs of four nodes or more.

Did you just start with two nodes and just keep adding in FAS2250s or
something? :-)

John
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
On Mon, Jun 27, 2016 at 10:44 PM, John Stoffel <john@stoffel.org> wrote:
> It would be nice if you could show the output of 'show int' for one of
> your 10g nodes and your 1g nodes. But I also find it surprising that
> you don't have a set of 10g switches for the InterClusterLinks, since
> that's the standard for Netapp installs of four nodes or more.
>
> Did you just start with two nodes and just keep adding in FAS2250s or
> something? :-)

We tarted with 8040 x2 nodes, however I've never heard about the 10G
infra for intercluster communications ..

Here is my current setup:

na101::> network interface show -role intercluster -fields
lif,curr-node,curr-port
vserver lif curr-node curr-port
------- ------------------- ------------ ---------
na101 na101node-1a-icl-01 na101node-1a e2a <<-- 1Gb from here
na101 na101node-1b-icl-01 na101node-1b e2a
na101 na101node-2a-icl-01 na101node-2a e2a
na101 na101node-2b-icl-01 na101node-2b e2a
na101 na101node-3a-icl-01 na101node-3a e2a
na101 na101node-3b-icl-01 na101node-3b e0i <<-- 1Gb till here
na101 na101node-4a-icl-02 na101node-4a a0a <<-- 10Gb
na101 na101node-4b-icl-02 na101node-4b a0a <<-- 10Gb
8 entries were displayed.

na101::> cluster peer show
Peer Cluster Name Cluster Serial Number Availability Authentication
------------------------- --------------------- -------------- --------------
na201 XXXXXXXXX Available ok

Like I said it works fine, I just want to make it better =)
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
You confuse cluster interconnect 10G switches and intercluster peering.

I am actually surprised that such simple question is apparently even not understood. Let's try to ask differently - which local and remote LIFs are selected by Data ONTAP when it connects to another cluster? What are rules?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS

FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com
Web: ru.fujitsu.com
Company details: ts.fujitsu.com/imprint
This communication contains information that is confidential, proprietary in nature and/or privileged.  It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

> -----Original Message-----
> From: toasters-bounces@teaparty.net [mailto:toasters-
> bounces@teaparty.net] On Behalf Of John Stoffel
> Sent: Monday, June 27, 2016 11:45 PM
> To: momonth@gmail.com
> Cc: toasters@teaparty.net
> Subject: Re: cDOT cluster replication aka peering via specific LIFs
>
>
> Momonth> Sorry, if I was not clear enough in my first message. I meant
> Momonth> to say that the 6 nodes on 1Gb network is my infrastructure
> Momonth> limitation, ie those node can not be connected to 10Gb network,
> Momonth> but only 1Gb.
>
> It would be nice if you could show the output of 'show int' for one of
> your 10g nodes and your 1g nodes. But I also find it surprising that
> you don't have a set of 10g switches for the InterClusterLinks, since
> that's the standard for Netapp installs of four nodes or more.
>
> Did you just start with two nodes and just keep adding in FAS2250s or
> something? :-)
>
> John
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
On Tue, Jun 28, 2016 at 10:57 AM, andrei.borzenkov@ts.fujitsu.com
<andrei.borzenkov@ts.fujitsu.com> wrote:
> You confuse cluster interconnect 10G switches and intercluster peering.
>

"You" here is me (Vladimir) or John?

> I am actually surprised that such simple question is apparently even not understood. Let's try to ask differently - which local and remote LIFs are selected by Data ONTAP when it connects to another cluster? What are rules?
>

According to the documentation nodes of local and remote clusters
setup full-mesh connectivity, so LIFs are involved on both sides.

Cheers,
Vladimir
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
Intercluster communications are for communicating between clusters.

The Cluster Network (the private 10G switches) are used for Intracluster
activities.
i.e. local communications stay on the cluster network.

So, any communications from Cluster na101 to/from Cluster na201
will always go through the InterCluster LIFs.

Therefore, if the communication is from
na101:node1 to na201:node1 it will be 1Gig
na101:node1 to na201:node2 it will be 1Gig
na101:node1 to na201:node3 it will be 1Gig
na101:node1 to na201:node4 it will be 1Gig
na101:node1 to na201:node5 it will be 1Gig
na101:node1 to na201:node6 it will be 1Gig
na101:node5 to na201:node6 it will be 10Gig

In other words, if your mirror source/destinations are on nodes 5/6 then
you will use 10Gig.
If your mirror source/destinations are on anything but those two nodes, you
will be using 1Gig.

--tmac

*Tim McCarthy, **Principal Consultant*



On Tue, Jun 28, 2016 at 6:03 AM, Momonth <momonth@gmail.com> wrote:

> On Tue, Jun 28, 2016 at 10:57 AM, andrei.borzenkov@ts.fujitsu.com
> <andrei.borzenkov@ts.fujitsu.com> wrote:
> > You confuse cluster interconnect 10G switches and intercluster peering.
> >
>
> "You" here is me (Vladimir) or John?
>
> > I am actually surprised that such simple question is apparently even not
> understood. Let's try to ask differently - which local and remote LIFs are
> selected by Data ONTAP when it connects to another cluster? What are rules?
> >
>
> According to the documentation nodes of local and remote clusters
> setup full-mesh connectivity, so LIFs are involved on both sides.
>
> Cheers,
> Vladimir
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
On Tue, Jun 28, 2016 at 12:32 PM, tmac <tmacmd@gmail.com> wrote:
> Intercluster communications are for communicating between clusters.
>
> The Cluster Network (the private 10G switches) are used for Intracluster
> activities.
> i.e. local communications stay on the cluster network.
>
> So, any communications from Cluster na101 to/from Cluster na201
> will always go through the InterCluster LIFs.
>
> Therefore, if the communication is from
> na101:node1 to na201:node1 it will be 1Gig
> na101:node1 to na201:node2 it will be 1Gig
> na101:node1 to na201:node3 it will be 1Gig
> na101:node1 to na201:node4 it will be 1Gig
> na101:node1 to na201:node5 it will be 1Gig
> na101:node1 to na201:node6 it will be 1Gig
> na101:node5 to na201:node6 it will be 10Gig
>
> In other words, if your mirror source/destinations are on nodes 5/6 then you
> will use 10Gig.
> If your mirror source/destinations are on anything but those two nodes, you
> will be using 1Gig.
>

Awesome, thanks, now the question is:

Is it possible to invoke 10Gig LIFs, available on nodes 5/6, to mirror
volumes on nodes 1/2/3/4?

Cheers,
Vladimir
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
To the best of my knowledge: No.

The full mesh is set up so that all nodes can talk to each other on their
local LIFs.
Your speed will be the slowest link.

refer to my other email (sent privately yesterday). let me know.

--tmac

*Tim McCarthy, **Principal Consultant*


On Tue, Jun 28, 2016 at 6:37 AM, Momonth <momonth@gmail.com> wrote:

> On Tue, Jun 28, 2016 at 12:32 PM, tmac <tmacmd@gmail.com> wrote:
> > Intercluster communications are for communicating between clusters.
> >
> > The Cluster Network (the private 10G switches) are used for Intracluster
> > activities.
> > i.e. local communications stay on the cluster network.
> >
> > So, any communications from Cluster na101 to/from Cluster na201
> > will always go through the InterCluster LIFs.
> >
> > Therefore, if the communication is from
> > na101:node1 to na201:node1 it will be 1Gig
> > na101:node1 to na201:node2 it will be 1Gig
> > na101:node1 to na201:node3 it will be 1Gig
> > na101:node1 to na201:node4 it will be 1Gig
> > na101:node1 to na201:node5 it will be 1Gig
> > na101:node1 to na201:node6 it will be 1Gig
> > na101:node5 to na201:node6 it will be 10Gig
> >
> > In other words, if your mirror source/destinations are on nodes 5/6 then
> you
> > will use 10Gig.
> > If your mirror source/destinations are on anything but those two nodes,
> you
> > will be using 1Gig.
> >
>
> Awesome, thanks, now the question is:
>
> Is it possible to invoke 10Gig LIFs, available on nodes 5/6, to mirror
> volumes on nodes 1/2/3/4?
>
> Cheers,
> Vladimir
>
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
On Tue, Jun 28, 2016 at 12:41 PM, tmac <tmacmd@gmail.com> wrote:
> To the best of my knowledge: No.
>
> The full mesh is set up so that all nodes can talk to each other on their
> local LIFs.
> Your speed will be the slowest link.
>
> refer to my other email (sent privately yesterday). let me know.

OK, thanks.
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
"You" here is John.

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS

FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com
Web: ru.fujitsu.com
Company details: ts.fujitsu.com/imprint
This communication contains information that is confidential, proprietary in nature and/or privileged.  It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.


> -----Original Message-----
> From: vladimir.zhigulin@gmail.com [mailto:vladimir.zhigulin@gmail.com]
> On Behalf Of Momonth
> Sent: Tuesday, June 28, 2016 1:03 PM
> To: Borzenkov, Andrei
> Cc: John Stoffel; toasters@teaparty.net
> Subject: Re: cDOT cluster replication aka peering via specific LIFs
>
> On Tue, Jun 28, 2016 at 10:57 AM, andrei.borzenkov@ts.fujitsu.com
> <andrei.borzenkov@ts.fujitsu.com> wrote:
> > You confuse cluster interconnect 10G switches and intercluster
> peering.
> >
>
> "You" here is me (Vladimir) or John?
>
> > I am actually surprised that such simple question is apparently even
> not understood. Let's try to ask differently - which local and remote
> LIFs are selected by Data ONTAP when it connects to another cluster?
> What are rules?
> >
>
> According to the documentation nodes of local and remote clusters setup
> full-mesh connectivity, so LIFs are involved on both sides.
>
> Cheers,
> Vladimir

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
andrei> You confuse cluster interconnect 10G switches and intercluster peering.

Yup, I did.

andrei> I am actually surprised that such simple question is
andrei> apparently even not understood. Let's try to ask differently -
andrei> which local and remote LIFs are selected by Data ONTAP when it
andrei> connects to another cluster? What are rules?

The original question was a bit lacking on details... and the
distinction of IntERcluster and IntRAcluster can be hard when you are
-ENOCAFFEINE in the morning.

*grin*
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
>
> Awesome, thanks, now the question is:
>
> Is it possible to invoke 10Gig LIFs, available on nodes 5/6, to mirror
> volumes on nodes 1/2/3/4?
>
[Borzenkov, Andrei]
As I am on training anyway, I tried what happens if I use only partial intercluster connectivity. The result is

- I can in principle establish peer relationship if not all nodes have intercluster LIFs, but this relationship is shown as "partial" then.

- snapmirror does not work in this case. I.e. if I have node1 with intercluster LIF and node2 with aggregate and volume, attempt to update snapmirror simply times out.

- that happens also if I initially have full mesh and later remove one intercluster LIF from configuration (also modifying cluster peer relationship)

So it looks like connection always happens to node that owns volume and answer to your question is "no". This is with 8.3.2P2.


_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
Thanks Andrei, I appreciate your effort, and I actually concluded the
same on 8.2.3.

On Wed, Jul 13, 2016 at 10:12 AM, andrei.borzenkov@ts.fujitsu.com
<andrei.borzenkov@ts.fujitsu.com> wrote:
> As I am on training anyway, I tried what happens if I use only partial intercluster connectivity. The result is
>
> - I can in principle establish peer relationship if not all nodes have intercluster LIFs, but this relationship is shown as "partial" then.
>
> - snapmirror does not work in this case. I.e. if I have node1 with intercluster LIF and node2 with aggregate and volume, attempt to update snapmirror simply times out.
>
> - that happens also if I initially have full mesh and later remove one intercluster LIF from configuration (also modifying cluster peer relationship)
>
> So it looks like connection always happens to node that owns volume and answer to your question is "no". This is with 8.3.2P2.
>
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: cDOT cluster replication aka peering via specific LIFs [ In reply to ]
That makes sense. Mirroring is a node to node operation.
If it cannot communicate directly between the nodes on the intercluster
interfaces, it will fail.



--tmac

*Tim McCarthy, **Principal Consultant*

*Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>*

*I Blog at TMACsRack <https://tmacsrack.wordpress.com/>*



On Wed, Jul 13, 2016 at 7:18 AM, Momonth <momonth@gmail.com> wrote:

> Thanks Andrei, I appreciate your effort, and I actually concluded the
> same on 8.2.3.
>
> On Wed, Jul 13, 2016 at 10:12 AM, andrei.borzenkov@ts.fujitsu.com
> <andrei.borzenkov@ts.fujitsu.com> wrote:
> > As I am on training anyway, I tried what happens if I use only partial
> intercluster connectivity. The result is
> >
> > - I can in principle establish peer relationship if not all nodes have
> intercluster LIFs, but this relationship is shown as "partial" then.
> >
> > - snapmirror does not work in this case. I.e. if I have node1 with
> intercluster LIF and node2 with aggregate and volume, attempt to update
> snapmirror simply times out.
> >
> > - that happens also if I initially have full mesh and later remove one
> intercluster LIF from configuration (also modifying cluster peer
> relationship)
> >
> > So it looks like connection always happens to node that owns volume and
> answer to your question is "no". This is with 8.3.2P2.
> >
>