Mailing List Archive

startup problem DLM on ubuntu lucid
Hi,

when rebooting my cluster nodes they won't bring up the ocfs2-fs because of
resDLM failing. When I issue a '/etc/init.d/pacemaker restart' afterwards
everything is fine.

The machine needs quite a while to bring up the (bonding) network interfaces.
Do timeout values need to be adjusted? Or should I rather try to startup
pacemaker after the network is completely up?


my current config:

node server-c \
attributes standby="off"
node server-d
primitive failover-ip ocf:heartbeat:IPaddr \
params ip="192.168.5.150" \
op monitor interval="10s"
primitive resDLM ocf:pacemaker:controld \
op monitor interval="120s"
primitive resFS ocf:heartbeat:Filesystem \
params device="/dev/mapper/data-data" directory="/srv/data"
fstype="ocfs2" \
op monitor interval="120s"
primitive resO2CB ocf:pacemaker:o2cb \
op monitor interval="120s"
clone cloneDLM resDLM \
meta globally-unique="false" interleave="true"
clone cloneFS resFS \
meta interleave="true" ordered="true"
clone cloneO2CB resO2CB \
meta globally-unique="false" interleave="true"
colocation colFSO2CB inf: cloneFS cloneO2CB
colocation colO2CBDLM inf: cloneO2CB cloneDLM
order ordDLMO2CB 0: cloneDLM cloneO2CB
order ordO2CBFS 0: cloneO2CB cloneFS
property $id="cib-bootstrap-options" \
dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false" \
last-lrm-refresh="1272026744"


I tried something like
primitive resDLM ocf:pacemaker:controld \
op start timeout="100s" \
op monitor interval="120s"
but this didn't help.





TIA,
Oliver
Re: startup problem DLM on ubuntu lucid [ In reply to ]
Can you check your dmesg to see if DLM is segfaulting? I might be
experiencing the same problem. If corosync is started at boot DLM segfaults,
but if it's started manually everything is ok. Still trying to find out more
about what is going on, and I sadly can't provide more information before
Monday when I get to work. We did even try bootchart to see if that could
provide some more information, but sadly no. We also changed the start order
to corosync by renaming the init symlink to S98corosync, but that didn't
work out either.

On Sat, Apr 24, 2010 at 12:25 PM, Oliver Heinz <oheinz@fbihome.de> wrote:

> Hi,
>
> when rebooting my cluster nodes they won't bring up the ocfs2-fs because of
> resDLM failing. When I issue a '/etc/init.d/pacemaker restart' afterwards
> everything is fine.
>
> The machine needs quite a while to bring up the (bonding) network
> interfaces.
> Do timeout values need to be adjusted? Or should I rather try to startup
> pacemaker after the network is completely up?
>
>
> my current config:
>
> node server-c \
> attributes standby="off"
> node server-d
> primitive failover-ip ocf:heartbeat:IPaddr \
> params ip="192.168.5.150" \
> op monitor interval="10s"
> primitive resDLM ocf:pacemaker:controld \
> op monitor interval="120s"
> primitive resFS ocf:heartbeat:Filesystem \
> params device="/dev/mapper/data-data" directory="/srv/data"
> fstype="ocfs2" \
> op monitor interval="120s"
> primitive resO2CB ocf:pacemaker:o2cb \
> op monitor interval="120s"
> clone cloneDLM resDLM \
> meta globally-unique="false" interleave="true"
> clone cloneFS resFS \
> meta interleave="true" ordered="true"
> clone cloneO2CB resO2CB \
> meta globally-unique="false" interleave="true"
> colocation colFSO2CB inf: cloneFS cloneO2CB
> colocation colO2CBDLM inf: cloneO2CB cloneDLM
> order ordDLMO2CB 0: cloneDLM cloneO2CB
> order ordO2CBFS 0: cloneO2CB cloneFS
> property $id="cib-bootstrap-options" \
> dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="2" \
> stonith-enabled="false" \
> last-lrm-refresh="1272026744"
>
>
> I tried something like
> primitive resDLM ocf:pacemaker:controld \
> op start timeout="100s" \
> op monitor interval="120s"
> but this didn't help.
>
>
>
>
>
> TIA,
> Oliver
>
>
>
>
>
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
Re: startup problem DLM on ubuntu lucid [ In reply to ]
Am Samstag, 24. April 2010, um 17:27:42 schrieb Pål Simensen:
> Can you check your dmesg to see if DLM is segfaulting? I might be
> experiencing the same problem. If corosync is started at boot DLM
> segfaults, but if it's started manually everything is ok. Still trying to
> find out more about what is going on, and I sadly can't provide more
> information before Monday when I get to work. We did even try bootchart to
> see if that could provide some more information, but sadly no. We also
> changed the start order to corosync by renaming the init symlink to
> S98corosync, but that didn't work out either.


You are right, dlm is segfaulting and network is already up at that time.

[ 15.654093] br53: port 1(vlan53) entering forwarding state
[ 15.664083] br83: port 1(vlan83) entering forwarding state
...
[ 46.979087] dlm_controld.pc[2533]: segfault at 0 ip 00007f30f7d68022 sp
00007fffddf0e288 error 4 in libc-2.11.1.so[7f30f7ce5000+178000]

I rebuild the packages http://ppa.launchpad.net/ubuntu-ha/lucid-
cluster/ubuntu/pool/main/r/redhat-cluster on a freshly installed lucid VM but
this didn't change anything. I even upgraded them to current 3.0.11 still
segfaulting. So try and error seems not to work. Maybe someone with a little
more understanding what's going on can do an educated guess?

TIA,
Oliver


>
> On Sat, Apr 24, 2010 at 12:25 PM, Oliver Heinz <oheinz@fbihome.de> wrote:
> > Hi,
> >
> > when rebooting my cluster nodes they won't bring up the ocfs2-fs because
> > of resDLM failing. When I issue a '/etc/init.d/pacemaker restart'
> > afterwards everything is fine.
> >
> > The machine needs quite a while to bring up the (bonding) network
> > interfaces.
> > Do timeout values need to be adjusted? Or should I rather try to startup
> > pacemaker after the network is completely up?
> >
> >
> > my current config:
> >
> > node server-c \
> >
> > attributes standby="off"
> >
> > node server-d
> > primitive failover-ip ocf:heartbeat:IPaddr \
> >
> > params ip="192.168.5.150" \
> > op monitor interval="10s"
> >
> > primitive resDLM ocf:pacemaker:controld \
> >
> > op monitor interval="120s"
> >
> > primitive resFS ocf:heartbeat:Filesystem \
> >
> > params device="/dev/mapper/data-data" directory="/srv/data"
> >
> > fstype="ocfs2" \
> >
> > op monitor interval="120s"
> >
> > primitive resO2CB ocf:pacemaker:o2cb \
> >
> > op monitor interval="120s"
> >
> > clone cloneDLM resDLM \
> >
> > meta globally-unique="false" interleave="true"
> >
> > clone cloneFS resFS \
> >
> > meta interleave="true" ordered="true"
> >
> > clone cloneO2CB resO2CB \
> >
> > meta globally-unique="false" interleave="true"
> >
> > colocation colFSO2CB inf: cloneFS cloneO2CB
> > colocation colO2CBDLM inf: cloneO2CB cloneDLM
> > order ordDLMO2CB 0: cloneDLM cloneO2CB
> > order ordO2CBFS 0: cloneO2CB cloneFS
> > property $id="cib-bootstrap-options" \
> >
> > dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
> > cluster-infrastructure="openais" \
> > expected-quorum-votes="2" \
> > stonith-enabled="false" \
> > last-lrm-refresh="1272026744"
> >
> > I tried something like
> > primitive resDLM ocf:pacemaker:controld \
> >
> > op start timeout="100s" \
> > op monitor interval="120s"
> >
> > but this didn't help.
> >
> >
> >
> >
> >
> > TIA,
> > Oliver
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> >
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: startup problem DLM on ubuntu lucid [ In reply to ]
What versions of pacemaker and the dlm?
What does the stack trace from the core look like?

On Sun, Apr 25, 2010 at 1:15 PM, Oliver Heinz <oheinz@fbihome.de> wrote:
> Am Samstag, 24. April 2010, um 17:27:42 schrieb Pål Simensen:
>> Can you check your dmesg to see if DLM is segfaulting? I might be
>> experiencing the same problem. If corosync is started at boot DLM
>> segfaults, but if it's started manually everything is ok. Still trying to
>> find out more about what is going on, and I sadly can't provide more
>> information before Monday when I get to work. We did even try bootchart to
>> see if that could provide some more information, but sadly no. We also
>> changed the start order to corosync by renaming the init symlink to
>> S98corosync, but that didn't work out either.
>
>
> You are right, dlm is segfaulting and network is already up at that time.
>
> [   15.654093] br53: port 1(vlan53) entering forwarding state
> [   15.664083] br83: port 1(vlan83) entering forwarding state
> ...
> [   46.979087] dlm_controld.pc[2533]: segfault at 0 ip 00007f30f7d68022 sp
> 00007fffddf0e288 error 4 in libc-2.11.1.so[7f30f7ce5000+178000]
>
> I rebuild the packages http://ppa.launchpad.net/ubuntu-ha/lucid-
> cluster/ubuntu/pool/main/r/redhat-cluster on a freshly installed lucid VM but
> this didn't change anything. I even upgraded them to current 3.0.11 still
> segfaulting. So try and error seems not to work. Maybe someone with a little
> more understanding what's going on can do an educated guess?
>
> TIA,
> Oliver
>
>
>>
>> On Sat, Apr 24, 2010 at 12:25 PM, Oliver Heinz <oheinz@fbihome.de> wrote:
>> > Hi,
>> >
>> > when rebooting my cluster nodes they won't bring up the ocfs2-fs because
>> > of resDLM failing. When I issue a '/etc/init.d/pacemaker restart'
>> > afterwards everything is fine.
>> >
>> > The machine needs quite a while to bring up the (bonding) network
>> > interfaces.
>> > Do timeout values need to be adjusted? Or should I rather try to startup
>> > pacemaker after the network is completely up?
>> >
>> >
>> > my current config:
>> >
>> > node server-c \
>> >
>> >        attributes standby="off"
>> >
>> > node server-d
>> > primitive failover-ip ocf:heartbeat:IPaddr \
>> >
>> >        params ip="192.168.5.150" \
>> >        op monitor interval="10s"
>> >
>> > primitive resDLM ocf:pacemaker:controld \
>> >
>> >        op monitor interval="120s"
>> >
>> > primitive resFS ocf:heartbeat:Filesystem \
>> >
>> >        params device="/dev/mapper/data-data" directory="/srv/data"
>> >
>> > fstype="ocfs2" \
>> >
>> >        op monitor interval="120s"
>> >
>> > primitive resO2CB ocf:pacemaker:o2cb \
>> >
>> >        op monitor interval="120s"
>> >
>> > clone cloneDLM resDLM \
>> >
>> >        meta globally-unique="false" interleave="true"
>> >
>> > clone cloneFS resFS \
>> >
>> >        meta interleave="true" ordered="true"
>> >
>> > clone cloneO2CB resO2CB \
>> >
>> >        meta globally-unique="false" interleave="true"
>> >
>> > colocation colFSO2CB inf: cloneFS cloneO2CB
>> > colocation colO2CBDLM inf: cloneO2CB cloneDLM
>> > order ordDLMO2CB 0: cloneDLM cloneO2CB
>> > order ordO2CBFS 0: cloneO2CB cloneFS
>> > property $id="cib-bootstrap-options" \
>> >
>> >        dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
>> >        cluster-infrastructure="openais" \
>> >        expected-quorum-votes="2" \
>> >        stonith-enabled="false" \
>> >        last-lrm-refresh="1272026744"
>> >
>> > I tried something like
>> > primitive resDLM ocf:pacemaker:controld \
>> >
>> >        op start timeout="100s" \
>> >        op monitor interval="120s"
>> >
>> > but this didn't help.
>> >
>> >
>> >
>> >
>> >
>> > TIA,
>> > Oliver
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>> >
>> > Project Home: http://www.clusterlabs.org
>> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: startup problem DLM on ubuntu lucid [ In reply to ]
Am Montag, 26. April 2010 08:35:53 schrieb Andrew Beekhof:
> What versions of pacemaker and the dlm?
> What does the stack trace from the core look like?

I reinstalled the packages from http://ppa.launchpad.net/ubuntu-ha/lucid-
cluster

that's 3.0.7 for the dlm
and 1.0.8+hg15494 for pacemaker

report: http://users.fbihome.de/~oheinz/ha-cluster/report_1.tar.bz2
core-file: http://users.fbihome.de/~oheinz/ha-cluster/core.2606.bz2

I cc:ed the ubuntu-ha list as it might be packaging related.

TIA,
Oliver



>
> On Sun, Apr 25, 2010 at 1:15 PM, Oliver Heinz <oheinz@fbihome.de> wrote:
> > Am Samstag, 24. April 2010, um 17:27:42 schrieb Pål Simensen:
> >> Can you check your dmesg to see if DLM is segfaulting? I might be
> >> experiencing the same problem. If corosync is started at boot DLM
> >> segfaults, but if it's started manually everything is ok. Still trying
> >> to find out more about what is going on, and I sadly can't provide more
> >> information before Monday when I get to work. We did even try bootchart
> >> to see if that could provide some more information, but sadly no. We
> >> also changed the start order to corosync by renaming the init symlink
> >> to S98corosync, but that didn't work out either.
> >
> > You are right, dlm is segfaulting and network is already up at that time.
> >
> > [ 15.654093] br53: port 1(vlan53) entering forwarding state
> > [ 15.664083] br83: port 1(vlan83) entering forwarding state
> > ...
> > [ 46.979087] dlm_controld.pc[2533]: segfault at 0 ip 00007f30f7d68022
> > sp 00007fffddf0e288 error 4 in libc-2.11.1.so[7f30f7ce5000+178000]
> >
> > I rebuild the packages http://ppa.launchpad.net/ubuntu-ha/lucid-
> > cluster/ubuntu/pool/main/r/redhat-cluster on a freshly installed lucid VM
> > but this didn't change anything. I even upgraded them to current 3.0.11
> > still segfaulting. So try and error seems not to work. Maybe someone
> > with a little more understanding what's going on can do an educated
> > guess?
> >
> > TIA,
> > Oliver
> >
> >> On Sat, Apr 24, 2010 at 12:25 PM, Oliver Heinz <oheinz@fbihome.de> wrote:
> >> > Hi,
> >> >
> >> > when rebooting my cluster nodes they won't bring up the ocfs2-fs
> >> > because of resDLM failing. When I issue a '/etc/init.d/pacemaker
> >> > restart' afterwards everything is fine.
> >> >
> >> > The machine needs quite a while to bring up the (bonding) network
> >> > interfaces.
> >> > Do timeout values need to be adjusted? Or should I rather try to
> >> > startup pacemaker after the network is completely up?
> >> >
> >> >
> >> > my current config:
> >> >
> >> > node server-c \
> >> >
> >> > attributes standby="off"
> >> >
> >> > node server-d
> >> > primitive failover-ip ocf:heartbeat:IPaddr \
> >> >
> >> > params ip="192.168.5.150" \
> >> > op monitor interval="10s"
> >> >
> >> > primitive resDLM ocf:pacemaker:controld \
> >> >
> >> > op monitor interval="120s"
> >> >
> >> > primitive resFS ocf:heartbeat:Filesystem \
> >> >
> >> > params device="/dev/mapper/data-data" directory="/srv/data"
> >> >
> >> > fstype="ocfs2" \
> >> >
> >> > op monitor interval="120s"
> >> >
> >> > primitive resO2CB ocf:pacemaker:o2cb \
> >> >
> >> > op monitor interval="120s"
> >> >
> >> > clone cloneDLM resDLM \
> >> >
> >> > meta globally-unique="false" interleave="true"
> >> >
> >> > clone cloneFS resFS \
> >> >
> >> > meta interleave="true" ordered="true"
> >> >
> >> > clone cloneO2CB resO2CB \
> >> >
> >> > meta globally-unique="false" interleave="true"
> >> >
> >> > colocation colFSO2CB inf: cloneFS cloneO2CB
> >> > colocation colO2CBDLM inf: cloneO2CB cloneDLM
> >> > order ordDLMO2CB 0: cloneDLM cloneO2CB
> >> > order ordO2CBFS 0: cloneO2CB cloneFS
> >> > property $id="cib-bootstrap-options" \
> >> >
> >> > dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
> >> > cluster-infrastructure="openais" \
> >> > expected-quorum-votes="2" \
> >> > stonith-enabled="false" \
> >> > last-lrm-refresh="1272026744"
> >> >
> >> > I tried something like
> >> > primitive resDLM ocf:pacemaker:controld \
> >> >
> >> > op start timeout="100s" \
> >> > op monitor interval="120s"
> >> >
> >> > but this didn't help.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > TIA,
> >> > Oliver
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> >> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> >> >
> >> > Project Home: http://www.clusterlabs.org
> >> > Getting started:
> >> > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> >
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> >
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf




_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: startup problem DLM on ubuntu lucid [ In reply to ]
On Mon, Apr 26, 2010 at 11:17:26AM +0200, Oliver Heinz wrote:
> Am Montag, 26. April 2010 08:35:53 schrieb Andrew Beekhof:
> > What versions of pacemaker and the dlm?
> > What does the stack trace from the core look like?
>
> I reinstalled the packages from http://ppa.launchpad.net/ubuntu-ha/lucid-
> cluster
>
> that's 3.0.7 for the dlm
> and 1.0.8+hg15494 for pacemaker
>
> report: http://users.fbihome.de/~oheinz/ha-cluster/report_1.tar.bz2
> core-file: http://users.fbihome.de/~oheinz/ha-cluster/core.2606.bz2

Note that
http://ppa.launchpad.net/ubuntu-ha/lucid-cluster/ubuntu/pool/main/c/cluster-glue/cluster-glue_1.0.3+hg2366.orig.tar.gz

Unfortunately contains the binary incompatibility we reverted for glue 1.0.5
details:
https://bugs.launchpad.net/ubuntu/+source/cluster-glue/+bug/570096

It is very likely that this is the reason for those segfaults.

--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: startup problem DLM on ubuntu lucid [ In reply to ]
Am Montag, 26. April 2010 11:46:43 schrieb Lars Ellenberg:
...
> Note that
> http://ppa.launchpad.net/ubuntu-ha/lucid-cluster/ubuntu/pool/main/c/cluster
> -glue/cluster-glue_1.0.3+hg2366.orig.tar.gz
>
> Unfortunately contains the binary incompatibility we reverted for glue
> 1.0.5 details:
> https://bugs.launchpad.net/ubuntu/+source/cluster-glue/+bug/570096
>
> It is very likely that this is the reason for those segfaults.


Thanks for that information. I rebuild the complete stack with cluster-glue
1.0.5 (which made it to the ppa repository an hour ago). But dlm_controld.pcmk
is still segfaulting :-( But I probably missed something when rebuilding.

I will wait for the "official" ubuntu stack rebuild and report afterwards if
this fixed the issue.

Thanks so far,
Oliver




_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 26.04.2010 14:42, Oliver Heinz wrote:

> Thanks for that information. I rebuild the complete stack with cluster-glue
> 1.0.5 (which made it to the ppa repository an hour ago). But dlm_controld.pcmk
> is still segfaulting :-( But I probably missed something when rebuilding.

Have you rebuilded redhat cluster suite? dlm comes from that source.

> I will wait for the "official" ubuntu stack rebuild and report afterwards if
> this fixed the issue.

Try now.

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Montag, 26. April 2010 15:58:51 schrieb Ante Karamatić:
> On 26.04.2010 14:42, Oliver Heinz wrote:
> > Thanks for that information. I rebuild the complete stack with
> > cluster-glue 1.0.5 (which made it to the ppa repository an hour ago).
> > But dlm_controld.pcmk is still segfaulting :-( But I probably missed
> > something when rebuilding.
>
> Have you rebuilded redhat cluster suite? dlm comes from that source.

I rebuilded heartbeat, pacemaker and redhat-cluster-suite and updated the
corresponding pakages before building the next but it still segfaults. Even
rebuild the current debian redhat-cluster-suite (3.0.9) and integrated the
pacemaker-pakages, still segfaulting :-(

>
> > I will wait for the "official" ubuntu stack rebuild and report afterwards
> > if this fixed the issue.
>
> Try now.

Still segfaults. Updated coredump and reports are here:
http://users.fbihome.de/~oheinz/ha-cluster/core.2563.bz2
http://users.fbihome.de/~oheinz/ha-cluster/report_2.tar.bz2

Thanks so far,
Oliver

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On Mon, Apr 26, 2010 at 04:37:37PM +0200, Oliver Heinz wrote:
> Am Montag, 26. April 2010 15:58:51 schrieb Ante Karamatić:
> > On 26.04.2010 14:42, Oliver Heinz wrote:
> > > Thanks for that information. I rebuild the complete stack with
> > > cluster-glue 1.0.5 (which made it to the ppa repository an hour ago).
> > > But dlm_controld.pcmk is still segfaulting :-( But I probably missed
> > > something when rebuilding.
> >
> > Have you rebuilded redhat cluster suite? dlm comes from that source.
>
> I rebuilded heartbeat, pacemaker and redhat-cluster-suite and updated the
> corresponding pakages before building the next but it still segfaults. Even
> rebuild the current debian redhat-cluster-suite (3.0.9) and integrated the
> pacemaker-pakages, still segfaulting :-(
>
> >
> > > I will wait for the "official" ubuntu stack rebuild and report afterwards
> > > if this fixed the issue.
> >
> > Try now.
>
> Still segfaults. Updated coredump and reports are here:
> http://users.fbihome.de/~oheinz/ha-cluster/core.2563.bz2
> http://users.fbihome.de/~oheinz/ha-cluster/report_2.tar.bz2

can you please do a
gdb --core core.2563 -ex where -ex quit
(or better, if you know how to pilot gdb).

For it to be useful in any way, you may need to have matching debug
symbols of quite a few libraries around, probably including those of libc.
(that's those -dbg packages in debian).

Thanks,

--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 26.04.2010 16:37, Oliver Heinz wrote:

> Still segfaults.

Urgh... I can not reproduce this. I'm using virtualized 64bit KVM
machines with ppa:ubuntu-ha/lucid-cluster and I get no segfaults. Your
cib is almost identical as mine (I have no IP failover, but I do have drbd).

Could you add relevant part of /var/log/syslog?

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Montag, 26. April 2010, um 18:50:24 schrieb Lars Ellenberg:
> On Mon, Apr 26, 2010 at 04:37:37PM +0200, Oliver Heinz wrote:
> > Am Montag, 26. April 2010 15:58:51 schrieb Ante Karamatić:
> > > On 26.04.2010 14:42, Oliver Heinz wrote:
> > > > Thanks for that information. I rebuild the complete stack with
> > > > cluster-glue 1.0.5 (which made it to the ppa repository an hour ago).
> > > > But dlm_controld.pcmk is still segfaulting :-( But I probably missed
> > > > something when rebuilding.
> > >
> > > Have you rebuilded redhat cluster suite? dlm comes from that source.
> >
> > I rebuilded heartbeat, pacemaker and redhat-cluster-suite and updated the
> > corresponding pakages before building the next but it still segfaults.
> > Even rebuild the current debian redhat-cluster-suite (3.0.9) and
> > integrated the pacemaker-pakages, still segfaulting :-(
> >
> > > > I will wait for the "official" ubuntu stack rebuild and report
> > > > afterwards if this fixed the issue.
> > >
> > > Try now.
> >
> > Still segfaults. Updated coredump and reports are here:
> > http://users.fbihome.de/~oheinz/ha-cluster/core.2563.bz2
> > http://users.fbihome.de/~oheinz/ha-cluster/report_2.tar.bz2
>
> can you please do a
> gdb --core core.2563 -ex where -ex quit
> (or better, if you know how to pilot gdb).

I attached the output. If you need s.th. else you need to instruct me, i have
no experience with gdb, sorry.

> For it to be useful in any way, you may need to have matching debug
> symbols of quite a few libraries around, probably including those of libc.
> (that's those -dbg packages in debian).

I installed every -dbg package that is available for any package installed on
the system (just to be sure). There are no debug packages for most of the
cluster-stack, if it would help to have them recompiled without symbol
stripping I could do that.

TIA,
Oliver
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Dienstag, 27. April 2010, um 09:15:58 schrieb Ante Karamatić:
> On 26.04.2010 16:37, Oliver Heinz wrote:
> > Still segfaults.
>
> Urgh... I can not reproduce this. I'm using virtualized 64bit KVM
> machines with ppa:ubuntu-ha/lucid-cluster and I get no segfaults. Your
> cib is almost identical as mine (I have no IP failover, but I do have
> drbd).

So I'd better do some KVMs for testing just to rule out that I messed up the
system while installing/reinstalling all those packages. And it's probably
easier to share the KVM-Images than sending Servers.

PÃ¥l Simensen reported that he has the same segfault error, so it would be
interessting if the new packages fixed it for him. Did they?

>
> Could you add relevant part of /var/log/syslog?

http://users.fbihome.de/~oheinz/ha-cluster/syslog.txt


Oliver

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 27.04.2010 09:41, Oliver Heinz wrote:

> PÃ¥l Simensen reported that he has the same segfault error, so it would be
> interessting if the new packages fixed it for him. Did they?

We talked yesterday and it seems that segfaulting happens only on one
machine, but I'm sure he'll have more details...


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 27.04.2010 09:41, Oliver Heinz wrote:

> PÃ¥l Simensen reported that he has the same segfault error, so it would be
> interessting if the new packages fixed it for him. Did they?

What's your network setup? Do you use bonding or dhcp?


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Dienstag, 27. April 2010 12:44:57 schrieb Ante Karamatić:
> On 27.04.2010 09:41, Oliver Heinz wrote:
> > PÃ¥l Simensen reported that he has the same segfault error, so it would
> > be
> >
> > interessting if the new packages fixed it for him. Did they?
>
> What's your network setup? Do you use bonding or dhcp?

I use bonding, vlans and bridge interfaces. No dhcp just fixed adresses.

According to syslog the bridge interface is in state forwarding before
dlm_controld segfaults.

That's what the networking interfaces look like:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master
bond0 state UP qlen 1000
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master
bond0 state UP qlen 1000
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master
bond0 state UP qlen 1000
5: eth3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master
bond0 state UP qlen 1000
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue
state UP
inet6 fe80::225:b3ff:fee7:97de/64 scope link
valid_lft forever preferred_lft forever
7: vlan53@bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP
inet6 fe80::225:b3ff:fee7:97de/64 scope link
valid_lft forever preferred_lft forever
8: br53: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
inet 192.168.5.103/24 brd 192.168.5.255 scope global br53
inet 192.168.5.150/24 brd 192.168.5.255 scope global secondary br53:0
inet6 fe80::225:b3ff:fee7:97de/64 scope link
valid_lft forever preferred_lft forever
9: vlan83@bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP
inet6 fe80::225:b3ff:fee7:97de/64 scope link
valid_lft forever preferred_lft forever
10: br83: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
inet 192.168.7.103/24 brd 192.168.7.255 scope global br83
inet6 fe80::225:b3ff:fee7:97de/64 scope link
valid_lft forever preferred_lft forever


Oliver

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On Tue, Apr 27, 2010 at 09:33:27AM +0200, Oliver Heinz wrote:
> Am Montag, 26. April 2010, um 18:50:24 schrieb Lars Ellenberg:
> > On Mon, Apr 26, 2010 at 04:37:37PM +0200, Oliver Heinz wrote:
> > > Am Montag, 26. April 2010 15:58:51 schrieb Ante Karamatić:
> > > > On 26.04.2010 14:42, Oliver Heinz wrote:
> > > > > Thanks for that information. I rebuild the complete stack with
> > > > > cluster-glue 1.0.5 (which made it to the ppa repository an hour ago).
> > > > > But dlm_controld.pcmk is still segfaulting :-( But I probably missed
> > > > > something when rebuilding.
> > > >
> > > > Have you rebuilded redhat cluster suite? dlm comes from that source.
> > >
> > > I rebuilded heartbeat, pacemaker and redhat-cluster-suite and updated the
> > > corresponding pakages before building the next but it still segfaults.
> > > Even rebuild the current debian redhat-cluster-suite (3.0.9) and
> > > integrated the pacemaker-pakages, still segfaulting :-(
> > >
> > > > > I will wait for the "official" ubuntu stack rebuild and report
> > > > > afterwards if this fixed the issue.
> > > >
> > > > Try now.
> > >
> > > Still segfaults. Updated coredump and reports are here:
> > > http://users.fbihome.de/~oheinz/ha-cluster/core.2563.bz2
> > > http://users.fbihome.de/~oheinz/ha-cluster/report_2.tar.bz2
> >
> > can you please do a
> > gdb --core core.2563 -ex where -ex quit
> > (or better, if you know how to pilot gdb).
>
> I attached the output. If you need s.th. else you need to instruct me, i have
> no experience with gdb, sorry.
>
> > For it to be useful in any way, you may need to have matching debug
> > symbols of quite a few libraries around, probably including those of libc.
> > (that's those -dbg packages in debian).
>
> I installed every -dbg package that is available for any package installed on
> the system (just to be sure). There are no debug packages for most of the
> cluster-stack, if it would help to have them recompiled without symbol
> stripping I could do that.

Yes please ;-)

then,
gdb /usr/sbin/dlm_controld.pcmk core -ex where -ex quit

--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 27.04.2010 12:58, Oliver Heinz wrote:

> I use bonding, vlans and bridge interfaces. No dhcp just fixed adresses.

Other two people with this issue also use bonding and after disabling
bonding issue was gone. Could you please verify this?

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 27.04.2010 12:58, Oliver Heinz wrote:

> I use bonding, vlans and bridge interfaces. No dhcp just fixed adresses.

As a workaround, you could:

sudo update-rc.d -f corosync disable S

add 'post-up /etc/init.d/corosync start' to bonding interface in
/etc/network/interfaces.

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Dienstag, 27. April 2010 13:01:36 schrieb Ante Karamatić:
> On 27.04.2010 12:58, Oliver Heinz wrote:
> > I use bonding, vlans and bridge interfaces. No dhcp just fixed adresses.
>
> Other two people with this issue also use bonding and after disabling
> bonding issue was gone. Could you please verify this?

bonding disabled, vlan enabled and bridge enabled -> segfaults on both nodes
bonding disabled, vlan enabled and bridge disabled -> segfaults on both nodes
bonding disabled, vlan disabled and bridge enabled -> segfaults on both nodes

bonding enabled, vlan enabled and bridge enabled -> segfaults on both nodes
bonding enabled, vlan enabled and bridge disabled -> segfaults on both nodes
bonding enabled, vlan disabled and bridge disabled -> segfaults on both nodes

bonding disabled, vlan disabled and bridge disabled (aka plain interface)->
works :-)

bad news is: it's not only bonding related
good news is: vlan and bridging are easier to simulate in a VM I guess


Oliver

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 4/27/2010 3:26 PM, Oliver Heinz wrote:
> Am Dienstag, 27. April 2010 13:01:36 schrieb Ante Karamatić:
>> On 27.04.2010 12:58, Oliver Heinz wrote:
>>> I use bonding, vlans and bridge interfaces. No dhcp just fixed adresses.
>>
>> Other two people with this issue also use bonding and after disabling
>> bonding issue was gone. Could you please verify this?
>
> bonding disabled, vlan enabled and bridge enabled -> segfaults on both nodes
> bonding disabled, vlan enabled and bridge disabled -> segfaults on both nodes
> bonding disabled, vlan disabled and bridge enabled -> segfaults on both nodes
>
> bonding enabled, vlan enabled and bridge enabled -> segfaults on both nodes
> bonding enabled, vlan enabled and bridge disabled -> segfaults on both nodes
> bonding enabled, vlan disabled and bridge disabled -> segfaults on both nodes
>
> bonding disabled, vlan disabled and bridge disabled (aka plain interface)->
> works :-)
>
> bad news is: it's not only bonding related
> good news is: vlan and bridging are easier to simulate in a VM I guess
>

Please collect all those information and file a bug on
bugzilla.redhat.com component pacemaker in fedora/rawhide.

Cheers
Fabio


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Dienstag, 27. April 2010 14:18:27 schrieb Ante Karamatić:
> On 27.04.2010 12:58, Oliver Heinz wrote:
> > I use bonding, vlans and bridge interfaces. No dhcp just fixed adresses.
>
> As a workaround, you could:
>
> sudo update-rc.d -f corosync disable S
>
> add 'post-up /etc/init.d/corosync start' to bonding interface in
> /etc/network/interfaces.

The workaround works :-) But I'm still wondering because according to syslog
network interfaces are completely up before the segfault occurs :-(

I'll try to provide some more helpful gdb output of the core dumps to find the
cause.

Thanks so far,
Oliver





_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Am Dienstag, 27. April 2010 13:00:27 schrieb Lars Ellenberg:
> On Tue, Apr 27, 2010 at 09:33:27AM +0200, Oliver Heinz wrote:
..
> > I installed every -dbg package that is available for any package
> > installed on the system (just to be sure). There are no debug packages
> > for most of the cluster-stack, if it would help to have them recompiled
> > without symbol stripping I could do that.
>
> Yes please ;-)
>
> then,
> gdb /usr/sbin/dlm_controld.pcmk core -ex where -ex quit

Core was generated by `dlm_controld.pcmk -q 0'.
Program terminated with signal 11, Segmentation fault.
#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:31
31 ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory.
in ../sysdeps/x86_64/multiarch/../strlen.S
#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:31
#1 0x00007f4b1ccedd46 in *__GI___strdup (s=0x0) at strdup.c:42
#2 0x0000000000403f0c in dlm_process_node (key=<value optimized out>,
value=0x1e10a30, user_data=0x62a4f8) at /usr/src/packages/redhat-
cluster/3.0.7/redhat-cluster-3.0.7/group/dlm_controld/pacemaker.c:136
#3 0x00007f4b1d36cd73 in IA__g_hash_table_foreach (hash_table=0x1e12050,
func=0x403e40 <dlm_process_node>, user_data=0x62a4f8) at
/build/buildd/glib2.0-2.24.0/glib/ghash.c:1325
#4 0x0000000000403c9e in update_cluster () at /usr/src/packages/redhat-
cluster/3.0.7/redhat-cluster-3.0.7/group/dlm_controld/pacemaker.c:82
#5 0x0000000000415a4a in loop () at /usr/src/packages/redhat-
cluster/3.0.7/redhat-cluster-3.0.7/group/dlm_controld/main.c:986
#6 0x000000000041659c in main (argc=<value optimized out>, argv=<value
optimized out>) at /usr/src/packages/redhat-cluster/3.0.7/redhat-
cluster-3.0.7/group/dlm_controld/main.c:1295



hth,
Oliver

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
On 27.04.2010 15:26, Oliver Heinz wrote:

> bad news is: it's not only bonding related

It's not bonding at all. It's an upstart issue. corosync is started
before all network interfaces are up. This issue is most visible with
bridging, cause bridged interfaces become functional much later than
they are started.

Never the less, dlm shouldn't segfault if there's no network :) That's
why I'll create debugging packages and send as much info I can get to
the upstream.

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Re: [Ubuntu-ha] startup problem DLM on ubuntu lucid [ In reply to ]
Suggested patch:

diff --git a/group/dlm_controld/pacemaker.c b/group/dlm_controld/pacemaker.c
index c661343..93c1841 100644
--- a/group/dlm_controld/pacemaker.c
+++ b/group/dlm_controld/pacemaker.c
@@ -123,7 +123,7 @@ void dlm_process_node(gpointer key, gpointer
value, gpointer user_data)
} else if(rc == 0) {
do_remove = TRUE;

- } else if(is_active) {
+ } else if(is_active && node->addr) {
do_add = TRUE;
}



On Tue, Apr 27, 2010 at 4:20 PM, Oliver Heinz <oheinz@fbihome.de> wrote:
> Am Dienstag, 27. April 2010 13:00:27 schrieb Lars Ellenberg:
>> On Tue, Apr 27, 2010 at 09:33:27AM +0200, Oliver Heinz wrote:
> ..
>> > I installed every -dbg package that is available for any package
>> > installed on the system (just to be sure). There are no debug packages
>> > for most of the cluster-stack, if it would help to have them recompiled
>> > without symbol stripping I could do that.
>>
>> Yes please ;-)
>>
>> then,
>> gdb /usr/sbin/dlm_controld.pcmk core -ex where -ex quit
>
> Core was generated by `dlm_controld.pcmk -q 0'.
> Program terminated with signal 11, Segmentation fault.
> #0  __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:31
> 31      ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory.
>        in ../sysdeps/x86_64/multiarch/../strlen.S
> #0  __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:31
> #1  0x00007f4b1ccedd46 in *__GI___strdup (s=0x0) at strdup.c:42
> #2  0x0000000000403f0c in dlm_process_node (key=<value optimized out>,
> value=0x1e10a30, user_data=0x62a4f8) at /usr/src/packages/redhat-
> cluster/3.0.7/redhat-cluster-3.0.7/group/dlm_controld/pacemaker.c:136
> #3  0x00007f4b1d36cd73 in IA__g_hash_table_foreach (hash_table=0x1e12050,
> func=0x403e40 <dlm_process_node>, user_data=0x62a4f8) at
> /build/buildd/glib2.0-2.24.0/glib/ghash.c:1325
> #4  0x0000000000403c9e in update_cluster () at /usr/src/packages/redhat-
> cluster/3.0.7/redhat-cluster-3.0.7/group/dlm_controld/pacemaker.c:82
> #5  0x0000000000415a4a in loop () at /usr/src/packages/redhat-
> cluster/3.0.7/redhat-cluster-3.0.7/group/dlm_controld/main.c:986
> #6  0x000000000041659c in main (argc=<value optimized out>, argv=<value
> optimized out>) at /usr/src/packages/redhat-cluster/3.0.7/redhat-
> cluster-3.0.7/group/dlm_controld/main.c:1295
>
>
>
> hth,
> Oliver
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf

1 2  View All