Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Linux-HA>
  3. Pacemaker
killproc not found? o2cb shutdown via resource agent
matt at ecsorl
Nov 7, 2012, 2:43 PM
Post #1 of 14 (7066 views)
Permalink
Hi,

Trying to get a new cluster running, and hitting a snag when bringing
nodes offline. FWIW, this very well could be nothing, but for sake of
making everything run "correctly..." I pulled the following out of the
syslog:

Nov 7 16:59:23 hv02 o2cb[6088]: INFO: Stopping p_o2cb:0
Nov 7 16:59:23 hv02 o2cb[6088]: INFO: Stopping ocfs2_controld.cman
Nov 7 16:59:23 hv02 lrmd: [2955]: info: RA output:
(p_o2cb:0:stop:stderr) /usr/lib/ocf/resource.d//pacemaker/o2cb: line
169: killproc: command not found

Any thoughts on the above killproc failure? The apparent consequence of
this is that ocfs2_controld.cman is still running.

Thanks!

--
Thank you!
Matthew O'Connor
(GPG Key ID: 55F981C4)


CONFIDENTIAL NOTICE: The information contained in this electronic message is legally privileged, confidential and exempt from disclosure under applicable law. It is intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender immediately by return e-mail and delete the original message and any copies of it from your computer system. Thank you.

Attached Files:

Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
matt at ecsorl
Nov 7, 2012, 2:59 PM
Post #2 of 14 (6936 views)
Permalink
Follow-up and additional info:

System is Ubuntu 12.04. Not sure where killproc is supposed to be
derived from, or if there is an assumption for it to be a standalone
binary or script. I did find it defined in /lib/lsb/init-functions.
Adding a ". /lib/lsb/init-functions" to the start of the
/usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
process-kill work, but I suspect this is not the most desirable solution.

Any thoughts on the consequences of killproc not functioning? Any
suggestions on a cleaner way to fix this?

Thanks again!


On 11/07/2012 05:43 PM, Matthew O'Connor wrote:
> Hi,
>
> Trying to get a new cluster running, and hitting a snag when bringing
> nodes offline. FWIW, this very well could be nothing, but for sake of
> making everything run "correctly..." I pulled the following out of the
> syslog:
>
> Nov 7 16:59:23 hv02 o2cb[6088]: INFO: Stopping p_o2cb:0
> Nov 7 16:59:23 hv02 o2cb[6088]: INFO: Stopping ocfs2_controld.cman
> Nov 7 16:59:23 hv02 lrmd: [2955]: info: RA output:
> (p_o2cb:0:stop:stderr) /usr/lib/ocf/resource.d//pacemaker/o2cb: line
> 169: killproc: command not found
>
> Any thoughts on the above killproc failure? The apparent consequence of
> this is that ocfs2_controld.cman is still running.
>
> Thanks!
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

Attached Files:

Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
andrew at beekhof
Nov 7, 2012, 5:11 PM
Post #3 of 14 (6930 views)
Permalink
On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
> Follow-up and additional info:
>
> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
> from, or if there is an assumption for it to be a standalone binary or
> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
> /lib/lsb/init-functions" to the start of the
> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
> process-kill work, but I suspect this is not the most desirable solution.

I think thats as good a solution as any.
I wonder where other distros are getting it from.

>
> Any thoughts on the consequences of killproc not functioning? Any
> suggestions on a cleaner way to fix this?
>
> Thanks again!
>
>
>
> On 11/07/2012 05:43 PM, Matthew O'Connor wrote:
>
> Hi,
>
> Trying to get a new cluster running, and hitting a snag when bringing
> nodes offline. FWIW, this very well could be nothing, but for sake of
> making everything run "correctly..." I pulled the following out of the
> syslog:
>
> Nov 7 16:59:23 hv02 o2cb[6088]: INFO: Stopping p_o2cb:0
> Nov 7 16:59:23 hv02 o2cb[6088]: INFO: Stopping ocfs2_controld.cman
> Nov 7 16:59:23 hv02 lrmd: [2955]: info: RA output:
> (p_o2cb:0:stop:stderr) /usr/lib/ocf/resource.d//pacemaker/o2cb: line
> 169: killproc: command not found
>
> Any thoughts on the above killproc failure? The apparent consequence of
> this is that ocfs2_controld.cman is still running.
>
> Thanks!
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
tserong at suse
Nov 7, 2012, 10:16 PM
Post #4 of 14 (6928 views)
Permalink
On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>> Follow-up and additional info:
>>
>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>> from, or if there is an assumption for it to be a standalone binary or
>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>> /lib/lsb/init-functions" to the start of the
>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>> process-kill work, but I suspect this is not the most desirable solution.
>
> I think thats as good a solution as any.
> I wonder where other distros are getting it from.

SLES 11 SP2:

# rpm -qf /sbin/killproc
sysvinit-2.86-210.1

openSUSE 12.2:

# rpm -qf /sbin/killproc
sysvinit-tools-2.88+-77.3.1.x86_64

Can't speak for any others offhand...

Regards,

Tim
--
Tim Serong
Senior Clustering Engineer
SUSE
tserong@suse.com

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
andrew at beekhof
Nov 8, 2012, 12:56 AM
Post #5 of 14 (6985 views)
Permalink
On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>> Follow-up and additional info:
>>>
>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>> from, or if there is an assumption for it to be a standalone binary or
>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>> /lib/lsb/init-functions" to the start of the
>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>> process-kill work, but I suspect this is not the most desirable solution.
>>
>> I think thats as good a solution as any.
>> I wonder where other distros are getting it from.
>
> SLES 11 SP2:
>
> # rpm -qf /sbin/killproc
> sysvinit-2.86-210.1
>
> openSUSE 12.2:
>
> # rpm -qf /sbin/killproc
> sysvinit-tools-2.88+-77.3.1.x86_64
>
> Can't speak for any others offhand...

Definitely not on fedora or its derivatives

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
tserong at suse
Nov 8, 2012, 1:23 AM
Post #6 of 14 (6935 views)
Permalink
On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>> Follow-up and additional info:
>>>>
>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>> from, or if there is an assumption for it to be a standalone binary or
>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>> /lib/lsb/init-functions" to the start of the
>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>
>>> I think thats as good a solution as any.
>>> I wonder where other distros are getting it from.
>>
>> SLES 11 SP2:
>>
>> # rpm -qf /sbin/killproc
>> sysvinit-2.86-210.1
>>
>> openSUSE 12.2:
>>
>> # rpm -qf /sbin/killproc
>> sysvinit-tools-2.88+-77.3.1.x86_64
>>
>> Can't speak for any others offhand...
>
> Definitely not on fedora or its derivatives

Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
be willing to bet the o2cb RA was based on the upstream o2cb init
script, which uses killproc, but also sources /lib/lsb/init-functions.
Does Fedora have killproc buried somewhere in there maybe?

On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
pidofproc() but these just wrap binaries of the same name in /sbin
(which would explain why o2cb works fine on SUSE, as those "missing"
things are presumably in $PATH anyway).

I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
might be a bit broad? Presumably couldn't hurt to source it in the o2cb
RA though, unless there's some other cleaner solution...

Regards,

Tim
--
Tim Serong
Senior Clustering Engineer
SUSE
tserong@suse.com

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
andrew at beekhof
Nov 8, 2012, 1:30 AM
Post #7 of 14 (6999 views)
Permalink
On Thu, Nov 8, 2012 at 8:23 PM, Tim Serong <tserong@suse.com> wrote:
> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>> Follow-up and additional info:
>>>>>
>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>> /lib/lsb/init-functions" to the start of the
>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>
>>>> I think thats as good a solution as any.
>>>> I wonder where other distros are getting it from.
>>>
>>> SLES 11 SP2:
>>>
>>> # rpm -qf /sbin/killproc
>>> sysvinit-2.86-210.1
>>>
>>> openSUSE 12.2:
>>>
>>> # rpm -qf /sbin/killproc
>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>
>>> Can't speak for any others offhand...
>>
>> Definitely not on fedora or its derivatives
>
> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
> be willing to bet the o2cb RA was based on the upstream o2cb init
> script, which uses killproc, but also sources /lib/lsb/init-functions.
> Does Fedora have killproc buried somewhere in there maybe?

I see it in /etc/rc.d/init.d/functions

/etc/rc.d/init.d/functions-313-# A function to stop a program.
/etc/rc.d/init.d/functions:314:killproc() {

>
> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
> pidofproc() but these just wrap binaries of the same name in /sbin
> (which would explain why o2cb works fine on SUSE, as those "missing"
> things are presumably in $PATH anyway).
>
> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
> RA though, unless there's some other cleaner solution...
>
> Regards,
>
> Tim
> --
> Tim Serong
> Senior Clustering Engineer
> SUSE
> tserong@suse.com
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
dejanmm at fastmail
Nov 8, 2012, 3:02 AM
Post #8 of 14 (6942 views)
Permalink
Hi,

On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
> > On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
> >> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
> >>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
> >>>> Follow-up and additional info:
> >>>>
> >>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
> >>>> from, or if there is an assumption for it to be a standalone binary or
> >>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
> >>>> /lib/lsb/init-functions" to the start of the
> >>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
> >>>> process-kill work, but I suspect this is not the most desirable solution.
> >>>
> >>> I think thats as good a solution as any.
> >>> I wonder where other distros are getting it from.
> >>
> >> SLES 11 SP2:
> >>
> >> # rpm -qf /sbin/killproc
> >> sysvinit-2.86-210.1
> >>
> >> openSUSE 12.2:
> >>
> >> # rpm -qf /sbin/killproc
> >> sysvinit-tools-2.88+-77.3.1.x86_64
> >>
> >> Can't speak for any others offhand...
> >
> > Definitely not on fedora or its derivatives
>
> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
> be willing to bet the o2cb RA was based on the upstream o2cb init
> script, which uses killproc, but also sources /lib/lsb/init-functions.
> Does Fedora have killproc buried somewhere in there maybe?
>
> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
> pidofproc() but these just wrap binaries of the same name in /sbin
> (which would explain why o2cb works fine on SUSE, as those "missing"
> things are presumably in $PATH anyway).
>
> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
> RA though, unless there's some other cleaner solution...

I'd also say just in this particular RA. Unfortunately, the
distro specific stuff creeps now and again into agents supposed
to work everywhere.

Cheers,

Dejan

> Regards,
>
> Tim
> --
> Tim Serong
> Senior Clustering Engineer
> SUSE
> tserong@suse.com
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
matt at ecsorl
Nov 8, 2012, 4:14 PM
Post #9 of 14 (6968 views)
Permalink
I'm honestly beginning to wonder what exactly that killproc does for the
ocfs2_controld.cman process... For kicks, I created a script in /sbin
and /usr/sbin for killproc, which simply sources the lsb include and
calls the function with whatever was passed via the command-line.
Perhaps an equivalent fix to modifying the RA or the included shell
extensions file, but still not as friendly as installing a .deb. ;-)

However, I'm not sure if it's doing anything useful, even though I can
see (via echos) that it's being called. The ocfs2_controld.cman process
doesn't go away till pacemaker is stopped (and isn't started until
pacemaker is running and the node is online), which blunders into
another problem: the o2cb RA appears to be in charge of unloading any
modules it loaded, but it fails to unload the ocfs2_stack_user module.
This causes CMAN to fail when shutting down; manually running 'service
o2cb stop' before 'service cman stop' resolves the problem, but I would
believe the RA should be doing this. Even when the ocfs2_controld.cman
process dies with pacemaker, the module remains. :-/


On 11/08/2012 06:02 AM, Dejan Muhamedagic wrote:
> Hi,
>
> On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
>> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>>> Follow-up and additional info:
>>>>>>
>>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>>> /lib/lsb/init-functions" to the start of the
>>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>> I think thats as good a solution as any.
>>>>> I wonder where other distros are getting it from.
>>>> SLES 11 SP2:
>>>>
>>>> # rpm -qf /sbin/killproc
>>>> sysvinit-2.86-210.1
>>>>
>>>> openSUSE 12.2:
>>>>
>>>> # rpm -qf /sbin/killproc
>>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>>
>>>> Can't speak for any others offhand...
>>> Definitely not on fedora or its derivatives
>> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
>> be willing to bet the o2cb RA was based on the upstream o2cb init
>> script, which uses killproc, but also sources /lib/lsb/init-functions.
>> Does Fedora have killproc buried somewhere in there maybe?
>>
>> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
>> pidofproc() but these just wrap binaries of the same name in /sbin
>> (which would explain why o2cb works fine on SUSE, as those "missing"
>> things are presumably in $PATH anyway).
>>
>> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
>> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
>> RA though, unless there's some other cleaner solution...
> I'd also say just in this particular RA. Unfortunately, the
> distro specific stuff creeps now and again into agents supposed
> to work everywhere.
>
> Cheers,
>
> Dejan
>
>> Regards,
>>
>> Tim
>> --
>> Tim Serong
>> Senior Clustering Engineer
>> SUSE
>> tserong@suse.com
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

Attached Files:

Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
andrew at beekhof
Nov 8, 2012, 5:15 PM
Post #10 of 14 (6919 views)
Permalink
You're not starting it as a pacemaker resource are you?
CMAN should be doing that as part of the init script (which explains
why its still there until after pacemaker is gone).

On Fri, Nov 9, 2012 at 11:14 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
> I'm honestly beginning to wonder what exactly that killproc does for the
> ocfs2_controld.cman process... For kicks, I created a script in /sbin
> and /usr/sbin for killproc, which simply sources the lsb include and
> calls the function with whatever was passed via the command-line.
> Perhaps an equivalent fix to modifying the RA or the included shell
> extensions file, but still not as friendly as installing a .deb. ;-)
>
> However, I'm not sure if it's doing anything useful, even though I can
> see (via echos) that it's being called. The ocfs2_controld.cman process
> doesn't go away till pacemaker is stopped (and isn't started until
> pacemaker is running and the node is online), which blunders into
> another problem: the o2cb RA appears to be in charge of unloading any
> modules it loaded, but it fails to unload the ocfs2_stack_user module.
> This causes CMAN to fail when shutting down; manually running 'service
> o2cb stop' before 'service cman stop' resolves the problem, but I would
> believe the RA should be doing this. Even when the ocfs2_controld.cman
> process dies with pacemaker, the module remains. :-/
>
>
> On 11/08/2012 06:02 AM, Dejan Muhamedagic wrote:
>> Hi,
>>
>> On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
>>> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>>>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>>>> Follow-up and additional info:
>>>>>>>
>>>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>>>> /lib/lsb/init-functions" to the start of the
>>>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>>> I think thats as good a solution as any.
>>>>>> I wonder where other distros are getting it from.
>>>>> SLES 11 SP2:
>>>>>
>>>>> # rpm -qf /sbin/killproc
>>>>> sysvinit-2.86-210.1
>>>>>
>>>>> openSUSE 12.2:
>>>>>
>>>>> # rpm -qf /sbin/killproc
>>>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>>>
>>>>> Can't speak for any others offhand...
>>>> Definitely not on fedora or its derivatives
>>> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
>>> be willing to bet the o2cb RA was based on the upstream o2cb init
>>> script, which uses killproc, but also sources /lib/lsb/init-functions.
>>> Does Fedora have killproc buried somewhere in there maybe?
>>>
>>> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
>>> pidofproc() but these just wrap binaries of the same name in /sbin
>>> (which would explain why o2cb works fine on SUSE, as those "missing"
>>> things are presumably in $PATH anyway).
>>>
>>> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
>>> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
>>> RA though, unless there's some other cleaner solution...
>> I'd also say just in this particular RA. Unfortunately, the
>> distro specific stuff creeps now and again into agents supposed
>> to work everywhere.
>>
>> Cheers,
>>
>> Dejan
>>
>>> Regards,
>>>
>>> Tim
>>> --
>>> Tim Serong
>>> Senior Clustering Engineer
>>> SUSE
>>> tserong@suse.com
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
matt at ecsorl
Nov 8, 2012, 9:43 PM
Post #11 of 14 (6962 views)
Permalink
On 11/08/2012 08:15 PM, Andrew Beekhof wrote:
> You're not starting it as a pacemaker resource are you?
> CMAN should be doing that as part of the init script (which explains
> why its still there until after pacemaker is gone).
I thought that was the dlm_controld, not ocfs2_controld? dlm_controld
is certainly managed by CMAN, but it hasn't been starting ocfs2_controld
for me...and without it, the OCFS2 shares won't mount. For reference:

primitive p_iscsiclient-store0-sandbox ocf:heartbeat:iscsi \
params portal="10.16.16.5:3260" target="..." \
...
primitive p_mount-store0-sandbox ocf:heartbeat:Filesystem \
params device="-U 443d287f-b98f-45e4-bd6e-d64dd7af0169"
directory="/opt/store3" fstype="ocfs2" \
...
primitive p_o2cb ocf:pacemaker:o2cb \
params stack="cman" \
...

(ordering and colocation constraints omitted, along with uninteresting
arguments.) I'll feel quite dumb if there was just some additional
configuration required for CMAN and OCFS2 and I somehow missed it. I
guess that would explain why CMAN would try to restart the
ocfs2_controld if the ocfs2 modules were still loaded and configfs was
still alive and well...though technically it failed every time it tried.

>
> On Fri, Nov 9, 2012 at 11:14 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>> I'm honestly beginning to wonder what exactly that killproc does for the
>> ocfs2_controld.cman process... For kicks, I created a script in /sbin
>> and /usr/sbin for killproc, which simply sources the lsb include and
>> calls the function with whatever was passed via the command-line.
>> Perhaps an equivalent fix to modifying the RA or the included shell
>> extensions file, but still not as friendly as installing a .deb. ;-)
>>
>> However, I'm not sure if it's doing anything useful, even though I can
>> see (via echos) that it's being called. The ocfs2_controld.cman process
>> doesn't go away till pacemaker is stopped (and isn't started until
>> pacemaker is running and the node is online), which blunders into
>> another problem: the o2cb RA appears to be in charge of unloading any
>> modules it loaded, but it fails to unload the ocfs2_stack_user module.
>> This causes CMAN to fail when shutting down; manually running 'service
>> o2cb stop' before 'service cman stop' resolves the problem, but I would
>> believe the RA should be doing this. Even when the ocfs2_controld.cman
>> process dies with pacemaker, the module remains. :-/
>>
>>
>> On 11/08/2012 06:02 AM, Dejan Muhamedagic wrote:
>>> Hi,
>>>
>>> On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
>>>> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>>>>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>>>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>>>>> Follow-up and additional info:
>>>>>>>>
>>>>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>>>>> /lib/lsb/init-functions" to the start of the
>>>>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>>>> I think thats as good a solution as any.
>>>>>>> I wonder where other distros are getting it from.
>>>>>> SLES 11 SP2:
>>>>>>
>>>>>> # rpm -qf /sbin/killproc
>>>>>> sysvinit-2.86-210.1
>>>>>>
>>>>>> openSUSE 12.2:
>>>>>>
>>>>>> # rpm -qf /sbin/killproc
>>>>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>>>>
>>>>>> Can't speak for any others offhand...
>>>>> Definitely not on fedora or its derivatives
>>>> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
>>>> be willing to bet the o2cb RA was based on the upstream o2cb init
>>>> script, which uses killproc, but also sources /lib/lsb/init-functions.
>>>> Does Fedora have killproc buried somewhere in there maybe?
>>>>
>>>> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
>>>> pidofproc() but these just wrap binaries of the same name in /sbin
>>>> (which would explain why o2cb works fine on SUSE, as those "missing"
>>>> things are presumably in $PATH anyway).
>>>>
>>>> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
>>>> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
>>>> RA though, unless there's some other cleaner solution...
>>> I'd also say just in this particular RA. Unfortunately, the
>>> distro specific stuff creeps now and again into agents supposed
>>> to work everywhere.
>>>
>>> Cheers,
>>>
>>> Dejan
>>>
>>>> Regards,
>>>>
>>>> Tim
>>>> --
>>>> Tim Serong
>>>> Senior Clustering Engineer
>>>> SUSE
>>>> tserong@suse.com
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

Attached Files:

Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
andrew at beekhof
Nov 9, 2012, 1:26 PM
Post #12 of 14 (6930 views)
Permalink
On Fri, Nov 9, 2012 at 4:43 PM, Matthew O'Connor <matt@ecsorl.com> wrote:
>
> On 11/08/2012 08:15 PM, Andrew Beekhof wrote:
>> You're not starting it as a pacemaker resource are you?
>> CMAN should be doing that as part of the init script (which explains
>> why its still there until after pacemaker is gone).
> I thought that was the dlm_controld, not ocfs2_controld?

I know it starts gfs_controld when using GFS... I assume its the same for OCFS2

> dlm_controld
> is certainly managed by CMAN, but it hasn't been starting ocfs2_controld
> for me...and without it, the OCFS2 shares won't mount. For reference:
>
> primitive p_iscsiclient-store0-sandbox ocf:heartbeat:iscsi \
> params portal="10.16.16.5:3260" target="..." \
> ...
> primitive p_mount-store0-sandbox ocf:heartbeat:Filesystem \
> params device="-U 443d287f-b98f-45e4-bd6e-d64dd7af0169"
> directory="/opt/store3" fstype="ocfs2" \
> ...
> primitive p_o2cb ocf:pacemaker:o2cb \
> params stack="cman" \
> ...
>
> (ordering and colocation constraints omitted, along with uninteresting
> arguments.) I'll feel quite dumb if there was just some additional
> configuration required for CMAN and OCFS2 and I somehow missed it. I
> guess that would explain why CMAN would try to restart the
> ocfs2_controld if the ocfs2 modules were still loaded and configfs was
> still alive and well...though technically it failed every time it tried.
>
>>
>> On Fri, Nov 9, 2012 at 11:14 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>> I'm honestly beginning to wonder what exactly that killproc does for the
>>> ocfs2_controld.cman process... For kicks, I created a script in /sbin
>>> and /usr/sbin for killproc, which simply sources the lsb include and
>>> calls the function with whatever was passed via the command-line.
>>> Perhaps an equivalent fix to modifying the RA or the included shell
>>> extensions file, but still not as friendly as installing a .deb. ;-)
>>>
>>> However, I'm not sure if it's doing anything useful, even though I can
>>> see (via echos) that it's being called. The ocfs2_controld.cman process
>>> doesn't go away till pacemaker is stopped (and isn't started until
>>> pacemaker is running and the node is online), which blunders into
>>> another problem: the o2cb RA appears to be in charge of unloading any
>>> modules it loaded, but it fails to unload the ocfs2_stack_user module.
>>> This causes CMAN to fail when shutting down; manually running 'service
>>> o2cb stop' before 'service cman stop' resolves the problem, but I would
>>> believe the RA should be doing this. Even when the ocfs2_controld.cman
>>> process dies with pacemaker, the module remains. :-/
>>>
>>>
>>> On 11/08/2012 06:02 AM, Dejan Muhamedagic wrote:
>>>> Hi,
>>>>
>>>> On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
>>>>> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>>>>>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>>>>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>>>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>>>>>> Follow-up and additional info:
>>>>>>>>>
>>>>>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>>>>>> /lib/lsb/init-functions" to the start of the
>>>>>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>>>>> I think thats as good a solution as any.
>>>>>>>> I wonder where other distros are getting it from.
>>>>>>> SLES 11 SP2:
>>>>>>>
>>>>>>> # rpm -qf /sbin/killproc
>>>>>>> sysvinit-2.86-210.1
>>>>>>>
>>>>>>> openSUSE 12.2:
>>>>>>>
>>>>>>> # rpm -qf /sbin/killproc
>>>>>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>>>>>
>>>>>>> Can't speak for any others offhand...
>>>>>> Definitely not on fedora or its derivatives
>>>>> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
>>>>> be willing to bet the o2cb RA was based on the upstream o2cb init
>>>>> script, which uses killproc, but also sources /lib/lsb/init-functions.
>>>>> Does Fedora have killproc buried somewhere in there maybe?
>>>>>
>>>>> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
>>>>> pidofproc() but these just wrap binaries of the same name in /sbin
>>>>> (which would explain why o2cb works fine on SUSE, as those "missing"
>>>>> things are presumably in $PATH anyway).
>>>>>
>>>>> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
>>>>> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
>>>>> RA though, unless there's some other cleaner solution...
>>>> I'd also say just in this particular RA. Unfortunately, the
>>>> distro specific stuff creeps now and again into agents supposed
>>>> to work everywhere.
>>>>
>>>> Cheers,
>>>>
>>>> Dejan
>>>>
>>>>> Regards,
>>>>>
>>>>> Tim
>>>>> --
>>>>> Tim Serong
>>>>> Senior Clustering Engineer
>>>>> SUSE
>>>>> tserong@suse.com
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
matt at ecsorl
Nov 9, 2012, 2:41 PM
Post #13 of 14 (6930 views)
Permalink
On 11/09/2012 04:26 PM, Andrew Beekhof wrote:
> On Fri, Nov 9, 2012 at 4:43 PM, Matthew O'Connor <matt@ecsorl.com> wrote:
>> On 11/08/2012 08:15 PM, Andrew Beekhof wrote:
>>> You're not starting it as a pacemaker resource are you?
>>> CMAN should be doing that as part of the init script (which explains
>>> why its still there until after pacemaker is gone).
>> I thought that was the dlm_controld, not ocfs2_controld?
> I know it starts gfs_controld when using GFS... I assume its the same for OCFS2
Yes, I saw that in the cman script...though I can't seem to find the
magic combination of modules and/or configfs writes to make cman
actually configure ocfs2/o2cb, though there are times it detects o2cb's
presence on init (shortly before dying horribly).

I can configure ocfs2 manually (via /etc/ocfs2/cluster.conf), though
this has no effect on cman (and vice versa), except that cman does not
crash on shutdown and pacemaker then has no involvement with o2cb. The
presence of the "cman" option for cluster stack in the o2cb RA is a
little bewildering.

I will do more research and reading, perhaps trying GFS out just to get
my head around how it interacts with CMAN. Perhaps there is a
corollary, or something simple missing from cluster.conf. Perhaps GFS
is the way to go, obviating these problems with OCFS2?

Thank you for your help!

>
>> dlm_controld
>> is certainly managed by CMAN, but it hasn't been starting ocfs2_controld
>> for me...and without it, the OCFS2 shares won't mount. For reference:
>>
>> primitive p_iscsiclient-store0-sandbox ocf:heartbeat:iscsi \
>> params portal="10.16.16.5:3260" target="..." \
>> ...
>> primitive p_mount-store0-sandbox ocf:heartbeat:Filesystem \
>> params device="-U 443d287f-b98f-45e4-bd6e-d64dd7af0169"
>> directory="/opt/store3" fstype="ocfs2" \
>> ...
>> primitive p_o2cb ocf:pacemaker:o2cb \
>> params stack="cman" \
>> ...
>>
>> (ordering and colocation constraints omitted, along with uninteresting
>> arguments.) I'll feel quite dumb if there was just some additional
>> configuration required for CMAN and OCFS2 and I somehow missed it. I
>> guess that would explain why CMAN would try to restart the
>> ocfs2_controld if the ocfs2 modules were still loaded and configfs was
>> still alive and well...though technically it failed every time it tried.
>>
>>> On Fri, Nov 9, 2012 at 11:14 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>> I'm honestly beginning to wonder what exactly that killproc does for the
>>>> ocfs2_controld.cman process... For kicks, I created a script in /sbin
>>>> and /usr/sbin for killproc, which simply sources the lsb include and
>>>> calls the function with whatever was passed via the command-line.
>>>> Perhaps an equivalent fix to modifying the RA or the included shell
>>>> extensions file, but still not as friendly as installing a .deb. ;-)
>>>>
>>>> However, I'm not sure if it's doing anything useful, even though I can
>>>> see (via echos) that it's being called. The ocfs2_controld.cman process
>>>> doesn't go away till pacemaker is stopped (and isn't started until
>>>> pacemaker is running and the node is online), which blunders into
>>>> another problem: the o2cb RA appears to be in charge of unloading any
>>>> modules it loaded, but it fails to unload the ocfs2_stack_user module.
>>>> This causes CMAN to fail when shutting down; manually running 'service
>>>> o2cb stop' before 'service cman stop' resolves the problem, but I would
>>>> believe the RA should be doing this. Even when the ocfs2_controld.cman
>>>> process dies with pacemaker, the module remains. :-/
>>>>
>>>>
>>>> On 11/08/2012 06:02 AM, Dejan Muhamedagic wrote:
>>>>> Hi,
>>>>>
>>>>> On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
>>>>>> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>>>>>>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>>>>>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>>>>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>>>>>>> Follow-up and additional info:
>>>>>>>>>>
>>>>>>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>>>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>>>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>>>>>>> /lib/lsb/init-functions" to the start of the
>>>>>>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>>>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>>>>>> I think thats as good a solution as any.
>>>>>>>>> I wonder where other distros are getting it from.
>>>>>>>> SLES 11 SP2:
>>>>>>>>
>>>>>>>> # rpm -qf /sbin/killproc
>>>>>>>> sysvinit-2.86-210.1
>>>>>>>>
>>>>>>>> openSUSE 12.2:
>>>>>>>>
>>>>>>>> # rpm -qf /sbin/killproc
>>>>>>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>>>>>>
>>>>>>>> Can't speak for any others offhand...
>>>>>>> Definitely not on fedora or its derivatives
>>>>>> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
>>>>>> be willing to bet the o2cb RA was based on the upstream o2cb init
>>>>>> script, which uses killproc, but also sources /lib/lsb/init-functions.
>>>>>> Does Fedora have killproc buried somewhere in there maybe?
>>>>>>
>>>>>> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
>>>>>> pidofproc() but these just wrap binaries of the same name in /sbin
>>>>>> (which would explain why o2cb works fine on SUSE, as those "missing"
>>>>>> things are presumably in $PATH anyway).
>>>>>>
>>>>>> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
>>>>>> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
>>>>>> RA though, unless there's some other cleaner solution...
>>>>> I'd also say just in this particular RA. Unfortunately, the
>>>>> distro specific stuff creeps now and again into agents supposed
>>>>> to work everywhere.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Dejan
>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Tim
>>>>>> --
>>>>>> Tim Serong
>>>>>> Senior Clustering Engineer
>>>>>> SUSE
>>>>>> tserong@suse.com
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>
>>>>>> Project Home: http://www.clusterlabs.org
>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

Attached Files:

Re: killproc not found? o2cb shutdown via resource agent [ In reply to ]
andrew at beekhof
Nov 21, 2012, 4:54 PM
Post #14 of 14 (6856 views)
Permalink
On Sat, Nov 10, 2012 at 9:41 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>
> On 11/09/2012 04:26 PM, Andrew Beekhof wrote:
>> On Fri, Nov 9, 2012 at 4:43 PM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>> On 11/08/2012 08:15 PM, Andrew Beekhof wrote:
>>>> You're not starting it as a pacemaker resource are you?
>>>> CMAN should be doing that as part of the init script (which explains
>>>> why its still there until after pacemaker is gone).
>>> I thought that was the dlm_controld, not ocfs2_controld?
>> I know it starts gfs_controld when using GFS... I assume its the same for OCFS2
> Yes, I saw that in the cman script...though I can't seem to find the
> magic combination of modules and/or configfs writes to make cman
> actually configure ocfs2/o2cb, though there are times it detects o2cb's
> presence on init (shortly before dying horribly).
>
> I can configure ocfs2 manually (via /etc/ocfs2/cluster.conf), though
> this has no effect on cman (and vice versa), except that cman does not
> crash on shutdown and pacemaker then has no involvement with o2cb.

If you're using cman, then pacemaker should _not_ have anything to do with o2cb.
So this would be correct.

All pacemaker does is mount filesystems in this case.

> The
> presence of the "cman" option for cluster stack in the o2cb RA is a
> little bewildering.
>
> I will do more research and reading, perhaps trying GFS out just to get
> my head around how it interacts with CMAN. Perhaps there is a
> corollary, or something simple missing from cluster.conf. Perhaps GFS
> is the way to go, obviating these problems with OCFS2?

Maybe. I'm really not an authority on cluster filesystems.
About the only time I use them is when I'm updating clusters from
scratch and thats a very controlled environment.

>
> Thank you for your help!
>
>>
>>> dlm_controld
>>> is certainly managed by CMAN, but it hasn't been starting ocfs2_controld
>>> for me...and without it, the OCFS2 shares won't mount. For reference:
>>>
>>> primitive p_iscsiclient-store0-sandbox ocf:heartbeat:iscsi \
>>> params portal="10.16.16.5:3260" target="..." \
>>> ...
>>> primitive p_mount-store0-sandbox ocf:heartbeat:Filesystem \
>>> params device="-U 443d287f-b98f-45e4-bd6e-d64dd7af0169"
>>> directory="/opt/store3" fstype="ocfs2" \
>>> ...
>>> primitive p_o2cb ocf:pacemaker:o2cb \
>>> params stack="cman" \
>>> ...
>>>
>>> (ordering and colocation constraints omitted, along with uninteresting
>>> arguments.) I'll feel quite dumb if there was just some additional
>>> configuration required for CMAN and OCFS2 and I somehow missed it. I
>>> guess that would explain why CMAN would try to restart the
>>> ocfs2_controld if the ocfs2 modules were still loaded and configfs was
>>> still alive and well...though technically it failed every time it tried.
>>>
>>>> On Fri, Nov 9, 2012 at 11:14 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>> I'm honestly beginning to wonder what exactly that killproc does for the
>>>>> ocfs2_controld.cman process... For kicks, I created a script in /sbin
>>>>> and /usr/sbin for killproc, which simply sources the lsb include and
>>>>> calls the function with whatever was passed via the command-line.
>>>>> Perhaps an equivalent fix to modifying the RA or the included shell
>>>>> extensions file, but still not as friendly as installing a .deb. ;-)
>>>>>
>>>>> However, I'm not sure if it's doing anything useful, even though I can
>>>>> see (via echos) that it's being called. The ocfs2_controld.cman process
>>>>> doesn't go away till pacemaker is stopped (and isn't started until
>>>>> pacemaker is running and the node is online), which blunders into
>>>>> another problem: the o2cb RA appears to be in charge of unloading any
>>>>> modules it loaded, but it fails to unload the ocfs2_stack_user module.
>>>>> This causes CMAN to fail when shutting down; manually running 'service
>>>>> o2cb stop' before 'service cman stop' resolves the problem, but I would
>>>>> believe the RA should be doing this. Even when the ocfs2_controld.cman
>>>>> process dies with pacemaker, the module remains. :-/
>>>>>
>>>>>
>>>>> On 11/08/2012 06:02 AM, Dejan Muhamedagic wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On Thu, Nov 08, 2012 at 08:23:53PM +1100, Tim Serong wrote:
>>>>>>> On 11/08/2012 07:56 PM, Andrew Beekhof wrote:
>>>>>>>> On Thu, Nov 8, 2012 at 5:16 PM, Tim Serong <tserong@suse.com> wrote:
>>>>>>>>> On 11/08/2012 12:11 PM, Andrew Beekhof wrote:
>>>>>>>>>> On Thu, Nov 8, 2012 at 9:59 AM, Matthew O'Connor <matt@ecsorl.com> wrote:
>>>>>>>>>>> Follow-up and additional info:
>>>>>>>>>>>
>>>>>>>>>>> System is Ubuntu 12.04. Not sure where killproc is supposed to be derived
>>>>>>>>>>> from, or if there is an assumption for it to be a standalone binary or
>>>>>>>>>>> script. I did find it defined in /lib/lsb/init-functions. Adding a ".
>>>>>>>>>>> /lib/lsb/init-functions" to the start of the
>>>>>>>>>>> /usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs file makes the
>>>>>>>>>>> process-kill work, but I suspect this is not the most desirable solution.
>>>>>>>>>> I think thats as good a solution as any.
>>>>>>>>>> I wonder where other distros are getting it from.
>>>>>>>>> SLES 11 SP2:
>>>>>>>>>
>>>>>>>>> # rpm -qf /sbin/killproc
>>>>>>>>> sysvinit-2.86-210.1
>>>>>>>>>
>>>>>>>>> openSUSE 12.2:
>>>>>>>>>
>>>>>>>>> # rpm -qf /sbin/killproc
>>>>>>>>> sysvinit-tools-2.88+-77.3.1.x86_64
>>>>>>>>>
>>>>>>>>> Can't speak for any others offhand...
>>>>>>>> Definitely not on fedora or its derivatives
>>>>>>> Hrm. Well, I just had a quick skim of the ocfs2-tools source, and I'd
>>>>>>> be willing to bet the o2cb RA was based on the upstream o2cb init
>>>>>>> script, which uses killproc, but also sources /lib/lsb/init-functions.
>>>>>>> Does Fedora have killproc buried somewhere in there maybe?
>>>>>>>
>>>>>>> On SUSE, /lib/lsb/init-functions defines start_daemon(), killproc(), and
>>>>>>> pidofproc() but these just wrap binaries of the same name in /sbin
>>>>>>> (which would explain why o2cb works fine on SUSE, as those "missing"
>>>>>>> things are presumably in $PATH anyway).
>>>>>>>
>>>>>>> I don't know about sourcing /lib/lsb/init-functions in .ocf-shellfuncs -
>>>>>>> might be a bit broad? Presumably couldn't hurt to source it in the o2cb
>>>>>>> RA though, unless there's some other cleaner solution...
>>>>>> I'd also say just in this particular RA. Unfortunately, the
>>>>>> distro specific stuff creeps now and again into agents supposed
>>>>>> to work everywhere.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Dejan
>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Tim
>>>>>>> --
>>>>>>> Tim Serong
>>>>>>> Senior Clustering Engineer
>>>>>>> SUSE
>>>>>>> tserong@suse.com
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>
>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>> _______________________________________________
>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>
>>>>>> Project Home: http://www.clusterlabs.org
>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal