Mailing List Archive

On Sat, 31 Dec 2011 19:59:47 -0600
William Hubbs <williamh@gentoo.org> wrote:

> Udev, kmod (which is a replacement for module-init-tools which will be needed
> by >=udev-176), systemd, and soon others, are advocating a major change
> to the locations where binaries and libraries are stored on linux
> systems.
>
> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> understanding is that they want to move software that is installed in
> /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> everything from /lib to /usr/lib.

I coulda swore April was another four months away...


--
fonts, gcc-porting, it makes no sense how it makes no sense
toolchain, wxwidgets but i'll take it free anytime
@ gentoo.org EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

Hi,

On Sat, 2011-12-31 at 19:59 -0600, William Hubbs wrote:
> I have been working with robbat2 on solutions to the separate /usr issue
> (That is why I have specifically cc'd him on this email)
> which will allow people to not use an initramfs. If we migrate
> everything off of the root fs to /usr, all of those solutions become
> moot. On the other hand, if we don't migrate, we run the risk of
> eventually having our default configuration not supported by upstream.

I think the general consensus among other distros is that initramfs is
the new /. Many core elements of the Linux system will start installing
themselves in /usr, starting with udev, so we won't have a choice
anyway. Also, I doubt it's currently possible to boot a Gentoo system
without /usr mounted anyway.

> 1) Start migrating packages along with upstream and have everyone who
> has a separate /usr (including me by the way) start using an initramfs
> of some kind, either dracut or one that we generate specifically for
> gentoo. The reason I suggest the initramfs, is, unfortunately if we
> migrate everything, nothing else would work.

I also don't see a good reason to not adopt dracut, re-implementing
something that already works and is maintained by a competent upstream
seems wasteful to me. I really don't see why people resist using an
initramfs so much.

The udev/kmod/systemd/dracut effort to standardise the base userspace of
Linux is probably scary for quite a few Gentoo-ers as it means that the
end result of an installed Gentoo system will be less differentiated
than it was before. But it still is a step in the right direction as
most of these standardized pieces are much better than what we currently
have. The OpenRC/baselayout-2 fiasco, not much better than baselayout-1
and unmaintained upstream shows that even a relatively large
distribution like us can't maintain a competitive base system solution,
adopting the udev/kmod/systemd way will allow us to use all the work
that they are doing and instead concentrate on making a better system.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Sun, 01 Jan 2012 02:12:22 -0500
Olivier Crête <tester@gentoo.org> wrote:

> Hi,
>
> On Sat, 2011-12-31 at 19:59 -0600, William Hubbs wrote:
> > I have been working with robbat2 on solutions to the separate /usr
> > issue (That is why I have specifically cc'd him on this email)
> > which will allow people to not use an initramfs. If we migrate
> > everything off of the root fs to /usr, all of those solutions become
> > moot. On the other hand, if we don't migrate, we run the risk of
> > eventually having our default configuration not supported by
> > upstream.
>
> I think the general consensus among other distros is that initramfs is
> the new /. Many core elements of the Linux system will start
> installing themselves in /usr, starting with udev, so we won't have a
> choice anyway. Also, I doubt it's currently possible to boot a Gentoo
> system without /usr mounted anyway.
>
> > 1) Start migrating packages along with upstream and have everyone
> > who has a separate /usr (including me by the way) start using an
> > initramfs of some kind, either dracut or one that we generate
> > specifically for gentoo. The reason I suggest the initramfs, is,
> > unfortunately if we migrate everything, nothing else would work.
>
> I also don't see a good reason to not adopt dracut, re-implementing
> something that already works and is maintained by a competent upstream
> seems wasteful to me. I really don't see why people resist using an
> initramfs so much.
>
> The udev/kmod/systemd/dracut effort to standardise the base userspace
> of Linux is probably scary for quite a few Gentoo-ers as it means
> that the end result of an installed Gentoo system will be less
> differentiated than it was before. But it still is a step in the
> right direction as most of these standardized pieces are much better
> than what we currently have. The OpenRC/baselayout-2 fiasco, not much
> better than baselayout-1 and unmaintained upstream shows that even a
> relatively large distribution like us can't maintain a competitive
> base system solution, adopting the udev/kmod/systemd way will allow
> us to use all the work that they are doing and instead concentrate on
> making a better system.
>


All of my systems currently have a seperate /usr that is mounted at
boot. Unfortunately I do agree that this is not something that we can
fight. This was brought up earlier and the only thing we can do
for people like myself (who mount /usr at boot) is to create a simple
initramfs that only has the purpose of mounting /usr at boot. The main
thing I don't like about initramfs is that we have to regenerate it any
time we update the packages that get included in it.

--
Matthew Thode (prometheanfire)

On 01/01/12 15:12, Olivier Crête wrote:
> Hi,
>
> On Sat, 2011-12-31 at 19:59 -0600, William Hubbs wrote:
>> I have been working with robbat2 on solutions to the separate /usr issue
>> (That is why I have specifically cc'd him on this email)
>> which will allow people to not use an initramfs. If we migrate
>> everything off of the root fs to /usr, all of those solutions become
>> moot. On the other hand, if we don't migrate, we run the risk of
>> eventually having our default configuration not supported by upstream.
> I think the general consensus among other distros is that initramfs is
> the new /. Many core elements of the Linux system will start installing
> themselves in /usr, starting with udev, so we won't have a choice
> anyway. Also, I doubt it's currently possible to boot a Gentoo system
> without /usr mounted anyway.
"initramfs is the new /" ... and no one asked if maybe that doesn't
really make sense?

That people are now actively working on forcing one big system partition
is annoying, but I really don't see the need to add a layer or two of
complexification just because, well, why not.

>
>> 1) Start migrating packages along with upstream and have everyone who
>> has a separate /usr (including me by the way) start using an initramfs
>> of some kind, either dracut or one that we generate specifically for
>> gentoo. The reason I suggest the initramfs, is, unfortunately if we
>> migrate everything, nothing else would work.
> I also don't see a good reason to not adopt dracut,
Make it work and I'll reconsider it, until then genkernel wins by default.
> re-implementing
> something that already works and is maintained by a competent upstream
> seems wasteful to me. I really don't see why people resist using an
> initramfs so much.
What does it add, apart from time to the boot process? For some setups
(like my notebook with luks+lvm) there's no reasonable way around it,
but on my desktop it's worse than useless.
>
> The udev/kmod/systemd/dracut effort to standardise the base userspace of
> Linux is probably scary for quite a few Gentoo-ers as it means that the
> end result of an installed Gentoo system will be less differentiated
> than it was before. But it still is a step in the right direction as
> most of these standardized pieces are much better than what we currently
> have. The OpenRC/baselayout-2 fiasco, not much better than baselayout-1
> and unmaintained upstream shows that even a relatively large
> distribution like us can't maintain a competitive base system solution,
Eh what?

As part of the OpenRC upstream I find it weird that you call it a fiasco
when it works better than the other "solutions" and had about 10 commits
in the last 48 hours alone.

I don't see an advantage in replacing a known-good solution with some
random stuff that mostly doesn't work yet just because it's the future.


> adopting the udev/kmod/systemd way will allow us to use all the work
> that they are doing and instead concentrate on making a better system.
>
"Better" means no lennartware to me. I want to be able to fully debug
init script failures, which systemd makes very hard to impossible. On
some machines I have changes in the startup that would mean having to
hack up something in C and hope that it doesn't crash init for systemd
(what the bleeeep?)

Please don't try to bring the GnomeOS vision of having MacOS without
paying for it to my computing experience ...

On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> understanding is that they want to move software that is installed in
> /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> everything from /lib to /usr/lib.

I don't like this one bit. Things used to be simple with the "split" between
/bin and /usr/bin (and its related directories), this isn't going to make it
more simple.

> 1) Start migrating packages along with upstream and have everyone who
> has a separate /usr (including me by the way) start using an initramfs
> of some kind, either dracut or one that we generate specifically for
> gentoo. The reason I suggest the initramfs, is, unfortunately if we
> migrate everything, nothing else would work.

I use a separate /usr with LVM on all my systems. My root partition uses
RAID1. And I never had the need for an initramfs of any kind. Also, there
are some major hurdles to take when it comes to getting an initramfs working
with SELinux. Most initramfs implementations I saw are not SELinux aware, so
all changes they make to the system either result in failures when they try,
or failures when the root-switch occurs.

> 3) Try to maintain things the way they are as long as possible.

I'm all for this one.

But if people really want to focus on initramfs, I'd appreciate some
documentation help on it. Not only on how to create one, but also why it is
necessary, how to manage initramfs'es, the concepts underlying, etc.

Wkr,
Sven Vermeulen

On Sun, Jan 1, 2012 at 2:23 PM, Sven Vermeulen <swift@gentoo.org> wrote:
> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
>> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
>> understanding is that they want to move software that is installed in
>> /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
>> everything from /lib to /usr/lib.
>
> I don't like this one bit. Things used to be simple with the "split" between
> /bin and /usr/bin (and its related directories), this isn't going to make it
> more simple.

Actually, I've always thought that the split between /usr/bin and /bin
was quite arbitrary. However, I would like to note that merging /bin
and /usr/bin would break some app combinations like bzip2+pbzip2, so
that problem also needs to be solved (via eselect perhaps).

Overall, this change "feels" wrong to me, but there's nothing
intrinsically wrong with what they're suggesting. OTOH, it's probably
just going to cause chaos and further divergence between distros for
little gain.

--
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

William Hubbs posted on Sat, 31 Dec 2011 19:59:47 -0600 as excerpted:

> a significant change is taking place with several upstreams that will
> affect us in gentoo[. Boot-critical components such as Udev, kmod,
> etc], are advocating a major change to the locations where binaries
> and libraries are stored on linux systems.
>
> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> understanding is that they want to move software that is installed in
> /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> everything from /lib to /usr/lib.

> If we migrate everything off of the root fs to /usr, all of those
> solutions become moot. On the other hand, if we don't migrate,
> we run the risk of eventually having our default configuration
> not supported by upstream.
>
> I see three options:
>
> 1) Start migrating packages along with upstream[. Have separate /usr
> users] start using an initramfs [as previously discussed onlist].
>
> 2) Combine the sbin and bin directories both on the root filesystem and
> in /usr by moving things from /sbin to /bin and /usr/sbin to /usr/bin.
>
> 3) Try to maintain things the way they are as long as possible.
>
> Whether or not I like what is happening personally, I think we should
> consider the first option, because I think it will get more and more
> difficult for us to do anything else over time. And we will eventually
> find ourselves not supported by upstreams.

I'm for #1 (migrate along with upstream) as well.

Gentoo has historically been both "light patch", with a policy of staying
close to upstream if possible, and "customizer's choice", allowing users
far more flexibility than most distros. Keeping both goals in mind,
migrating with upstream is ultimately the only viable alternative for a
whole host of reasons including both staying close to upstream and simple
manpower resource issues.

That said, we can continue to try to support a separate /usr as long as
possible, while switching new installs to the new way and changing the
handbook to document it, preferably as soon as possible. Further, as
previously discussed, a near-static bare-minimal initramfs that can be
configured and forgotten about for long periods over many kernel upgrades
should make the switch as painless as possible at least for "simple" bare-
partition installations.


As for the switchover, I had already been thinking about it here and thus
have a couple ideas I'd very much like to see implemented in portage/PM/
base.eclass that could definitely help, along with a USE flag. I'll call
them "migrated-rootfs" and "migrateroot-strict" for purposes of
discussion, here, and assume they're both portage features and that
migrated-rootfs is also a USE flag

FEATURES=migrated-rootfs would set a USE-default for migrated-root-to-usr
so that it'd default to ON, and would indicate that a user is an "early
adopter" of the new layout, preferring migration as soon as possible.
Users could still set the USE flag as desired for specific packages, at
least at first, tho at some point it'd probably first be made a profile
default, and ultimately profile-masked either on or off.

Additionally, FEATURES=migrated-root would expand the collision-protect
feature to check for and warn on both same-package and cross-package
collisions between related dirs, with all four bin/sbin root/usr dirs
checked for name collisions, and both lib dirs (for single lib,
additional pairs for multilib) checked as well.

Optionally, if implementation is easy enough, have portage simply remove
symlinks to identically named files that would otherwise set off the
warning. This is a major reason for having it a feature and a USE flag
both, since if this is implemented, portage would take preventive action
quite apart from whether an ebuild had been updated to use the USE flag
or not.

FEATURES=migrateroot-strict would make those collision warnings fatal,
much like FEATURES=multilib-strict does for its multilib checks. (As an
amd64 user who had this feature on for years, until I switched to no-
multilib, that's what the idea is based on.)


The goal would be to allow early adopter users to set the less strict
version and run a --empty-tree @world update, then grep the logs for the
warnings it turned on. If none occurred and /usr is either already on
the same filesystem or they have the appropriate early-boot measures in
place, they could feel reasonably confident about setting up symlinks
pointing to the /usr/bin and /usr/lib dirs as appropriate, and could then
set FEATURES=migratedroot-strict to ensure it stays "clean", since after
that any attempted merge that would collide would be fatal.

Alternatively users could just set the strict version and see what
breaks, instead of grepping the logs for the warnings.

Since this would leverage the code already in place and tested for
collision-protect and multilib-strict, cross-package checking should be
fairly easy to implement, but same-package checking and/or actively
stripping colliding symlinks may involve rather more new code.

Zac? Other portage,/PM/PMS folks?

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

Sven Vermeulen wrote:
> But if people really want to focus on initramfs, I'd appreciate some
> documentation help on it. Not only on how to create one, but also why
> it is necessary, how to manage initramfs'es, the concepts underlying,
> etc. Wkr, Sven Vermeulen

This is my issue as well. I tried to make a init* to deal with this and
have yet to get one to work, not one single working boot up. I have
tried different howtos and not one of them produced anything that
works. I have not found a dracut howto that makes any sense either.
Sorry but genkernel left a bad taste in my mouth ages ago.

As bad as I hate to even use a init*, I hate even worse being told I
have to have one and not being able to get one to work following the
docs. There needs to be some really well tested docs for people to use
before this kicks in.

Now to go brush my teeth. :/ This init* thingy tastes really bad.

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Miss the compile output? Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

On 01/01/2012 01:23 AM, Duncan wrote:
> As for the switchover, I had already been thinking about it here and thus
> have a couple ideas I'd very much like to see implemented in portage/PM/
> base.eclass that could definitely help, along with a USE flag. I'll call
> them "migrated-rootfs" and "migrateroot-strict" for purposes of
> discussion, here, and assume they're both portage features and that
> migrated-rootfs is also a USE flag
>
> FEATURES=migrated-rootfs would set a USE-default for migrated-root-to-usr
> so that it'd default to ON, and would indicate that a user is an "early
> adopter" of the new layout, preferring migration as soon as possible.
> Users could still set the USE flag as desired for specific packages, at
> least at first, tho at some point it'd probably first be made a profile
> default, and ultimately profile-masked either on or off.

I'm not sure if a USE flag for FEATURES setting would be necessary. If
we want to enforce a global policy, then I guess a QA warning would be
warranted.

Overall, a migration like this should go pretty smoothly as long as
people with separate /usr take appropriate actions to make sure their
systems will boot. People without separate /usr can basically relax and
enjoy the ride.
--
Thanks,
Zac

On Sun, Jan 1, 2012 at 4:45 AM, Dale <rdalek1967@gmail.com> wrote:
> This is my issue as well.  I tried to make a init* to deal with this and
> have yet to get one to work, not one single working boot up.  I have tried
> different howtos and not one of them produced anything that works.  I have
> not found a dracut howto that makes any sense either.  Sorry but genkernel
> left a bad taste in my mouth ages ago.

While I think that the direction we're moving in (/usr on root or use
an initramfs) is inevitable, I do agree that it needs some
improvement.

I've gotten Dracut to work fine in VMs (even with fairly complex
setups), but every time I try it on my server it doesn't set up the
raid for some reason, and drops me to a dash shell. If I just run
mdadm_auto at that point it takes about 15 seconds to find and setup
my raid, and just typing exit boots the system just fine. I keep it
as an option in grub so that whenever my raid device numbers get mixed
up (seems to happen every other month for no apparent reason) I can
boot the system, since dracut does use the mdadm.conf and UUIDs to put
everything in the right place and find the right root (something you
can't do without an initramfs). One of these days I'll figure out how
to hack an mdadm_auto into the script as a last-ditch measure and then
I'll just have to deal with a one-minute delay.

For being the wave of the future it seems like the only documentation
Dracut has is a single page website with a few paragraphs of info.

I think the bottom line is that the future may look inevitable, but it
isn't actually here yet. I advocate being ready for the future, but
let's try not to force its arrival before everything is mature (though
having it in portage as an option is completely in the spirit of
Gentoo).

Rich

Rich Freeman wrote:
> On Sun, Jan 1, 2012 at 4:45 AM, Dale<rdalek1967@gmail.com> wrote:
>> This is my issue as well. I tried to make a init* to deal with this and
>> have yet to get one to work, not one single working boot up. I have tried
>> different howtos and not one of them produced anything that works. I have
>> not found a dracut howto that makes any sense either. Sorry but genkernel
>> left a bad taste in my mouth ages ago.
> While I think that the direction we're moving in (/usr on root or use
> an initramfs) is inevitable, I do agree that it needs some
> improvement.
>
> I've gotten Dracut to work fine in VMs (even with fairly complex
> setups), but every time I try it on my server it doesn't set up the
> raid for some reason, and drops me to a dash shell. If I just run
> mdadm_auto at that point it takes about 15 seconds to find and setup
> my raid, and just typing exit boots the system just fine. I keep it
> as an option in grub so that whenever my raid device numbers get mixed
> up (seems to happen every other month for no apparent reason) I can
> boot the system, since dracut does use the mdadm.conf and UUIDs to put
> everything in the right place and find the right root (something you
> can't do without an initramfs). One of these days I'll figure out how
> to hack an mdadm_auto into the script as a last-ditch measure and then
> I'll just have to deal with a one-minute delay.
>
> For being the wave of the future it seems like the only documentation
> Dracut has is a single page website with a few paragraphs of info.
>
> I think the bottom line is that the future may look inevitable, but it
> isn't actually here yet. I advocate being ready for the future, but
> let's try not to force its arrival before everything is mature (though
> having it in portage as an option is completely in the spirit of
> Gentoo).
>
> Rich
>
>


I installed dracut and it did something, although I'm not sure what
yet. I googled until I think google should be tired of me to find docs
to help. Getting it to run was on one page, then figuring out some
options was on yet another page, then grub was on yet another. I don't
know yet if some of these are outdated or anything so I don't know if it
is going to work or just blow up something. Please, don't ask me what it
did. I said I did it, I didn't say I understand it. If it is broken or
breaks in the future, I'm screwed. (Sort of starting to wonder why I
left Mandriva now. I could have swore I wanted to be rid of init* stuff
and make my on freakin kernels. )

I might also add for those considering dracut being the official way, it
is keyworded at this point. I would think someone needs to get this
stable or whatever needs doing before telling folks to use this. It may
work fine but if that is the case, then it needs to be unkeyworded.
Spell check doesn't like our terms here.

I think the future is actually taking several steps backward. That
point has been discussed to death tho.

I hope Sven has his thinking hat on. Maybe I can help some with the
docs. If it works for me, it should work for most. lol

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Miss the compile output? Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

On Sun, Jan 01, 2012 at 02:12:22AM -0500, Olivier Crête wrote:
> > 1) Start migrating packages along with upstream and have everyone who
> > has a separate /usr (including me by the way) start using an initramfs
> > of some kind, either dracut or one that we generate specifically for
> > gentoo. The reason I suggest the initramfs, is, unfortunately if we
> > migrate everything, nothing else would work.
>
> I also don't see a good reason to not adopt dracut, re-implementing
> something that already works and is maintained by a competent upstream
> seems wasteful to me. I really don't see why people resist using an
> initramfs so much.
>
> The udev/kmod/systemd/dracut effort to standardise the base userspace of
> Linux is probably scary for quite a few Gentoo-ers as it means that the
> end result of an installed Gentoo system will be less differentiated
> than it was before. But it still is a step in the right direction as
> most of these standardized pieces are much better than what we currently
> have. The OpenRC/baselayout-2 fiasco, not much better than baselayout-1
> and unmaintained upstream shows that even a relatively large
> distribution like us can't maintain a competitive base system solution,
> adopting the udev/kmod/systemd way will allow us to use all the work
> that they are doing and instead concentrate on making a better system.

The problem you are missing here is that gentoo isn't just for linux. We
are cross-platform, so we have to have a cross-platform init system as
the default. Unless they port systemd to *bsd, we may have to keep
openrc as the default init system for some time afaik.

Also, your statement about openrc not being maintained upstream is not
correct. It is correct that Roy doesn't work on it any more, but there
is a team that does maintain it.

William

Zac Medico posted on Sun, 01 Jan 2012 02:15:49 -0800 as excerpted:

> I'm not sure if a USE flag for FEATURES setting would be necessary. If
> we want to enforce a global policy, then I guess a QA warning would be
> warranted.

I didn't state why I suggested that, but here's the reasoning:

Unless I missed an update somewhere, USE flags are covered by PMS and
thus available to be used in ebuilds, etc. AFAIK, portage FEATURES are
just that, portage FEATURES, and thus are supposed to be opaque to
ebuilds, which shouldn't need to care which PM is running or its
features, as long as it's PMS compliant.

Thus, the split between the FEATURES bit which the ebuild shouldn't need
to know about (the user sets up the symlinks and sets the features and
portage takes care of it managing the rest for existing versions without
rewriting), and the USE flag, for where upstreams and/or ebuilds are
actually rewritten with the possibility of both layouts (and later only
the /usr locations) in mind and the ebuild installs to the targeted
location based on the USE flag.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Sat, 31 Dec 2011 19:59:47 -0600
William Hubbs <williamh@gentoo.org> wrote:

> I see three options:
>
> 1) Start migrating packages along with upstream and have everyone who
> has a separate /usr (including me by the way) start using an initramfs
> of some kind, either dracut or one that we generate specifically for
> gentoo. The reason I suggest the initramfs, is, unfortunately if we
> migrate everything, nothing else would work.
>
> 2) Combine the sbin and bin directories both on the root
> filesystem and in /usr by moving things from /sbin to /bin
> and /usr/sbin to /usr/bin.
>
> 3) Try to maintain things the way they are as long as possible.

We should *at least* start doing some preparations, like ensuring that
random things don't unnecessarily hardcode /sbin or /bin paths. Most
importantly, if a particular tool runs in a complete environment (which
is true for most of the cases), there is no need to hardcode paths to
tools.

As I see it, 2) seems achievable within a reasonable time now. It
shouldn't require any specific changes from users (except for fixing
their own broken scripts) yet some tree-wide action of fixing hardcoded
paths.

We could create /sbin and /usr/sbin symlinks as well but that shouldn't
be really necessary, especially if we prepare packages well beforehand.
Introducing such symlinks would be hard to perform and getting rid of
them later would be even harder; mostly due to PMS-enforced limitations.

For a long-term view, 1) is the only way to go. Splitting packages
randomly between rootfs and /usr was never really correct, and we
especially shouldn't force users to junk their systems with shattered
packages and cheap glue to keep it all working.

I'd suggest going the other way than I did with kmod. Temporary IUSE
like 'install-to-usr', disabled by default for now. Packages having
that IUSE should have correct USE-dependencies, and users who need not
to use that could just enable 'install-to-usr' globally (we'd probably
want to mask it first).

Of course, that way will require removing hardcoded paths as well, or
supporting switching them on IUSE. For the latter, it may be simpler to
use '[install-to-usr=]' kind of dependencies.

--
Best regards,
Michał Górny

On Sun, Jan 01, 2012 at 09:23:11AM +0000, Duncan wrote:
> Gentoo has historically been both "light patch", with a policy of staying
> close to upstream if possible, and "customizer's choice", allowing users
> far more flexibility than most distros. Keeping both goals in mind,
> migrating with upstream is ultimately the only viable alternative for a
> whole host of reasons including both staying close to upstream and simple
> manpower resource issues.

> That said, we can continue to try to support a separate /usr as long as
> possible, while switching new installs to the new way and changing the
> handbook to document it, preferably as soon as possible.

In this situation, I don't know how we will be able to support both
ways because there is more involved than just where things are
installed.

Udev 175 is currently masked, because the way it operates has changed
enough that it doesn't work correctly if it starts before /usr is
mounted, and the failures you find will be very difficult to track down.
So, udev isn't only changing where it is installed, but how it runs.

Supporting a separate /usr without an initramfs will require one of two
things.

One possibility is a customization to openrc, which we have been
looking at, that would be able to run fsck on /usr and mount /usr all
before udev starts. The drawback here is this will only work for openrc
users.

Another possibility is a script that would run as the init= parameter to
the kernel, which would fsck /usr and mount /usr then start the real
init process. This would be more generic and would be able to run
regardless of whether you are using sysvinit/openrc.

Here is the big problem with both of these solutions as I see them. They
depend on several pieces of software remaining on the root filesystem,
and if/when this software is migrated, we will be back to having this discussion
again.

> Further, as
> previously discussed, a near-static bare-minimal initramfs that can be
> configured and forgotten about for long periods over many kernel upgrades
> should make the switch as painless as possible at least for "simple" bare-
> partition installations.

I want to look at dracut and see if there is a way to configure it to
create a minimal initramfs like you are suggesting, because, if there is
we don't need to re-invent the wheel.

I don't think your portage feature/use flag suggestion is a good idea,
because, right now most of this is about where things are installed,
but, eventually we might have to start actually patching software to
make it work if it is installed on / instead of /usr, and there is no
way to know how much patching we would have to do.

What are your thoughts?

William

On Sun, 01 Jan 2012 02:12:22 -0500
Olivier Crête <tester@gentoo.org> wrote:
> The udev/kmod/systemd/dracut effort to standardise the base userspace
> of Linux is probably scary for quite a few Gentoo-ers as it means
> that the end result of an installed Gentoo system will be less
> differentiated than it was before. But it still is a step in the
> right direction as most of these standardized pieces are much better
> than what we currently have. The OpenRC/baselayout-2 fiasco, not much
> better than baselayout-1 and unmaintained upstream shows that even a
> relatively large distribution like us can't maintain a competitive
> base system solution, adopting the udev/kmod/systemd way will allow
> us to use all the work that they are doing and instead concentrate on
> making a better system.

Doesn't all this mess just show that no-one else can maintain a
"competitive" base system solution either?

--
Ciaran McCreesh

Yeah I know I"m replying to my own message, but I wanted to add a
thought about the symbolic links issue.

I don't think the /{bin,sbin,lib} and /usr/sbin directories should be
deleted.

However, what I would like to see is that the package maintainers would
be responsible for creating any compatibility symlinks their package
needs, not portage. I don't think it is a good idea to have portage or
any package manager controling the migration.

That way, once the package maintainer knows that the symbolic links are
not needed, they can be removed and eventually all of these directories
would be empty and they could eventually be removed if desired.

Thoughts?

William

On Sun, 2012-01-01 at 01:33 -0600, Matthew Thode wrote:
> On Sun, 01 Jan 2012 02:12:22 -0500
> Olivier Crête <tester@gentoo.org> wrote:
> All of my systems currently have a seperate /usr that is mounted at
> boot. Unfortunately I do agree that this is not something that we can
> fight. This was brought up earlier and the only thing we can do
> for people like myself (who mount /usr at boot) is to create a simple
> initramfs that only has the purpose of mounting /usr at boot. The main
> thing I don't like about initramfs is that we have to regenerate it any
> time we update the packages that get included in it.

That's why you have dracut to do it for you.


--
Olivier Crête
tester@gentoo.org
Gentoo Developer

Hi,

On Sun, 2012-01-01 at 12:46 -0600, William Hubbs wrote:
> I don't think the /{bin,sbin,lib} and /usr/sbin directories should be
> deleted.
>
> However, what I would like to see is that the package maintainers would
> be responsible for creating any compatibility symlinks their package
> needs, not portage. I don't think it is a good idea to have portage or
> any package manager controling the migration.

The other option is to do mv /bin/* /sbin/* /usr/sbin/* /usr/bin; and
then create symlinks from the other dirs to /usr/bin.. That can be done
in big move, it's the way Fedora is going to do it.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Sun, 1 Jan 2012 01:33:05 -0600
Matthew Thode (prometheanfire) <prometheanfire@gentoo.org> wrote:

> All of my systems currently have a seperate /usr that is mounted at
> boot. Unfortunately I do agree that this is not something that we can
> fight. This was brought up earlier and the only thing we can do
> for people like myself (who mount /usr at boot) is to create a simple
> initramfs that only has the purpose of mounting /usr at boot. The
> main thing I don't like about initramfs is that we have to regenerate
> it any time we update the packages that get included in it.

Eh, I think I'll end up writing myself that four lines of shell code
for you which mounts /usr and runs init.

Or maybe I'll leave that task to you. Attaching my /init for
micro-initramfs on klibc. It looks like that:

├── bin
│   ├── ls
│   ├── mount
│   ├── run-init
│   └── sh
├── dev
│   ├── null
│   └── sda2
├── init
├── lib64
│   ├── klibc-MZ_9MGQEe6NFqmrPjRpy5i6WlV8.so
│   └── libc.so -> klibc-MZ_9MGQEe6NFqmrPjRpy5i6WlV8.so
├── newroot
└── proc

$ du -sh
176K .

--
Best regards,
Michał Górny

On Sun, 2012-01-01 at 15:33 +0800, Patrick Lauer wrote:
> On 01/01/12 15:12, Olivier Crête wrote:
> > Hi,
> >
> > On Sat, 2011-12-31 at 19:59 -0600, William Hubbs wrote:
> >> I have been working with robbat2 on solutions to the separate /usr issue
> >> (That is why I have specifically cc'd him on this email)
> >> which will allow people to not use an initramfs. If we migrate
> >> everything off of the root fs to /usr, all of those solutions become
> >> moot. On the other hand, if we don't migrate, we run the risk of
> >> eventually having our default configuration not supported by upstream.
> > I think the general consensus among other distros is that initramfs is
> > the new /. Many core elements of the Linux system will start installing
> > themselves in /usr, starting with udev, so we won't have a choice
> > anyway. Also, I doubt it's currently possible to boot a Gentoo system
> > without /usr mounted anyway.
> "initramfs is the new /" ... and no one asked if maybe that doesn't
> really make sense?
>
> That people are now actively working on forcing one big system partition
> is annoying, but I really don't see the need to add a layer or two of
> complexification just because, well, why not.

We're absolutely not forcing a single system partition. We're just
saying that the bits required to mount all the partitions you want
should be in an initramfs.

> >> 1) Start migrating packages along with upstream and have everyone who
> >> has a separate /usr (including me by the way) start using an initramfs
> >> of some kind, either dracut or one that we generate specifically for
> >> gentoo. The reason I suggest the initramfs, is, unfortunately if we
> >> migrate everything, nothing else would work.
> > I also don't see a good reason to not adopt dracut,
> Make it work and I'll reconsider it, until then genkernel wins by default.
> > re-implementing
> > something that already works and is maintained by a competent upstream
> > seems wasteful to me. I really don't see why people resist using an
> > initramfs so much.
> What does it add, apart from time to the boot process? For some setups
> (like my notebook with luks+lvm) there's no reasonable way around it,
> but on my desktop it's worse than useless.

I don't see how it adds time to the boot process. Either you have a
single big partition (and then you don't even need an initramfs), or you
have multiple partitions and then most of the time is mounting them
anyway.

> > The udev/kmod/systemd/dracut effort to standardise the base userspace of
> > Linux is probably scary for quite a few Gentoo-ers as it means that the
> > end result of an installed Gentoo system will be less differentiated
> > than it was before. But it still is a step in the right direction as
> > most of these standardized pieces are much better than what we currently
> > have. The OpenRC/baselayout-2 fiasco, not much better than baselayout-1
> > and unmaintained upstream shows that even a relatively large
> > distribution like us can't maintain a competitive base system solution,
> Eh what?
>
> I don't see an advantage in replacing a known-good solution with some
> random stuff that mostly doesn't work yet just because it's the future.

Random stuff that was well though to work together and works well enough
that all other major distros are adopting it.

> > adopting the udev/kmod/systemd way will allow us to use all the work
> > that they are doing and instead concentrate on making a better system.
> >
> "Better" means no lennartware to me. I want to be able to fully debug
> init script failures, which systemd makes very hard to impossible. On
> some machines I have changes in the startup that would mean having to
> hack up something in C and hope that it doesn't crash init for systemd
> (what the bleeeep?)

You can start services with a shell scripts in systemd, you just have to
aim the .service unit file to you shellscript..

> Please don't try to bring the GnomeOS vision of having MacOS without
> paying for it to my computing experience ...

Honestly, so many things just work on MacOS and just need hours of
tweaking for us..

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Sun, 2012-01-01 at 08:53 +0000, Sven Vermeulen wrote:
> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
> > 1) Start migrating packages along with upstream and have everyone who
> > has a separate /usr (including me by the way) start using an initramfs
> > of some kind, either dracut or one that we generate specifically for
> > gentoo. The reason I suggest the initramfs, is, unfortunately if we
> > migrate everything, nothing else would work.
>
> I use a separate /usr with LVM on all my systems. My root partition uses
> RAID1. And I never had the need for an initramfs of any kind. Also, there
> are some major hurdles to take when it comes to getting an initramfs working
> with SELinux. Most initramfs implementations I saw are not SELinux aware, so
> all changes they make to the system either result in failures when they try,
> or failures when the root-switch occurs.

dracut fully supports SELinux (it's used in Fedora which has this
SELinux horror on by default).

> > 3) Try to maintain things the way they are as long as possible.
>
> I'm all for this one.
>
> But if people really want to focus on initramfs, I'd appreciate some
> documentation help on it. Not only on how to create one, but also why it is
> necessary, how to manage initramfs'es, the concepts underlying, etc.

Short version: use dracut.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Sun, 01 Jan 2012 15:21:24 -0500
Olivier Crête <tester@gentoo.org> wrote:
> Honestly, so many things just work on MacOS and just need hours of
> tweaking for us..

The problem with "just works" is that when it breaks, it can't be
fixed. Not being able to handle /usr on its own filesystem is a perfect
example of this.

--
Ciaran McCreesh

Olivier Crête wrote:
> On Sun, 2012-01-01 at 01:33 -0600, Matthew Thode wrote:
>> On Sun, 01 Jan 2012 02:12:22 -0500
>> Olivier Crête<tester@gentoo.org> wrote:
>> All of my systems currently have a seperate /usr that is mounted at
>> boot. Unfortunately I do agree that this is not something that we can
>> fight. This was brought up earlier and the only thing we can do
>> for people like myself (who mount /usr at boot) is to create a simple
>> initramfs that only has the purpose of mounting /usr at boot. The main
>> thing I don't like about initramfs is that we have to regenerate it any
>> time we update the packages that get included in it.
> That's why you have dracut to do it for you.
>
>

Which is keyworded at this point. Stable users do what?

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Miss the compile output? Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

On Sun, 2012-01-01 at 14:51 -0600, Dale wrote:
> Olivier Crête wrote:
> > On Sun, 2012-01-01 at 01:33 -0600, Matthew Thode wrote:
> >> On Sun, 01 Jan 2012 02:12:22 -0500
> >> Olivier Crête<tester@gentoo.org> wrote:
> >> All of my systems currently have a seperate /usr that is mounted at
> >> boot. Unfortunately I do agree that this is not something that we can
> >> fight. This was brought up earlier and the only thing we can do
> >> for people like myself (who mount /usr at boot) is to create a simple
> >> initramfs that only has the purpose of mounting /usr at boot. The main
> >> thing I don't like about initramfs is that we have to regenerate it any
> >> time we update the packages that get included in it.
> > That's why you have dracut to do it for you.
> >
> >
>
> Which is keyworded at this point. Stable users do what?

This is a discussion about the future... Changing keywords is trivial if
we care.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Sun, 2012-01-01 at 20:23 +0000, Ciaran McCreesh wrote:
> On Sun, 01 Jan 2012 15:21:24 -0500
> Olivier Crête <tester@gentoo.org> wrote:
> > Honestly, so many things just work on MacOS and just need hours of
> > tweaking for us..
>
> The problem with "just works" is that when it breaks, it can't be
> fixed. Not being able to handle /usr on its own filesystem is a perfect
> example of this.

systemd/dracut/etc handles /usr on its own filesystem just fine. What is
required is that /usr must be mounted before the pivot_root away from
the initramfs.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Sun, Jan 01, 2012 at 02:12:22AM -0500, Olivier Crête wrote:
> The OpenRC/baselayout-2 fiasco, not much better than baselayout-1 and
> unmaintained upstream shows that even a relatively large
Why do you say that OpenRC is unmaintained upstream? OpenRC is actively
maintained in Gentoo, with the largest contributors being WilliamH,
vapier, idl0r and myself.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Olivier Crête posted on Sun, 01 Jan 2012 15:17:50 -0500 as excerpted:

> On Sun, 2012-01-01 at 12:46 -0600, William Hubbs wrote:
>> I don't think the /{bin,sbin,lib} and /usr/sbin directories should be
>> deleted.
>>
>> However, what I would like to see is that the package maintainers would
>> be responsible for creating any compatibility symlinks their package
>> needs, not portage. I don't think it is a good idea to have portage or
>> any package manager controling the migration.
>
> The other option is to do mv /bin/* /sbin/* /usr/sbin/* /usr/bin; and
> then create symlinks from the other dirs to /usr/bin.. That can be done
> in big move, it's the way Fedora is going to do it.

That's what I had in mind, and in fact have already been thinking about
trying, here.

Which is why I don't really like the idea of packages placing symlinks,
since then it'd likely be the symlink copied last, overwriting the actual
binary with the symlink... pointing at itself due to the symlinked dirs!

Which is why I suggested a portage feature that would detect such
collisions and die before installing them, potentially overwriting the
binary with a symlink to itself!

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Sun, Jan 1, 2012 at 8:59 AM, William Hubbs <williamh@gentoo.org> wrote:
> Udev, kmod (which is a replacement for module-init-tools which will be needed
> by >=udev-176), systemd, and soon others, are advocating a major change
> to the locations where binaries and libraries are stored on linux
> systems.

Could you please point me to the discussions where udev, kmod and soon
others are advocating /usr/bin and /bin unification?
--
Duy

On 01/01/2012 09:39 PM, Duncan wrote:
> Olivier Crête posted on Sun, 01 Jan 2012 15:17:50 -0500 as excerpted:
>
>> On Sun, 2012-01-01 at 12:46 -0600, William Hubbs wrote:
>>> I don't think the /{bin,sbin,lib} and /usr/sbin directories should be
>>> deleted.
>>>
>>> However, what I would like to see is that the package maintainers would
>>> be responsible for creating any compatibility symlinks their package
>>> needs, not portage. I don't think it is a good idea to have portage or
>>> any package manager controling the migration.
>>
>> The other option is to do mv /bin/* /sbin/* /usr/sbin/* /usr/bin; and
>> then create symlinks from the other dirs to /usr/bin.. That can be done
>> in big move, it's the way Fedora is going to do it.
>
> That's what I had in mind, and in fact have already been thinking about
> trying, here.
>
> Which is why I don't really like the idea of packages placing symlinks,
> since then it'd likely be the symlink copied last, overwriting the actual
> binary with the symlink... pointing at itself due to the symlinked dirs!
>
> Which is why I suggested a portage feature that would detect such
> collisions and die before installing them, potentially overwriting the
> binary with a symlink to itself!

It should not be a problem because merge of symlinks is automatically
delayed in cases when the symlink target doesn't exist yet. There's a
loop where it merges as many regular files as it can, and if there are
any symlinks that can't be resolved then it merges them after all the
regular files have been merged.
--
Thanks,
Zac

On 2012-01-01 11:50 PM, Olivier Crête wrote:
> systemd/dracut/etc handles /usr on its own filesystem just fine. What is
> required is that /usr must be mounted before the pivot_root away from
> the initramfs.

Nitpick: I don't think pivot_root is used anymore. It is basically
unlink, mount, chroot and exec now (i.e. switch_root).

RedHat made some bad design decisions on RPM (.rpmnew files anyone?) and
udev. Udev was probably salvagable before systemd but noone has the
motivation or the man-power to manage the huge delta that would result
now. It would probably amount to forking udev. So people are following
along even if they are unhappy.

Plus Redhat did not support in-place upgrades last time I checked. So
they don't really care for a lot of problems that are important for us.

Regarding the original question, I belive there are 2 issues here:
1. Requiring initramfs (or dracut or whatever) for separate /usr
2. Migrating /bin to /usr/bin, /sbin to /usr/sbin etc.

For the first point, we should start requiring initramfs of some sort
for separate /usr now. That train has passed.

For the second point, we should hold on as long as we deem appropriate.
Then reconsider and -most probably- move ahead with the migration. Main
point is not to break existing installations by making the move too
quickly. Give sysadmins time to to adjust, resize partitions if
necessary etc. Do not go for half way solutions (i.e. number 2 in the
original email).

As a side note, thank you and keep up your good work on OpenRC. I find
it is easier, cleaner and in general superior to systemd especially in
server settings. And for my laptop, I don't really care which init
system is used anyway.

--
Eray Aslan <eras@gentoo.org>

On Mon, 2 Jan 2012 13:24:00 +0700
Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

> On Sun, Jan 1, 2012 at 8:59 AM, William Hubbs <williamh@gentoo.org>
> wrote:
> > Udev, kmod (which is a replacement for module-init-tools which will
> > be needed by >=udev-176), systemd, and soon others, are advocating
> > a major change to the locations where binaries and libraries are
> > stored on linux systems.
>
> Could you please point me to the discussions where udev, kmod and soon
> others are advocating /usr/bin and /bin unification?

I think he meant 'kmod doesn't provide hacks necessary to installs to
two roots at once'.

--
Best regards,
Michał Górny

On Sun, Jan 01, 2012 at 03:21:47PM -0500, Olivier Crête wrote:
> > But if people really want to focus on initramfs, I'd appreciate some
> > documentation help on it. Not only on how to create one, but also why it is
> > necessary, how to manage initramfs'es, the concepts underlying, etc.
>
> Short version: use dracut.

And how does dracut know which files it needs to mount my /usr?

Wkr,
Sven Vermeulen

On Mon, Jan 2, 2012 at 6:47 AM, Sven Vermeulen <swift@gentoo.org> wrote:
> And how does dracut know which files it needs to mount my /usr?

I assume based on the selection of modules that you enabled when
building/running it.

I believe dracut builds static binaries, so it mainly needs updating
when you build a new kernel.

Of course, right now dracut won't mount your /usr in any case, so it
needs to be improved.

Rich

On 3 January 2012 01:58, Rich Freeman <rich0@gentoo.org> wrote:

> On Mon, Jan 2, 2012 at 6:47 AM, Sven Vermeulen <swift@gentoo.org> wrote:
> > And how does dracut know which files it needs to mount my /usr?
>
> I assume based on the selection of modules that you enabled when
> building/running it.
>
> I believe dracut builds static binaries, so it mainly needs updating
> when you build a new kernel.
>
>
It also takes a list of filesystem types to add support for mounting , and
copies the mount.* binaries across to the tmpfs root image, along with any
dependencies.

May help to understand if you see the listing of the cpio'd archive:
https://gist.github.com/1550652

Its sort of magical really.

You'll note its only got a smattering of console fonts, particularly
"usr/share/consolefonts/ter-112n.psf", which I'm not even sure how it works
out.

The only place I have that configured is /etc/conf.d/consolefont and it
somehow works it out.

You may also notice a non-standard 'v86d' in sbin/ , for uvesafb support.

Its added by a simple dracut module I whipped up in 10 minutes, and its far
simpler to do with dracut than it ever was with genkernel.

/usr/share/dracut/modules.d/95v86d/module-setup.sh

#!/bin/bash
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
# ex: ts=8 sw=4 sts=4 et filetype=sh

check() {
return 0
}

depends() {
return 0
}

install() {
local _d

dracut_install "/sbin/v86d"
}


Been using dracut for 2+ months now and love it.

Sure, I've had to re-implement the bits of genkernel I actually used , and
I still lack an "add it to grub.conf for me" option, but its still overall
a net improvement in my eyes. ( I only ever really used genkernel to
automate compile and initrd creation, all the things that would touch the
.config file I disabled )

Here is basically what I run after I have my .config ready:

https://gist.github.com/1550685


--
Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 )
for ( 9,8,0,7,1,6,5,4,3,2 );"

On Mon, Jan 02, 2012 at 05:39:39AM +0000, Duncan wrote:
> Olivier Crête posted on Sun, 01 Jan 2012 15:17:50 -0500 as excerpted:
>
> > On Sun, 2012-01-01 at 12:46 -0600, William Hubbs wrote:
> >> I don't think the /{bin,sbin,lib} and /usr/sbin directories should be
> >> deleted.
> >>
> >> However, what I would like to see is that the package maintainers would
> >> be responsible for creating any compatibility symlinks their package
> >> needs, not portage. I don't think it is a good idea to have portage or
> >> any package manager controling the migration.
> >
> > The other option is to do mv /bin/* /sbin/* /usr/sbin/* /usr/bin; and
> > then create symlinks from the other dirs to /usr/bin.. That can be done
> > in big move, it's the way Fedora is going to do it.
>
> That's what I had in mind, and in fact have already been thinking about
> trying, here.
>
> Which is why I don't really like the idea of packages placing symlinks,
> since then it'd likely be the symlink copied last, overwriting the actual
> binary with the symlink... pointing at itself due to the symlinked dirs!

If we don't use symlinks at all, we do not have to
force everything in one big move like this, just allow the
package maintainers to update things as new releases come out or do rev
bumps to move their packages. Eventually everything will be off of
/{bin,sbin,lib} and be installed in /usr.

William

On Mon, 2 Jan 2012 07:58:00 -0500
Rich Freeman <rich0@gentoo.org> wrote:

> On Mon, Jan 2, 2012 at 6:47 AM, Sven Vermeulen <swift@gentoo.org>
> wrote:
> > And how does dracut know which files it needs to mount my /usr?
>
> I assume based on the selection of modules that you enabled when
> building/running it.
>
> I believe dracut builds static binaries, so it mainly needs updating
> when you build a new kernel.

AFAIK dracut doesn't build anything. It just copies files from
the running system.

--
Best regards,
Michał Górny

On Mon, Jan 02, 2012 at 01:24:00PM +0700, Nguyen Thai Ngoc Duy wrote:
> On Sun, Jan 1, 2012 at 8:59 AM, William Hubbs <williamh@gentoo.org> wrote:
> > Udev, kmod (which is a replacement for module-init-tools which will be needed
> > by >=udev-176), systemd, and soon others, are advocating a major change
> > to the locations where binaries and libraries are stored on linux
> > systems.
>
> Could you please point me to the discussions where udev, kmod and soon
> others are advocating /usr/bin and /bin unification?

http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
http://fedoraproject.org/wiki/Features/UsrMove

Also udev git now installs by default in /usr/bin and kmod installs
by default in /usr/bin.

Udev has now also been changed so that it doesn't care if rules fail to
run if/usr is not mounted. It doesn't keep track of this type of failure
and doesn't give us a way to re-run these rules.

That is just the beginning; I'm sure more packages will be changed to
follow this setup.

William

On Sun, 1 Jan 2012 00:16:29 -0600
Ryan Hill <dirtyepic@gentoo.org> wrote:

> On Sat, 31 Dec 2011 19:59:47 -0600
> William Hubbs <williamh@gentoo.org> wrote:
>
> > Udev, kmod (which is a replacement for module-init-tools which will
> > be needed by >=udev-176), systemd, and soon others, are advocating
> > a major change to the locations where binaries and libraries are
> > stored on linux systems.
> >
> > The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> > understanding is that they want to move software that is installed
> > in /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> > everything from /lib to /usr/lib.
>
> I coulda swore April was another four months away...
>
>

no, in April they will declare the zillions of #!/bin/sh scripts
broken because they rely on a so old unix standard :=)

A.

On Sun, Jan 01, 2012 at 06:54:34PM +0100, Michał Górny wrote:
> On Sat, 31 Dec 2011 19:59:47 -0600
> We should *at least* start doing some preparations, like ensuring that
> random things don't unnecessarily hardcode /sbin or /bin paths. Most
> importantly, if a particular tool runs in a complete environment (which
> is true for most of the cases), there is no need to hardcode paths to
> tools.
>
> As I see it, 2) seems achievable within a reasonable time now. It
> shouldn't require any specific changes from users (except for fixing
> their own broken scripts) yet some tree-wide action of fixing hardcoded
> paths.
>
> We could create /sbin and /usr/sbin symlinks as well but that shouldn't
> be really necessary, especially if we prepare packages well beforehand.
> Introducing such symlinks would be hard to perform and getting rid of
> them later would be even harder; mostly due to PMS-enforced limitations.

I'm not really a fan of the compatibility symlinks either the more I
think about it. I think that basically packages can revbump to make this
migration happen, or in the case of udev and kmod and other upstreams
that support it, not hard code the configuration.

>
> For a long-term view, 1) is the only way to go. Splitting packages
> randomly between rootfs and /usr was never really correct, and we
> especially shouldn't force users to junk their systems with shattered
> packages and cheap glue to keep it all working.
>
> I'd suggest going the other way than I did with kmod. Temporary IUSE
> like 'install-to-usr', disabled by default for now. Packages having
> that IUSE should have correct USE-dependencies, and users who need not
> to use that could just enable 'install-to-usr' globally (we'd probably
> want to mask it first).

I'm not sure that a use flag is a good idea for this, because there will
definitely be people who would turn it off, and with upstreams assuming
that this is how things are installed, those who turn it off will have
broken systems.

Here is My thought about how we can make this transition happen:

* Right now, I can re-work our kmod ebuild to install to its default
locations in /usr.

* When udev 176 is released, I'll put it in the tree, masked, installing
to its default locations. I can actually rework the live ebuild right
now to do this.

* I will then open a tracker for issues that will need to be fixed
before we can unmask it.

* Next, we will need people to unmask udev-176 and test to find issues.
I will definitely be one of these since I have separate
/usr on my desktop, however, I can't test all possible
configurations/packages here, so the more the merrier.

* We also need to get a better understanding of dracut at this point. I
believe that making an initramfs is as simple as running:
dracut "" -H
as root. Then you have to add it to your boot loader. The one thing I
haven't figured out yet is whether it is possible to create an
initramfs that doesn't have to be updated with every kernel upgrade.

* Once the issues are ironed out we unmask udev-176 and go from there.

What does everyone think? What am I leaving out?

William

On Mon, 2 Jan 2012 14:54:39 -0300
Alexis Ballier <aballier@gentoo.org> wrote:

> On Sun, 1 Jan 2012 00:16:29 -0600
> Ryan Hill <dirtyepic@gentoo.org> wrote:
>
> > On Sat, 31 Dec 2011 19:59:47 -0600
> > William Hubbs <williamh@gentoo.org> wrote:
> >
> > > Udev, kmod (which is a replacement for module-init-tools which
> > > will be needed by >=udev-176), systemd, and soon others, are
> > > advocating a major change to the locations where binaries and
> > > libraries are stored on linux systems.
> > >
> > > The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> > > understanding is that they want to move software that is installed
> > > in /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> > > everything from /lib to /usr/lib.
> >
> > I coulda swore April was another four months away...
>
> no, in April they will declare the zillions of #!/bin/sh scripts
> broken because they rely on a so old unix standard :=)

Yes, we should migrate to #!/usr/bin/env sh :P.


--
Best regards,
Michał Górny

On Mon, 2 Jan 2012 11:54:57 -0600
William Hubbs <williamh@gentoo.org> wrote:

> > For a long-term view, 1) is the only way to go. Splitting packages
> > randomly between rootfs and /usr was never really correct, and we
> > especially shouldn't force users to junk their systems with
> > shattered packages and cheap glue to keep it all working.
> >
> > I'd suggest going the other way than I did with kmod. Temporary IUSE
> > like 'install-to-usr', disabled by default for now. Packages having
> > that IUSE should have correct USE-dependencies, and users who need
> > not to use that could just enable 'install-to-usr' globally (we'd
> > probably want to mask it first).
>
> I'm not sure that a use flag is a good idea for this, because there
> will definitely be people who would turn it off, and with upstreams
> assuming that this is how things are installed, those who turn it off
> will have broken systems.

But it will give some of us a chance to carefully test changes without
enforcing them on all users or leaving them to lag behind with
packages.

Another, maybe even better solution is keeping modded packages in an
overlay.

> What does everyone think? What am I leaving out?

I think you are missing the long, necessary transition period.

Also, I don't think we should move udev (or anything else to /usr)
before we move all of its deps there. Well, at least library-level deps
which would be systemd. But I think I'll need to move it there anyway
because we don't have libdbus in rootfs.

--
Best regards,
Michał Górny

On Sun, 1 Jan 2012 14:39:57 +0530
Nirbheek Chauhan <nirbheek@gentoo.org> wrote:

> On Sun, Jan 1, 2012 at 2:23 PM, Sven Vermeulen <swift@gentoo.org>
> wrote:
> > On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
> >> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> >> understanding is that they want to move software that is installed
> >> in /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> >> everything from /lib to /usr/lib.
> >
> > I don't like this one bit. Things used to be simple with the
> > "split" between /bin and /usr/bin (and its related directories),
> > this isn't going to make it more simple.
>
> Actually, I've always thought that the split between /usr/bin and /bin
> was quite arbitrary. However, I would like to note that merging /bin
> and /usr/bin would break some app combinations like bzip2+pbzip2, so
> that problem also needs to be solved (via eselect perhaps).

Well, that /usr/bin idea of mine was quite hacky anyway. I don't think
we should even consider it here.

--
Best regards,
Michał Górny

On Sun, 1 Jan 2012 08:53:26 +0000
Sven Vermeulen <swift@gentoo.org> wrote:

> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
> > The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> > understanding is that they want to move software that is installed
> > in /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
> > everything from /lib to /usr/lib.
>
> I don't like this one bit. Things used to be simple with the "split"
> between /bin and /usr/bin (and its related directories), this isn't
> going to make it more simple.

Simple? Should I start requesting additional packages moved into rootfs
because I feel like needing them? Things can and will go more ugly,
and I wouldn't be surprised if anytime soon people will start
complaining because they ran out of space on their rootfs.

--
Best regards,
Michał Górny

On Mon, Jan 02, 2012 at 07:37:41PM +0100, Michał Górny wrote:
> On Mon, 2 Jan 2012 11:54:57 -0600
> William Hubbs <williamh@gentoo.org> wrote:
>
> > > For a long-term view, 1) is the only way to go. Splitting packages
> > > randomly between rootfs and /usr was never really correct, and we
> > > especially shouldn't force users to junk their systems with
> > > shattered packages and cheap glue to keep it all working.
> > >
> > > I'd suggest going the other way than I did with kmod. Temporary IUSE
> > > like 'install-to-usr', disabled by default for now. Packages having
> > > that IUSE should have correct USE-dependencies, and users who need
> > > not to use that could just enable 'install-to-usr' globally (we'd
> > > probably want to mask it first).
> >
> > I'm not sure that a use flag is a good idea for this, because there
> > will definitely be people who would turn it off, and with upstreams
> > assuming that this is how things are installed, those who turn it off
> > will have broken systems.
>
> But it will give some of us a chance to carefully test changes without
> enforcing them on all users or leaving them to lag behind with
> packages.

Stable or ~arch users wouldn't be affected, just those unmasking things
in p.mask.

> Another, maybe even better solution is keeping modded packages in an
> overlay.

Absolutely not. I don't see a reason to use an overlay for this; that's
what p.mask is for.

> > What does everyone think? What am I leaving out?
>
> I think you are missing the long, necessary transition period.

I'm not sure I agree. I don't see why there has to be a long transition
period if we coordinate everything correctly.

William

On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote

> I see three options:
>
> 1) Start migrating packages along with upstream and have everyone who
> has a separate /usr (including me by the way) start using an initramfs
> of some kind, either dracut or one that we generate specifically for
> gentoo. The reason I suggest the initramfs, is, unfortunately if we
> migrate everything, nothing else would work.
>
> 2) Combine the sbin and bin directories both on the root
> filesystem and in /usr by moving things from /sbin to /bin and /usr/sbin
> to /usr/bin.
>
> 3) Try to maintain things the way they are as long as possible.

4) Following pointers from Zac Medico and others, I've managed to get
Gentoo running with busybox's mdev, instead of udev. See
http://archives.gentoo.org/gentoo-user/msg_bc91b392ee0f76376104591cdf7dc5f0.xml
Executive summary... look Ma; no udev!

In the instructions there, I've set up a revised dev-manager ebuild in
an overlay. I've requested the changes to be incorporated into the
official ebuild and it appears to have been accepted. See...

https://bugs.gentoo.org/show_bug.cgi?id=395319

It should be rolled out eventually, and the overlay won't be necessary.

I think I've found one item so far that requires udev. My laptop's
graphics chip needs a binary blob from radeon-ucode. That binary blob,
in turn, requires the presence of /usr/lib/libudev.so.0 which is a
symlink to /usr/lib/libudev.so.0.9.3 (which is also required). I can
emerge udev
move or copy the 2 files over to /root
unmerge udev
move or copy the 2 files from /root to /usr/lib/
Note that /usr/lib/ is a symlink to /usr/lib64 on my 64-bit gentoo

> Please discuss; I want to hear what you think.

--
Walter Dnes <waltdnes@waltdnes.org>

Hi Walter,

On Tue, Jan 03, 2012 at 04:51:57AM -0500, Walter Dnes wrote:
> 4) Following pointers from Zac Medico and others, I've managed to get
> Gentoo running with busybox's mdev, instead of udev. See
> http://archives.gentoo.org/gentoo-user/msg_bc91b392ee0f76376104591cdf7dc5f0.xml
> Executive summary... look Ma; no udev!

Unfortunately, it isn't going to be as simple as switching away from
udev. This move is going to move all software from /bin, /sbin and /lib
to /usr/bin and /usr/lib. The end result is going to be that regardless
of whether you are using mdev or udev you will have to use an initramfs
if /usr is on a separate partition.

William

On 01/01/12 03:53 AM, Sven Vermeulen wrote:
> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
>> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
>> understanding is that they want to move software that is installed in
>> /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
>> everything from /lib to /usr/lib.
>
> I don't like this one bit. Things used to be simple with the "split" between
> /bin and /usr/bin (and its related directories), this isn't going to make it
> more simple.

I concurr. I will admit that I've been rather out of touch with what
other distros are doing (and have been for ~3-4 years), but combining
everything into /usr/bin just seems plain backwards and I am rather
shocked that all the distros are moving that way.

Has the LFH been updated?? Googling seems to say no, as the last mod
seems to have been in 2004... I know that, technically, these are
'userspace' programs in that they aren't kernel-space, but they're still
'system' programs so to me it still makes sense for them to be on the
'system' side of the filesystem hierarchy, doesn't it?


>
>> 3) Try to maintain things the way they are as long as possible.
>
> I'm all for this one.

I second this too. IMO, unless the FSSTND matches the new proposal from
udev/systemd/etc upstreams, then I think we should stick with what the
LFH describes. It may be possible, too, on this basis, to file bugs
against the upstreams to enforce compatible behaviour??

Of course, 'as long as possible' may depend a bit on what the timelines
are.. I would hope that we can support existing behaviour for at least
the next 6+ months? (at least then, if the Mayan calendar's right, the
end of the world will keep us from having to implement the change)..

On 01/01/12 05:15 AM, Zac Medico wrote:
>
> Overall, a migration like this should go pretty smoothly as long as
> people with separate /usr take appropriate actions to make sure their
> systems will boot. People without separate /usr can basically relax and
> enjoy the ride.


If a separate /usr is the only holdback, would it not be possible to
simply add static devnodes to the pre-udev /dev , and make a pre-init
wrapper script that mounts /usr ?

I know that the genkernel initramfs more or less does this (except that
it only attempts to mount 'root' instead of /usr, iirc), but it wouldn't
be hard to implement this behaviour in the main system either.. In
fact, it would probably be possible to emerge a small package that would
do this very thing (although getting behind a udev-controlled /dev might
be tricky at emerge-time -- it would need to have a rather heavy
pkg-postinst).

On 01/03/2012 10:53 AM, Ian Stakenvicius wrote:
> On 01/01/12 03:53 AM, Sven Vermeulen wrote:
>> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
>>> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
>>> understanding is that they want to move software that is installed in
>>> /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
>>> everything from /lib to /usr/lib.
>>
>> I don't like this one bit. Things used to be simple with the "split"
>> between
>> /bin and /usr/bin (and its related directories), this isn't going to
>> make it
>> more simple.
>
> I concurr. I will admit that I've been rather out of touch with what
> other distros are doing (and have been for ~3-4 years), but combining
> everything into /usr/bin just seems plain backwards and I am rather
> shocked that all the distros are moving that way.
>
> Has the LFH been updated?? Googling seems to say no, as the last mod
> seems to have been in 2004... I know that, technically, these are
> 'userspace' programs in that they aren't kernel-space, but they're
> still 'system' programs so to me it still makes sense for them to be
> on the 'system' side of the filesystem hierarchy, doesn't it?
The problem is that one group of developers is ignoring years of history
and purpose in the separation of /bin and /usr/bin and the ability of
having a separate /usr. This is in the udev development team and they
/deliberately/ placed or used some programs in /usr/bin instead /bin and
requiring that /usr bee in the root partition.

I will note that the historical separation of the /usr stems from the
days of user home directories being in /usr/home instead of /home. It
is getting to the point that the security aspects of having a read-only
mount for userspace executables is being overridden by developer fiat.

Lay this one at the RedHat/Fedora developers of udev.

--
G.Wolfe Woodbury
aka redwolfe@gmail.com

On 02/01/12 12:54 PM, William Hubbs wrote:

> The one thing I
> haven't figured out yet is whether it is possible to create an
> initramfs that doesn't have to be updated with every kernel upgrade.

I'm not sure if there is dracut-specific issues that would relate to
this, but the genkernel initramfs (as long as it didn't include any
kernel modules) used to work fine when used against a newer-built
kernel--note though that this was probably not 'supported' behaviour.
So I expect this to be the case with dracut as well.

On 03/01/12 10:10 AM, William Hubbs wrote:

> Unfortunately, it isn't going to be as simple as switching away from
> udev. This move is going to move all software from /bin, /sbin and /lib
> to /usr/bin and /usr/lib. The end result is going to be that regardless
> of whether you are using mdev or udev you will have to use an initramfs
> if /usr is on a separate partition.
>

I don't think anyone's asked this yet:

Do we NEED to deprecate /bin,/sbin,/usr/sbin,/lib ? I realize that
udev/kmod/systemd are moving, but there isn't anything in particular
that would require everything else to move, is there?

I know that there was a compatibility-symlink discussion and in general
it was thought that symlinks would be a bad idea.. but we could, for
anything that would be required in /bin,/sbin for mdev AND /usr/bin for
the new udev,etc (assuming there would actually be overlap, which I
expect there wouldn't be) make the /usr/bin version be a symlink and
keep /bin,/sbin,etc. around.. It would work at least as a temporary
measure, until the new udev/kmod/systemd becomes the de-facto default?



Side note - if /lib is getting moved, does that mean /lib/modules is
moving to /usr/lib/modules too? So kernel modules are no longer on root?

On Tue, 03 Jan 2012 11:08:09 -0500
"G.Wolfe Woodbury" <redwolfe@gmail.com> wrote:

> On 01/03/2012 10:53 AM, Ian Stakenvicius wrote:
> > On 01/01/12 03:53 AM, Sven Vermeulen wrote:
> >> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
> >>> The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
> >>> understanding is that they want to move software that is
> >>> installed in /bin, /sbin and /usr/sbin to /usr/bin. Also, they
> >>> want to move everything from /lib to /usr/lib.
> >>
> >> I don't like this one bit. Things used to be simple with the
> >> "split" between
> >> /bin and /usr/bin (and its related directories), this isn't going
> >> to make it
> >> more simple.
> >
> > I concurr. I will admit that I've been rather out of touch with
> > what other distros are doing (and have been for ~3-4 years), but
> > combining everything into /usr/bin just seems plain backwards and I
> > am rather shocked that all the distros are moving that way.
> >
> > Has the LFH been updated?? Googling seems to say no, as the last
> > mod seems to have been in 2004... I know that, technically, these
> > are 'userspace' programs in that they aren't kernel-space, but
> > they're still 'system' programs so to me it still makes sense for
> > them to be on the 'system' side of the filesystem hierarchy,
> > doesn't it?
> The problem is that one group of developers is ignoring years of
> history and purpose in the separation of /bin and /usr/bin and the
> ability of having a separate /usr. This is in the udev development
> team and they /deliberately/ placed or used some programs in /usr/bin
> instead /bin and requiring that /usr bee in the root partition.

Please, explain to me, how did they do it? As far as I am aware,
autotools installs files where it is told to.

--
Best regards,
Michał Górny

On Tue, Jan 03, 2012 at 11:03:04AM -0500, Ian Stakenvicius wrote:
> On 01/01/12 05:15 AM, Zac Medico wrote:
> >
> > Overall, a migration like this should go pretty smoothly as long as
> > people with separate /usr take appropriate actions to make sure their
> > systems will boot. People without separate /usr can basically relax and
> > enjoy the ride.
>
>
> If a separate /usr is the only holdback, would it not be possible to
> simply add static devnodes to the pre-udev /dev , and make a pre-init
> wrapper script that mounts /usr ?

I've thought about this, but a wrapper script assumes that the things it
needs are still available in /, so any wrapper script we make will break
as soon as something it needs migrates to /usr. For example, consider
what happens when bash or all of coreutils migrate to /usr.

William

On 03-01-2012 10:51:00 -0600, William Hubbs wrote:
> > If a separate /usr is the only holdback, would it not be possible to
> > simply add static devnodes to the pre-udev /dev , and make a pre-init
> > wrapper script that mounts /usr ?
>
> I've thought about this, but a wrapper script assumes that the things it
> needs are still available in /, so any wrapper script we make will break
> as soon as something it needs migrates to /usr. For example, consider
> what happens when bash or all of coreutils migrate to /usr.

Do you mean us, or upstream?
I don't think bash or coreutils do that. We explicitly configure them
with --prefix=/ or move some utils back and forth.


--
Fabian Groffen
Gentoo on a different level

On 03/01/12 11:51 AM, William Hubbs wrote:

> For example, consider
> what happens when bash or all of coreutils migrate to /usr.
>


..well, when /bin/sh no longer exists then there -will- be issues,
system-wide, on a massive scale. Unless shells or environments can
dynamically map that hash-bang to an appropriate interpreter (ie,
themselves) automatically.

*shudder*.. I don't even want to think about the migration i'd have to
do to handle that change.

Ian Stakenvicius posted on Tue, 03 Jan 2012 12:03:32 -0500 as excerpted:

> On 03/01/12 11:51 AM, William Hubbs wrote:
>
>> For example, consider what happens when bash or all of coreutils
>> migrate to /usr.
>
> ..well, when /bin/sh no longer exists then there -will- be issues,
> system-wide, on a massive scale. Unless shells or environments can
> dynamically map that hash-bang to an appropriate interpreter (ie,
> themselves) automatically.
>
> *shudder*.. I don't even want to think about the migration i'd have to
> do to handle that change.

FWIW, I was reading a review of [.was it GOBO Linux?, some distro that's
famous for reorganizing things much like MS does, a program files dir,
etc], and it was said to still contained a /bin with only a couple
symlinks, /bin/bash and /bin/sh, for this very reason.

Of course fedora uses an initr* so real-root and /usr will be mounted at
the same time, and they're doing a /bin -> /usr/bin symlink at least for
now, so they don't need to worry about that in the short term either.
Longer term, possibly they'll try to get rid of it, but I expect at least
some form of /bin/sh and/or /bin/bash symlink to remain around for quite
some time.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

Ian Stakenvicius posted on Tue, 03 Jan 2012 10:53:45 -0500 as excerpted:

> Has the LFH been updated?? Googling seems to say no, as the last mod
> seems to have been in 2004...

That was covered here in the last discussion. The FHS and LSB are
getting updated too, with the people driving the update being the very
same people, the fedora/redhat folks behind udev/udisks/upower/dbus/
systemd/etc, with gnome as well now talking about a "gnomeos" that
integrates and requires the whole set, plus X (or whatever the X
successor is called, I forgot ATM) and gnome, of course, so that future
gnome will assume systemd, etc, as well.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Tue, Jan 3, 2012 at 11:08 AM, G.Wolfe Woodbury <redwolfe@gmail.com> wrote:
>  It
> is getting to the point that the security aspects of having a read-only
> mount for userspace executables is being overridden by developer fiat.
>

Can you clarify what you mean by this? I think the whole reason that
RedHat is doing this is so that they can make /usr read-only, so that
it only changes when you perform upgrades. I imagine the next step
would be to use a trusted boot path and verify that partition when it
is mounted.

FHS has been brought up - I suspect the upstream projects that are
sparking this move are quite aware that they're breaking compliance,
so I doubt they're going to care if you file bugs pointing this out.
No doubt after the change is made they'll lobby to revise FHS, and at
that point since everybody will have gone along with it already there
won't be much point in voicing objection.

As with anything in FOSS - whoever has the developers gets to decide
how things work. Anybody can file bugs or post on mailing lists, but
the people writing the code will do what they do...

Rich

On Tue, Jan 03, 2012 at 05:35:51PM +0000, Duncan wrote:
> Ian Stakenvicius posted on Tue, 03 Jan 2012 12:03:32 -0500 as excerpted:
>
> > On 03/01/12 11:51 AM, William Hubbs wrote:
> >
> >> For example, consider what happens when bash or all of coreutils
> >> migrate to /usr.
> >
> > ..well, when /bin/sh no longer exists then there -will- be issues,
> > system-wide, on a massive scale. Unless shells or environments can
> > dynamically map that hash-bang to an appropriate interpreter (ie,
> > themselves) automatically.
> >
> > *shudder*.. I don't even want to think about the migration i'd have to
> > do to handle that change.
>
> FWIW, I was reading a review of [.was it GOBO Linux?, some distro that's
> famous for reorganizing things much like MS does, a program files dir,
> etc], and it was said to still contained a /bin with only a couple
> symlinks, /bin/bash and /bin/sh, for this very reason.
>
> Of course fedora uses an initr* so real-root and /usr will be mounted at
> the same time, and they're doing a /bin -> /usr/bin symlink at least for
> now, so they don't need to worry about that in the short term either.
> Longer term, possibly they'll try to get rid of it, but I expect at least
> some form of /bin/sh and/or /bin/bash symlink to remain around for quite
> some time.

Yes, the symlinks will be around for some time for this reason, but,
/bin/sh will point to /usr/bin/bash, so you have the same affect if /usr
is not mounted since the symlink can't be resolved.

William

Ian Stakenvicius posted on Tue, 03 Jan 2012 11:40:02 -0500 as excerpted:

> Side note - if /lib is getting moved, does that mean /lib/modules is
> moving to /usr/lib/modules too? So kernel modules are no longer on
> root?

Yes. Again, the whole thing is being designed from the perspective of a
binary distro which already uses an initr* to handle loading the modules
necessary for mounting real-root, and from that perspective, all they're
doing is having it handle /usr in the same way, mounting it right after
real-root in early-init, before control switches to it from the initr*.

The set of folks behind this don't particularly care about anyone doing
it a different way, which they consider Unix legacy, just as they
consider the BSDs, etc, legacy, integrating Linux-only solutions and
refusing to hold up "progress" (as they view it) just because someone
else can't keep up.

What we're really seeing now is the effect of letting RedHat with its
paid developers be the core behind so many core Linux systems, forcing
udev, systemd, /usr-as-the-new-root, etc, down everyone else's throats
because they can, because the entire community is so dependent on RedHat
(with Ubuntu and SuSE as well for some but not all of it) and its devs
and its money that it's no longer feasible for anyone else to fork all
the core programs RedHat devs lead on, and keep up. Sure, they could be
forked, but the forks would be left with few enough resources it'd be
like xfree86, they might still be there but in a few years they'd be
forgotten about by the rest of the community... One project, not a
problem, all of them together, just not feasible.

What about when glibc also begins to assume everything's in /usr/? ...

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On 03/01/12 12:45 PM, Duncan wrote:
> Ian Stakenvicius posted on Tue, 03 Jan 2012 10:53:45 -0500 as excerpted:
>
>> Has the LFH been updated?? Googling seems to say no, as the last mod
>> seems to have been in 2004...
>
> That was covered here in the last discussion. The FHS and LSB are
> getting updated too, with the people driving the update being the very
> same people, the fedora/redhat folks behind udev/udisks/upower/dbus/
> systemd/etc, with gnome as well now talking about a "gnomeos" that
> integrates and requires the whole set, plus X (or whatever the X
> successor is called, I forgot ATM) and gnome, of course, so that future
> gnome will assume systemd, etc, as well.
>

FHS, thank you. I didn't think I had that acronym right...


OK, well, if this is becoming the official standard then we will
definitely need to adhere to it; I guess it's just a matter of when and
what contingency/update plans we want to use to do it..

... if Gentoo's installed on a system (regardless of platform, and
leaving out the Prefix installs), the filesystem is up to gentoo right?
ie, there wouldn't be a need for a particular platform to stick with
FHS-2.3 would there? Once we migrate to FHS-3.0, we can migrate all
archs and profiles at the same time?

On Tue, Jan 03, 2012 at 11:40:02AM -0500, Ian Stakenvicius wrote:
> I don't think anyone's asked this yet:
>
> Do we NEED to deprecate /bin,/sbin,/usr/sbin,/lib ? I realize that
> udev/kmod/systemd are moving, but there isn't anything in particular
> that would require everything else to move, is there?

Well, I don't think everything is going to move immediately. The way I
see this happening is, udev/systemd/kmod are moving first, then other
upstreams will move their software.

If we don't move with them, I'm afraid that we will have more and more
customizations to maintain, either in the form of ebuilds using custom
build options, or maybe even software patches if programs operate with
the assumption that they are installed in /usr.

> I know that there was a compatibility-symlink discussion and in general
> it was thought that symlinks would be a bad idea.. but we could, for
> anything that would be required in /bin,/sbin for mdev AND /usr/bin for
> the new udev,etc (assuming there would actually be overlap, which I
> expect there wouldn't be) make the /usr/bin version be a symlink and
> keep /bin,/sbin,etc. around.. It would work at least as a temporary
> measure, until the new udev/kmod/systemd becomes the de-facto default?

Hmm, I'm not really interested in putting symbolic links in /usr/bin
linking to things in /bin or /sbin. I'm not following what that does for
us.

> Side note - if /lib is getting moved, does that mean /lib/modules is
> moving to /usr/lib/modules too? So kernel modules are no longer on root?

This is an interesting question. I haven't heard one way or the other
what is happening with /lib/modules.

William

Hi,

On Mon, 2012-01-02 at 10:41 +0200, Eray Aslan wrote:
> On 2012-01-01 11:50 PM, Olivier Crête wrote:
> > systemd/dracut/etc handles /usr on its own filesystem just fine. What is
> > required is that /usr must be mounted before the pivot_root away from
> > the initramfs.
>
> RedHat made some bad design decisions on RPM (.rpmnew files anyone?) and
> udev. Udev was probably salvagable before systemd but noone has the
> motivation or the man-power to manage the huge delta that would result
> now. It would probably amount to forking udev. So people are following
> along even if they are unhappy.

This is completely unrelated to RPMs. And udev was not developed by
RedHat, but by a Gentoo developer who works for Suse, then it was
maintained by a Suse dev who just very recently joined RedHat.

> Plus Redhat did not support in-place upgrades last time I checked. So
> they don't really care for a lot of problems that are important for us.

There is a good reason for that, because in-place upgrades are
impossible to do safely (and RedHat customers don't accept weird
breakages like Gentoo users do). For example, if you replace a library
or even a resource file (like a .ui file for GtkBuilder), the only way
to make it work is to make sure that no currently running application is
using it. And that just can't happen with system libraries like glibc or
system packages like udev or dbus. So the only safe way to upgrade those
is to reboot.

> Regarding the original question, I belive there are 2 issues here:
> 2. Migrating /bin to /usr/bin, /sbin to /usr/sbin etc.
>
> For the second point, we should hold on as long as we deem appropriate.
> Then reconsider and -most probably- move ahead with the migration. Main
> point is not to break existing installations by making the move too
> quickly. Give sysadmins time to to adjust, resize partitions if
> necessary etc. Do not go for half way solutions (i.e. number 2 in the
> original email).

I don't see what breakage would be caused by a big-bang update (move
everything in /sbin,/bin/,usr/sbin to usr/bin and add symlinks. I really
doubt any system has a /usr so tight that adding the couple things that
are in / to /usr/bin would break it.. Btw, this also includes /lib*
to /usr/lib*.


--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Tue, 03 Jan 2012 13:50:25 -0500
Olivier Crête <tester@gentoo.org> wrote:
> There is a good reason for that, because in-place upgrades are
> impossible to do safely (and RedHat customers don't accept weird
> breakages like Gentoo users do). For example, if you replace a library
> or even a resource file (like a .ui file for GtkBuilder), the only way
> to make it work is to make sure that no currently running application
> is using it. And that just can't happen with system libraries like
> glibc or system packages like udev or dbus. So the only safe way to
> upgrade those is to reboot.

Uhm... Unix filesystems don't work that way; you can unlink an open file
and anything that has that file still opened will continue to work.
You're thinking of Windows; Unix supports in-place upgrades just fine.

--
Ciaran McCreesh

On Tue, Jan 03, 2012 at 01:50:25PM -0500, Olivier Crête wrote:
> I don't see what breakage would be caused by a big-bang update (move
> everything in /sbin,/bin/,usr/sbin to usr/bin and add symlinks. I really
> doubt any system has a /usr so tight that adding the couple things that
> are in / to /usr/bin would break it.. Btw, this also includes /lib*
> to /usr/lib*.

I think the best way to do this part of it is going to be to just follow
the upstream packages. When they release a new version that installs in
/usr, just allow that to happen. Eventually there will be very little in
/{bin,sbin,lib}, maybe nothing besides a couple of symbolic links like
/bin/sh.

I am not for what fedora is doing with the
/bin->/usr/bin, /sbin->/usr/sbin and /lib->/usr/lib symlinks.

William

On Tue, 3 Jan 2012 18:54:27 +0000
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Tue, 03 Jan 2012 13:50:25 -0500
> Olivier Crête <tester@gentoo.org> wrote:
> > There is a good reason for that, because in-place upgrades are
> > impossible to do safely (and RedHat customers don't accept weird
> > breakages like Gentoo users do). For example, if you replace a
> > library or even a resource file (like a .ui file for GtkBuilder),
> > the only way to make it work is to make sure that no currently
> > running application is using it. And that just can't happen with
> > system libraries like glibc or system packages like udev or dbus.
> > So the only safe way to upgrade those is to reboot.
>
> Uhm... Unix filesystems don't work that way; you can unlink an open
> file and anything that has that file still opened will continue to
> work. You're thinking of Windows; Unix supports in-place upgrades
> just fine.

Considering that all applications keep all files open just for the fun
of it.

--
Best regards,
Michał Górny

On 03-01-2012 13:02:55 -0600, William Hubbs wrote:
> On Tue, Jan 03, 2012 at 01:50:25PM -0500, Olivier Crête wrote:
> > I don't see what breakage would be caused by a big-bang update (move
> > everything in /sbin,/bin/,usr/sbin to usr/bin and add symlinks. I really
> > doubt any system has a /usr so tight that adding the couple things that
> > are in / to /usr/bin would break it.. Btw, this also includes /lib*
> > to /usr/lib*.
>
> I think the best way to do this part of it is going to be to just follow
> the upstream packages. When they release a new version that installs in
> /usr, just allow that to happen. Eventually there will be very little in
> /{bin,sbin,lib}, maybe nothing besides a couple of symbolic links like
> /bin/sh.

What packages would that be? If you're thinking about coreutils, just
trim down the ebuild by not moving some of the tools to /bin. Our
ebuild makes it conform to FHS, not the coreutils buildsystem itself.


--
Fabian Groffen
Gentoo on a different level

On Tue, 2012-01-03 at 12:36 -0600, William Hubbs wrote:
> On Tue, Jan 03, 2012 at 11:40:02AM -0500, Ian Stakenvicius wrote:
> > Side note - if /lib is getting moved, does that mean /lib/modules is
> > moving to /usr/lib/modules too? So kernel modules are no longer on root?
>
> This is an interesting question. I haven't heard one way or the other
> what is happening with /lib/modules.

I doubt the kernel will move its install location, they're a very
conservative bunch. But I heard that kmod will start looking for modules
in /usr/lib/modules ...

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Tue, 2012-01-03 at 13:02 -0600, William Hubbs wrote:
> On Tue, Jan 03, 2012 at 01:50:25PM -0500, Olivier Crête wrote:
> > I don't see what breakage would be caused by a big-bang update (move
> > everything in /sbin,/bin/,usr/sbin to usr/bin and add symlinks. I really
> > doubt any system has a /usr so tight that adding the couple things that
> > are in / to /usr/bin would break it.. Btw, this also includes /lib*
> > to /usr/lib*.
>
> I think the best way to do this part of it is going to be to just follow
> the upstream packages. When they release a new version that installs in
> /usr, just allow that to happen. Eventually there will be very little in
> /{bin,sbin,lib}, maybe nothing besides a couple of symbolic links like
> /bin/sh.
>
> I am not for what fedora is doing with the
> /bin->/usr/bin, /sbin->/usr/sbin and /lib->/usr/lib symlinks.

At least the upstreams that work for RedHat and Suse (and that's almost
all system packages) will come to expect that these symlinks exist. For
example, I just heard that kmod will expect kernel modules
in /usr/lib/modules even though the kernel installs them
in /lib/modules.. So yes, upstream will force these symlinks on us too.

A couple years ago, Gentoo was the forward looking distribution, ready
to try radical changes that break existing assumption, like our init
scripts with dependencies or our early use of udev. These days, I see so
much resistance to progress, it makes me sad.


--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Tue, Jan 3, 2012 at 1:36 PM, William Hubbs <williamh@gentoo.org> wrote:
> Well, I don't think everything is going to move immediately. The way I
> see this happening is, udev/systemd/kmod are moving first, then other
> upstreams will move their software.

Agreed. If only a few packages have issues we don't have to subject
our users to a huge overnight change. We can just move gradually with
upstream.

> Hmm, I'm not really interested in putting symbolic links in /usr/bin
> linking to things in /bin or /sbin. I'm not following what that does for
> us.

Perhaps we could consider compatibility packages, like a bash-links
that just installs a symlink to /usr/bin/bash in /bin. Ideally we'd
never create them, but if a package can't be fixed in a
straightforward way we could add a link package and then have that
package depend on it. Then we can get rid of those links over time as
nothing depends on them when upstream catches up.

Before anything changes at all we need to have a solution in
dracut/etc for mounting /usr. Once that is in place packages can move
at any pace we wish, so there is no reason to short-cut QA.

Rch

On 03-01-2012 14:19:22 -0500, Olivier Crête wrote:
> On Tue, 2012-01-03 at 12:36 -0600, William Hubbs wrote:
> > On Tue, Jan 03, 2012 at 11:40:02AM -0500, Ian Stakenvicius wrote:
> > > Side note - if /lib is getting moved, does that mean /lib/modules is
> > > moving to /usr/lib/modules too? So kernel modules are no longer on root?
> >
> > This is an interesting question. I haven't heard one way or the other
> > what is happening with /lib/modules.
>
> I doubt the kernel will move its install location, they're a very
> conservative bunch. But I heard that kmod will start looking for modules
> in /usr/lib/modules ...

Seems they just made it a configure argument though [1].


[1] http://git.profusion.mobi/cgit.cgi/kmod.git/commit/?id=a308abec371364eec8344681cfe1fb50d624e43e

--
Fabian Groffen
Gentoo on a different level

2012/1/3 Olivier Crête <tester@gentoo.org>:
> A couple years ago, Gentoo was the forward looking distribution, ready
> to try radical changes that break existing assumption, like our init
> scripts with dependencies or our early use of udev. These days, I see so
> much resistance to progress, it makes me sad.

I think the key is to keep huge changes optional to start with. This
one feels like it is being pushed upon us.

I don't really have a big problem with moving to /usr and all that.
However, I do have some concerns with the larger direction that
everybody is taking with vertical integration (which this is just a
part of). For example, if eventually you can't run gnome without
systemd where does that leave bsd gentoo users? Gentoo is about
choice, and various upstream efforts are moving in the direction of
giving users only one choice - take it or leave it. How do you
install KDE and Gnome on the same system when they eventually want
different sysvinit implementations. Will the RedHat and Ubuntu of the
future have no more in common than Tivo and Android do today?

Rich

On Tue, Jan 03, 2012 at 02:19:39PM -0500, Olivier Crête wrote:
> On Tue, 2012-01-03 at 13:02 -0600, William Hubbs wrote:
> > On Tue, Jan 03, 2012 at 01:50:25PM -0500, Olivier Crête wrote:
> > > I don't see what breakage would be caused by a big-bang update (move
> > > everything in /sbin,/bin/,usr/sbin to usr/bin and add symlinks. I really
> > > doubt any system has a /usr so tight that adding the couple things that
> > > are in / to /usr/bin would break it.. Btw, this also includes /lib*
> > > to /usr/lib*.
> >
> > I think the best way to do this part of it is going to be to just follow
> > the upstream packages. When they release a new version that installs in
> > /usr, just allow that to happen. Eventually there will be very little in
> > /{bin,sbin,lib}, maybe nothing besides a couple of symbolic links like
> > /bin/sh.
> >
> > I am not for what fedora is doing with the
> > /bin->/usr/bin, /sbin->/usr/sbin and /lib->/usr/lib symlinks.
>
> At least the upstreams that work for RedHat and Suse (and that's almost
> all system packages) will come to expect that these symlinks exist. For
> example, I just heard that kmod will expect kernel modules
> in /usr/lib/modules even though the kernel installs them
> in /lib/modules.. So yes, upstream will force these symlinks on us too.

I just looked at the commit in kmod for this. It can be worked around
with a ./configure switch until the kernel switches their install
location, so they aren't forcing this one.

William

On Wed, Jan 4, 2012 at 12:06 AM, William Hubbs <williamh@gentoo.org> wrote:
> On Tue, Jan 03, 2012 at 11:40:02AM -0500, Ian Stakenvicius wrote:
>> I don't think anyone's asked this yet:
>>
>> Do we NEED to deprecate /bin,/sbin,/usr/sbin,/lib ?  I realize that
>> udev/kmod/systemd are moving, but there isn't anything in particular
>> that would require everything else to move, is there?
>
> Well, I don't think everything is going to move immediately. The way I
> see this happening is, udev/systemd/kmod are moving first, then other
> upstreams will move their software.
>

From what I can make out, *only* udev and systemd are making this a
hard-and-fast thing so far. kmod has added a configure argument, and
no one else has moved yet. In reality, fedora is patching a lot of
packages to make them install in /usr.

*If* GNU packages decide to move to /usr, *then* we can contemplate
moving. Till then, I say let's just do whatever Debian is doing.

--
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

On 03-01-2012 13:39:14 -0600, William Hubbs wrote:
> > in /usr/lib/modules even though the kernel installs them
> > in /lib/modules.. So yes, upstream will force these symlinks on us too.
>
> I just looked at the commit in kmod for this. It can be worked around
> with a ./configure switch until the kernel switches their install
> location, so they aren't forcing this one.

Current git even sets the default to "", so in fact they went back to
just /lib/modules.


--
Fabian Groffen
Gentoo on a different level

On Wed, Jan 4, 2012 at 1:05 AM, Rich Freeman <rich0@gentoo.org> wrote:
> part of).  For example, if eventually you can't run gnome without
> systemd where does that leave bsd gentoo users?

Is Mesa support on BSD really all that up-to-date these days? I don't
expect that they keep up with bugfixes that well. For instance, Radeon
works best with KMS + Gallium and afaik that has no driver for BSD at
all. I don't think GNOME Shell is stable on BSD at all.

> Gentoo is about
> choice, and various upstream efforts are moving in the direction of
> giving users only one choice - take it or leave it.

If you're arguing purely on the basis of BSD, I think it's a lost
cause. They're so far behind Linux that it's not even funny anymore.

> How do you
> install KDE and Gnome on the same system when they eventually want
> different sysvinit implementations.

I doubt that's going to happen, though. No DE other than GNOME is
interested in vertical integration. Even if someone is, it's a
technical problem to be solved with a technical solution. "Every
problem in computer science can be solved by adding another layer of
indirection".

> Will the RedHat and Ubuntu of the
> future have no more in common than Tivo and Android do today?
>

Setting aside the silly parts of comparison you've made, the rest is
already true from the user's PoV. The two have drastically different
UIs, and their packages are incompatible (rpm vs deb, yum vs apt). If
some applications are indeed common between the two, it's no surprise
since most of those run on Windows too.

--
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

On Tue, Jan 03, 2012 at 08:12:06PM +0100, Fabian Groffen wrote:
> > I think the best way to do this part of it is going to be to just follow
> > the upstream packages. When they release a new version that installs in
> > /usr, just allow that to happen. Eventually there will be very little in
> > /{bin,sbin,lib}, maybe nothing besides a couple of symbolic links like
> > /bin/sh.
>
> What packages would that be? If you're thinking about coreutils, just
> trim down the ebuild by not moving some of the tools to /bin. Our
> ebuild makes it conform to FHS, not the coreutils buildsystem itself.

Yes, coreutils will have to be reworked in that case. I don't know what
the upstream build system does, but we will have to take a look at it.

I'll have to go through on my system at
least and find all of the ebuilds that install things in
/{bin,sbin,lib}. I'll open a tracker bug as soon as udev-176 is
released; this will list all of the things we need to do to complete the
migration.

Basically I have these in my head:

* mask udev-176 in the tree.
* figure out and document how to make a simple initramfs with dracut.
* unmask udev 176 making sure to point users with a separate /usr
partition to how to make an initramfs (I could probably do this with
ewarns in the ebuild and maybe a news item before we go stable).
* stabilize a version of dracut.
* stabilize >=udev-176 and kmod.
* Now start stabilizing packages with things installed in /usr instead
of /.

If you do not have separate /usr, you could just enjoy the ride. All
of this would be transparent to you. If you do have separate /usr, the
critical step will be bringing up an initramfs once you upgrade to
udev-176. Once that is done, you could also just enjoy the ride.

Thoughts?

William

On Tue, 2012-01-03 at 14:35 -0500, Rich Freeman wrote:
> 2012/1/3 Olivier Crête <tester@gentoo.org>:
> > A couple years ago, Gentoo was the forward looking distribution, ready
> > to try radical changes that break existing assumption, like our init
> > scripts with dependencies or our early use of udev. These days, I see so
> > much resistance to progress, it makes me sad.
>
> I think the key is to keep huge changes optional to start with. This
> one feels like it is being pushed upon us.
>
> I don't really have a big problem with moving to /usr and all that.
> However, I do have some concerns with the larger direction that
> everybody is taking with vertical integration (which this is just a
> part of). For example, if eventually you can't run gnome without
> systemd where does that leave bsd gentoo users? Gentoo is about
> choice, and various upstream efforts are moving in the direction of
> giving users only one choice - take it or leave it. How do you
> install KDE and Gnome on the same system when they eventually want
> different sysvinit implementations. Will the RedHat and Ubuntu of the
> future have no more in common than Tivo and Android do today?

Well, don't worry, the KDE people don't have the will or the means to
make their own init system.. And rumor is that Ubuntu may be switching
to systemd in the near future too.

With a Linux kernel, you already need some Linux specific things like
udev, ifconfig/ip, etc. In the new world, you also have a Linux specific
init system. The BSD people are free to do whatever they want, they can
try to keep up with the Linux kernel, but good luck to them. Or they can
stay in the 80s. My advice to them is to admit that Linux is so far
ahead that they can't catch up and just join us.

Honestly, we should not promote choice at the expense of quality,
maintainability and reliability and these are the decisions that have
been made by the udev/systemd/etc upstreams. All of the init systems of
each Linux distribution has been doing the same thing in slightly
incompatible ways, so everyone has to maintain separate init scripts, on
each distro you have to remember where to set things like the hostname
(/etc/conf.d/hostname, /etc/hostname, /etc/rc.d/hostname, /etc/system/hostname, etc/wtf), etc. One of the key goals of systemd is to reduce this confusion by standardising the boot process across all distributions.

Vertical integration is the only way we can make things "Just Work" for
the users, we tried to do abstraction layers (HAL for example), but it
has been a failure. In the GNOME project, we're trying to make the Linux
desktop awesome, and we plan to fix any part of the puzzle that would
prevent us from achieving that goal.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Tue, Jan 03, 2012 at 01:50:25PM -0500, Olivier Crête wrote:
> On Mon, 2012-01-02 at 10:41 +0200, Eray Aslan wrote:
> > RedHat made some bad design decisions on RPM (.rpmnew files anyone?) and
> > udev. Udev was probably salvagable before systemd but noone has the
> > motivation or the man-power to manage the huge delta that would result
> > now. It would probably amount to forking udev. So people are following
> > along even if they are unhappy.
>
> This is completely unrelated to RPMs.

The same packages are moving towards a system where config files reside
under /usr/lib with users overriding the defaults in /etc
(/usr/lib/udev/rules.d/ and /etc/udev/rules.d). I doubt this would have
come about if RPM had a sensible way of dealing with config files -if
they had etc-update/dispatch-conf for example.

--
Eray Aslan <eras@gentoo.org>

On 03-01-2012 14:01:20 -0600, William Hubbs wrote:
> On Tue, Jan 03, 2012 at 08:12:06PM +0100, Fabian Groffen wrote:
> > > I think the best way to do this part of it is going to be to just follow
> > > the upstream packages. When they release a new version that installs in
> > > /usr, just allow that to happen. Eventually there will be very little in
> > > /{bin,sbin,lib}, maybe nothing besides a couple of symbolic links like
> > > /bin/sh.
> >
> > What packages would that be? If you're thinking about coreutils, just
> > trim down the ebuild by not moving some of the tools to /bin. Our
> > ebuild makes it conform to FHS, not the coreutils buildsystem itself.
>
> Yes, coreutils will have to be reworked in that case. I don't know what
> the upstream build system does, but we will have to take a look at it.

Upstream build system has all binaries just installed in $(PREFIX)/bin,
like normal autoconf-based projects.

> I'll have to go through on my system at
> least and find all of the ebuilds that install things in
> /{bin,sbin,lib}. I'll open a tracker bug as soon as udev-176 is
> released; this will list all of the things we need to do to complete the
> migration.

I would suggest not to do this. It's more interesting to know what udev
really requires to be in /usr/bin.

> Basically I have these in my head:
>
> * mask udev-176 in the tree.
> * figure out and document how to make a simple initramfs with dracut.
> * unmask udev 176 making sure to point users with a separate /usr
> partition to how to make an initramfs (I could probably do this with
> ewarns in the ebuild and maybe a news item before we go stable).
> * stabilize a version of dracut.
> * stabilize >=udev-176 and kmod.
> * Now start stabilizing packages with things installed in /usr instead
> of /.

If the system can work with things being in /, why would we have to move
away from that? I don't like my systems breaking, and this is a likely
candidate to do so. E.g. also gcc-config needs changing for this. And
baselayout/openrc. How many locations for functions.sh are there
already in scripts now?

> If you do not have separate /usr, you could just enjoy the ride. All
> of this would be transparent to you. If you do have separate /usr, the
> critical step will be bringing up an initramfs once you upgrade to
> udev-176. Once that is done, you could also just enjoy the ride.
>
> Thoughts?

Let's just first see how the rest of the world reacts to all of this.
We've waited serveral years with baselayout-1 -> openrc/baselayout-2, so
I wouldn't mind doing some waiting here too. It doesn't look like
there's much going to change. I can't imagine bash devs dropping /bin
from bare default PATH (we control the default PATH), neither that glibc
folks drop /lib from the search path (although more likely since it's
more limited to Linux).

Perhaps, even start to experiment with this in an overlay (it's
relatively easy to start, just disable gen_usr_ldscript, fix bash to
pass --prefix=/usr and coreutils not to move bins) and try to see how
hard migration is with zlib moving around (shouldn't be too hard with
ELF, is a downright drama with Mach-O && without portage's
preserve-libs).

But also, starting to experiment to see how easy it is to let things as
they are. udev might consider a world without /bin and /lib, but maybe
what's in there isn't much interesting to it anyway. Most stuff is
installed in /usr for a reason.


--
Fabian Groffen
Gentoo on a different level

On Sun, Jan 01, 2012 at 03:21:47PM -0500, Olivier Crête wrote:
> > I use a separate /usr with LVM on all my systems. My root partition uses
> > RAID1. And I never had the need for an initramfs of any kind. Also, there
> > are some major hurdles to take when it comes to getting an initramfs working
> > with SELinux. Most initramfs implementations I saw are not SELinux aware, so
> > all changes they make to the system either result in failures when they try,
> > or failures when the root-switch occurs.
>
> dracut fully supports SELinux (it's used in Fedora which has this
> SELinux horror on by default).

Yes... but no.

Fedora uses SELinux but using a policy where most domains run unconfined
(meaning they're allowed to do almost anything) and mostly the
network-facing services are confined.

I just got dracut working on a SELinux system here (took me a few hours to
compile a SELinux domain for dracut, because the application doesn't work
with the standard privileges of an administrator) and it boots up (up to
and including "dracut: Switching root") until SELinux is activated.

From that point onwards, it's dead since its using wrong labels and wrong
context.

It is SELinux-aware (it mounts the selinuxfs and such) but I think I'll need
to edit the /usr/lib/dracut/* stuff to get it to boot up properly on a
SELinux system that doesn't use unconfined domains...

I'll try to get it working the next few days. Once (or when) it does, I'll
submit the necessary patches to wherever is necessary.

Wkr,
Sven Vermeulen

On Tue, Jan 03, 2012 at 10:22:15PM +0100, Fabian Groffen wrote:
> > I'll have to go through on my system at
> > least and find all of the ebuilds that install things in
> > /{bin,sbin,lib}. I'll open a tracker bug as soon as udev-176 is
> > released; this will list all of the things we need to do to complete the
> > migration.
>
> I would suggest not to do this. It's more interesting to know what udev
> really requires to be in /usr/bin.

The issues involve binaries in /{bin,sbin} that link to libraries in
/usr/lib as well as packages that install udev rules that run binaries.

>
> > Basically I have these in my head:
> >
> > * mask udev-176 in the tree.
> > * figure out and document how to make a simple initramfs with dracut.
> > * unmask udev 176 making sure to point users with a separate /usr
> > partition to how to make an initramfs (I could probably do this with
> > ewarns in the ebuild and maybe a news item before we go stable).
> > * stabilize a version of dracut.
> > * stabilize >=udev-176 and kmod.

The part of the process above is the part I am the most concerned about.
I think we need to get everyone who is using separate /usr switched over
to an initramfs with udev 176, and this needs to happen sooner than
later, without using things like wrapper scripts or ways to avoid the
initramfs. Those are just stop-gap options that will only work until
some package they are depending on migrates to /usr.

Once we get to this point in the process, I think we could take each
package that installs things in / individually and migrate it. But, I
think the part of the process listed above needs to happen sooner than
later.

What are your thoughts?

William

On Tue, 3 Jan 2012 17:09:18 -0600
William Hubbs <williamh@gentoo.org> wrote:

> On Tue, Jan 03, 2012 at 10:22:15PM +0100, Fabian Groffen wrote:
> > > I'll have to go through on my system at
> > > least and find all of the ebuilds that install things in
> > > /{bin,sbin,lib}. I'll open a tracker bug as soon as udev-176 is
> > > released; this will list all of the things we need to do to
> > > complete the migration.
> >
> > I would suggest not to do this. It's more interesting to know what
> > udev really requires to be in /usr/bin.
>
> The issues involve binaries in /{bin,sbin} that link to libraries in
> /usr/lib as well as packages that install udev rules that run
> binaries.
>
> >
> > > Basically I have these in my head:
> > >
> > > * mask udev-176 in the tree.
> > > * figure out and document how to make a simple initramfs with
> > > dracut.
> > > * unmask udev 176 making sure to point users with a separate /usr
> > > partition to how to make an initramfs (I could probably do this
> > > with ewarns in the ebuild and maybe a news item before we go
> > > stable).
> > > * stabilize a version of dracut.
> > > * stabilize >=udev-176 and kmod.
>
> The part of the process above is the part I am the most concerned
> about. I think we need to get everyone who is using separate /usr
> switched over to an initramfs with udev 176, and this needs to happen
> sooner than later, without using things like wrapper scripts or ways
> to avoid the initramfs. Those are just stop-gap options that will
> only work until some package they are depending on migrates to /usr.
>
> Once we get to this point in the process, I think we could take each
> package that installs things in / individually and migrate it. But, I
> think the part of the process listed above needs to happen sooner than
> later.
>
> What are your thoughts?

I agree. Especially with the last part.

Thus, we need to:

1) fix and stabilize packages necessary to create an initramfs,
2) prepare really good instructions for creating one,
3) prepare a news item for users.

For the case of really simple initramfs mounting / and /usr only, I can
even create a small tool on klibc if anyone's interested.

--
Best regards,
Michał Górny

Ian Stakenvicius posted on Tue, 03 Jan 2012 13:18:06 -0500 as excerpted:

> ... if Gentoo's installed on a system (regardless of platform, and
> leaving out the Prefix installs), the filesystem is up to gentoo right?
> ie, there wouldn't be a need for a particular platform to stick with
> FHS-2.3 would there? Once we migrate to FHS-3.0, we can migrate all
> archs and profiles at the same time?

Filesystem or file hierarchy? At least on Gentoo, the filesystem choice
has always been up to the installing admin, but in context I believe you
mean file hierarchy.

As for archs, I'm not familiar with the minor ones and really only know
about x86/amd64, but I don't see any reason the others, possibly with an
exception or two that could be single-cased as needed, can't migrate at
the same time. I do know some of them get stuck on for instance, older
glibc, for long periods, tho, and it wouldn't surprise me at all to have
one or more of the minor ones stuck on an old version of something that
would either need monkey-patch-backported to deal with the new FHS, or
have special-cases for various packages to deal with older layout due to
the old versions of one or two packages holding things up for that arch.

AFAIK vapier's the gentoo guy with the knowledge there, due to all his
work on exotic stuff like superh/sh and the new x32 stuff, or at least
he's the one that seems most active in discussions such as this.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Tue, 2012-01-03 at 22:47 +0000, Sven Vermeulen wrote:
> On Sun, Jan 01, 2012 at 03:21:47PM -0500, Olivier Crête wrote:
> > > I use a separate /usr with LVM on all my systems. My root partition uses
> > > RAID1. And I never had the need for an initramfs of any kind. Also, there
> > > are some major hurdles to take when it comes to getting an initramfs working
> > > with SELinux. Most initramfs implementations I saw are not SELinux aware, so
> > > all changes they make to the system either result in failures when they try,
> > > or failures when the root-switch occurs.
> >
> > dracut fully supports SELinux (it's used in Fedora which has this
> > SELinux horror on by default).
>
> Yes... but no.
>
> Fedora uses SELinux but using a policy where most domains run unconfined
> (meaning they're allowed to do almost anything) and mostly the
> network-facing services are confined.
>
> I just got dracut working on a SELinux system here (took me a few hours to
> compile a SELinux domain for dracut, because the application doesn't work
> with the standard privileges of an administrator) and it boots up (up to
> and including "dracut: Switching root") until SELinux is activated.
>
> From that point onwards, it's dead since its using wrong labels and wrong
> context.
>
> It is SELinux-aware (it mounts the selinuxfs and such) but I think I'll need
> to edit the /usr/lib/dracut/* stuff to get it to boot up properly on a
> SELinux system that doesn't use unconfined domains...
>
> I'll try to get it working the next few days. Once (or when) it does, I'll
> submit the necessary patches to wherever is necessary.

My understanding is that the dracut maintainer recently removed SELinux
support and moved it into systemd. So patches that go in the other
directions aren't likely to go very far. My understanding is also that
it is now systemd doing all the SELinux magic (relabelling, etc), if you
don't want to use systemd, you should at least look at the relevant code
[1] [2] in systemd and do that in your own init system. And if you have
any questions, just ask Lennart, he's actually surprisingly helpful.

[1] http://cgit.freedesktop.org/systemd/tree/src/selinux-setup.c
[2] http://cgit.freedesktop.org/systemd/tree/src/mount-setup.c#n386

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

William Hubbs schrieb:
> On Tue, Jan 03, 2012 at 10:22:15PM +0100, Fabian Groffen wrote:
>>> I'll have to go through on my system at
>>> least and find all of the ebuilds that install things in
>>> /{bin,sbin,lib}. I'll open a tracker bug as soon as udev-176 is
>>> released; this will list all of the things we need to do to complete the
>>> migration.
>>
>> I would suggest not to do this. It's more interesting to know what udev
>> really requires to be in /usr/bin.
>
> The issues involve binaries in /{bin,sbin} that link to libraries in
> /usr/lib as well as packages that install udev rules that run binaries.
>
>>
>>> Basically I have these in my head:
>>>
>>> * mask udev-176 in the tree.
>>> * figure out and document how to make a simple initramfs with dracut.
>>> * unmask udev 176 making sure to point users with a separate /usr
>>> partition to how to make an initramfs (I could probably do this with
>>> ewarns in the ebuild and maybe a news item before we go stable).
>>> * stabilize a version of dracut.
>>> * stabilize >=udev-176 and kmod.
>
> The part of the process above is the part I am the most concerned about.
> I think we need to get everyone who is using separate /usr switched over
> to an initramfs with udev 176, and this needs to happen sooner than
> later, without using things like wrapper scripts or ways to avoid the
> initramfs. Those are just stop-gap options that will only work until
> some package they are depending on migrates to /usr.
>
> Once we get to this point in the process, I think we could take each
> package that installs things in / individually and migrate it. But, I
> think the part of the process listed above needs to happen sooner than
> later.
>
> What are your thoughts?
>
> William
>

If i did not miss something in this long thread, we are currently mostly
talking about udev needing /usr to be mounted when it starts. Systemd is
not our default init system and other packages are currently not
changing their default. Since udev is our default and that requires a
mounted /usr, we should show our users their options with a suggested
default:

1. minimal initramfs (default suggestion)
2. switching from udev to mdev (avoids required /usr of udev)
3. some wrapper script to mount /usr before udev starts

for 1: I use myself a self-created initramfs, which has worked fine for
many kernel releases, the only adjustment needed was back with kernel
2.6.27. With this in mind, some default minimal initramfs should be
doable without much maintaincence work.

for 2: from what i did read up to now, this option looks interesting for
those people, who dont want to use an initramfs or other mounting script
and dont use advanced features of udev, so especially server setups or
minimal desktop systems



For the idea of complete migration to /usr, i see no reason to go this
route in advance. Just keep with our default install locations and
follow upstream, if and where needed.
If a package switches to /usr and depending packages follow, fine, let
them do that and we can follow without much additional work.
There is no need to do such migration beforehand or changing our install
location in advance.


So in short:

1. Tell our users about the change in udev and show them their options
with a suggested default one.
2. for the rest, just calm down, follow upstreams and only change the
install location where needed. No need for mental pressure without an
actual pressure. ;-)

--

Thomas Sachau
Gentoo Linux Developer

On Wed, 04 Jan 2012 01:47:38 +0100
Thomas Sachau <tommy@gentoo.org> wrote:

> 2. switching from udev to mdev (avoids required /usr of udev)
> 3. some wrapper script to mount /usr before udev starts

These two should be really discouraged as a cheap, temporary solution.
We should not support hate-admining. I personally think that busybox is
ready to go into /usr even earlier than udev.

> For the idea of complete migration to /usr, i see no reason to go this
> route in advance. Just keep with our default install locations and
> follow upstream, if and where needed.

What about upstreams who do not care? In other words, all those
packages which we hack to install into rootfs?

--
Best regards,
Michał Górny

The 03/01/12, G.Wolfe Woodbury wrote:

> The problem is that one group of developers is ignoring years of history
> and purpose in the separation of /bin and /usr/bin and the ability of
> having a separate /usr. This is in the udev development team and they
> /deliberately/ placed or used some programs in /usr/bin instead /bin and
> requiring that /usr bee in the root partition.

The udev team has nothing to do with the /usr mount requirement. Lot of
packages hooked themselves via udev while they had binaries or
dependencies in /usr.

> I will note that the historical separation of the /usr stems from the
> days of user home directories being in /usr/home instead of /home. It
> is getting to the point that the security aspects of having a read-only
> mount for userspace executables is being overridden by developer fiat.

It's a joke?

--
Nicolas Sebrecht

Michał Górny schrieb:
> On Wed, 04 Jan 2012 01:47:38 +0100
> Thomas Sachau <tommy@gentoo.org> wrote:
>
>> 2. switching from udev to mdev (avoids required /usr of udev)
>> 3. some wrapper script to mount /usr before udev starts
>
> These two should be really discouraged as a cheap, temporary solution.
> We should not support hate-admining. I personally think that busybox is
> ready to go into /usr even earlier than udev.

Please give us a bit more than just your opinion.

Why do you see mdev as a temporary solution?

And this part was not about the movement to /usr at all, so why do you
suggest another movement here? And while you answer that, please also
tell us, why you want to migrate packages to a different install
location without a need.

>
>> For the idea of complete migration to /usr, i see no reason to go this
>> route in advance. Just keep with our default install locations and
>> follow upstream, if and where needed.
>
> What about upstreams who do not care? In other words, all those
> packages which we hack to install into rootfs?

They install and work fine, so just keep it this way. I did not see any
argument to move packages around, that work well and have no issue with
their current install location.


--

Thomas Sachau
Gentoo Linux Developer

On 3 January 2012 15:21, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote
>
>> I see three options:
>>
>> 1) Start migrating packages along with upstream and have everyone who
>> has a separate /usr (including me by the way) start using an initramfs
>> of some kind, either dracut or one that we generate specifically for
>> gentoo. The reason I suggest the initramfs, is, unfortunately if we
>> migrate everything, nothing else would work.
>>
>> 2) Combine the sbin and bin directories both  on the root
>> filesystem and in /usr by moving things from /sbin to /bin and /usr/sbin
>> to /usr/bin.
>>
>> 3) Try to maintain  things the way they are as long as possible.
>
>  4) Following pointers from Zac Medico and others, I've managed to get
> Gentoo running with busybox's mdev, instead of udev.  See
> http://archives.gentoo.org/gentoo-user/msg_bc91b392ee0f76376104591cdf7dc5f0.xml
> Executive summary... look Ma; no udev!

Does mdev support all the rules we have in /lib/udev/rules.d/? The
Internet is surprisingly mute on this subject, but a quick grep
through the busybox source doesn't turn up anything that suggests that
it might.

--
Arun Raghavan
http://arunraghavan.net/
(Ford_Prefect | Gentoo) & (arunsr | GNOME)

On Wed, Jan 4, 2012 at 7:58 AM, Arun Raghavan <ford_prefect@gentoo.org> wrote:
> Does mdev support all the rules we have in /lib/udev/rules.d/? The
> Internet is surprisingly mute on this subject, but a quick grep
> through the busybox source doesn't turn up anything that suggests that
> it might.

I think the main use case for mdev is to do a one-time creation of
typical device nodes with minimal use of resources. Perhaps you might
say mdev is to udev as dash is to bash (though dash is
syntax-compatible with bash, or at least it aims to be, and I'm not
sure the same is true of mdev vs udev).

If you're running a server or embedded device and you just need it to
detect your hard drives and maybe a few devices you're willing to
write scripts for, then it is a perfect choice. I have no idea how
well it supports hotplugging of usb devices and such.

For a desktop - it seems like a poor choice. By the time you enhanced
it to do everything udev does you'll ruin it for embedded use and
probably be stuck with all the same issues we have with udev. Fork
udev if you must (good luck with that), but I don't really see mdev as
being a real competitor.

By all means write up an mdev howto and link it in the embedded guide
or if enough users are passionate about it perhaps even link it in the
handbook (as an alternative for adventurous users with special needs).
However, I just can't see it ever becoming the default on a
general-purpose distro like Gentoo (which aims to be all things to all
people as much as is supportable). Certainly it is in the spirit of
Gentoo to support it as an option for those willing to deal with the
downsides (don't expect your bluetooth keyboard to work automagically,
etc).

Rich

On Wed, Jan 4, 2012 at 6:43 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Wed, Jan 4, 2012 at 7:58 AM, Arun Raghavan <ford_prefect@gentoo.org> wrote:
>> Does mdev support all the rules we have in /lib/udev/rules.d/? The
>> Internet is surprisingly mute on this subject, but a quick grep
>> through the busybox source doesn't turn up anything that suggests that
>> it might.
>
> I think the main use case for mdev is to do a one-time creation of
> typical device nodes with minimal use of resources.

In that case, you don't need a userspace daemon at all. Just use
devtmpfs. That'll use even lower resources.

--
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Michał Górny wrote:

> On Sun, 1 Jan 2012 08:53:26 +0000
> Sven Vermeulen <swift@gentoo.org> wrote:
>
>> On Sat, Dec 31, 2011 at 07:59:47PM -0600, William Hubbs wrote:
>> > The goal is to deprecate /bin, /lib, /sbin and /usr/sbin. My
>> > understanding is that they want to move software that is installed
>> > in /bin, /sbin and /usr/sbin to /usr/bin. Also, they want to move
>> > everything from /lib to /usr/lib.
>>
>> I don't like this one bit. Things used to be simple with the "split"
>> between /bin and /usr/bin (and its related directories), this isn't
>> going to make it more simple.
>
> Simple? Should I start requesting additional packages moved into rootfs
> because I feel like needing them? Things can and will go more ugly,
> and I wouldn't be surprised if anytime soon people will start
> complaining because they ran out of space on their rootfs.
>
Well, it is conceptually quite simple: if it's needed in single-user mode to
get your system up and running again, it belongs in rootfs, and if it isn't,
then leave it in user-land.

The thing I don't understand is why it is necessary to move stuff from /bin
to /usr/bin. After all, if you're running the "approved" setup you don't
have a separate /usr so all the binaries are available from the get-go.

What does moving them enable that can't be done now?


Sure, if you have binaries in /bin that link to libraries in /usr/lib that
could be an issue, but only if you're running with a separate /usr and don't
have it mounted when udev starts. So again, not the "approved" setup, and
something you as an admin already have to deal with by making sure /usr is
mounted when udev starts (either via an initramfs, or by a tweak to udev
startup scripts[1].)

wrt GNU coreutils installing to /usr by default, that's so of every GNU
package, since they default to a prefix of /usr/local and it's up to a
distro (or the end-user) to configure them differently; in general the
package assumes it's an addition to the system, unless told otherwise.

(Additionally I'd say that binaries installed to /bin that require libraries
installed to /usr is a bug, but something that should be dealt with
separately. Though with the direction people seem to think is needed, I'm
not sure how much effort anyone will put into that.)

[1] http://forums.gentoo.org/viewtopic-p-6866484.html
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

On Wed, Jan 4, 2012 at 8:50 AM, Steven J Long
<slong@rathaus.eclipse.co.uk> wrote:
> The thing I don't understand is why it is necessary to move stuff from /bin
> to /usr/bin. After all, if you're running the "approved" setup you don't
> have a separate /usr so all the binaries are available from the get-go.

Where is this approved setup documented? Consider guides like this one:
http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml

While you're fixing that you might want to write up an "easy migration
guide" for anybody who followed our official docs in "the past" (with
"the past" including up to the moment that the raid+lvm guide is
updated)...

> Sure, if you have binaries in /bin that link to libraries in /usr/lib that
> could be an issue, but only if you're running with a separate /usr and don't
> have it mounted when udev starts. So again, not the "approved" setup, and
> something you as an admin already have to deal with by making sure /usr is
> mounted when udev starts (either via an initramfs, or by a tweak to udev
> startup scripts[1].)

Well, it is hard to think of a meaningful raid+lvm configuration that
doesn't require an initramfs of some sort with the dependence on files
in /usr during boot. So, getting our initramfs options improved and
supporting this configuration just makes sense regardless before we
unmask newer versions of udev.

Raid+lvm isn't exactly an unusual use-case. Many distros actually use
at least lvm by default now.

Rich

Rich Freeman wrote:

> On Wed, Jan 4, 2012 at 8:50 AM, Steven J Long
> <slong@rathaus.eclipse.co.uk> wrote:
>> The thing I don't understand is why it is necessary to move stuff from
>> /bin to /usr/bin. After all, if you're running the "approved" setup you
>> don't have a separate /usr so all the binaries are available from the
>> get-go.
>
> Where is this approved setup documented?

I could swear we were told in prior discussions on this list that a separate
/usr partition is not considered supported by upstream udev, but searching
all I can find is that an initramfs is required.[1]

Having read the other page[2] that has been pointed out, the answer to my
question "what does this enable that we can't do currently?" is:
snapshotting the OS by backing up just the /usr partition.

I can see the attraction in that, especially for organisations. Though it
does make me smile that it depends on having a separate /usr partition to
work. ;)

The whole saga has just seemed confused to me: one minute a separate /usr is
a terrible idea, the next we have to move every binary to /usr in order to
snapshot a separate /usr. Loath as I am to agree with him, I have to concur
with Ciaran McCreesh that this is "a case of carelessly letting the horse
escape and then trying to convince everyone that no-one needs a horse
anyway."[3]

> Well, it is hard to think of a meaningful raid+lvm configuration that
> doesn't require an initramfs of some sort with the dependence on files
> in /usr during boot. So, getting our initramfs options improved and
> supporting this configuration just makes sense regardless before we
> unmask newer versions of udev.
>
I was under the impression that anyone using lvm+raid (+luks) on root
already has an initramfs, and there are docs out there about that, but sure,
improving those docs and the software is always a good idea.

> Raid+lvm isn't exactly an unusual use-case. Many distros actually use
> at least lvm by default now.
>
Yeah I've been using lvm for several years now, with a separate /usr and no
initramfs, though not on root. For the last few months, I've been running
with the tweaked udev startup scripts I mentioned before, so /usr is mounted
before udev starts (which is possible since I don't have any requirement on
udev-initialised hardware to mount local drives.)

Regardless of the ability to backup just /usr, I'm still not convinced about
moving every binary there. It certainly isn't necessary, in that the
packages we install respect prefix, and there's no need to change the
ebuilds to make packages work; further most admins already have their own
backup scripts in-place.

I for one, would like to be able to run in single-user mode off just the
rootfs, in case for instance, something goes wrong with lvm and /usr won't
mount; and I don't want to duplicate all those utilities in an initramfs. If
that's not going to be possible, fair enough: that's life.

[1] http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
[2] http://fedoraproject.org/wiki/Features/UsrMove
[3] http://article.gmane.org/gmane.linux.gentoo.devel/72130
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

On Wed, Jan 4, 2012 at 10:19 AM, Steven J Long
<slong@rathaus.eclipse.co.uk> wrote:
> I was under the impression that anyone using lvm+raid (+luks) on root
> already has an initramfs, and there are docs out there about that, but sure,
> improving those docs and the software is always a good idea.

Anybody running root on lvm+raid(+luks) is already using an initramfs.
However, the guide I linked does not run root on lvm/luks - just on
raid1, which does not require an initramfs. That guide puts
everything BUT root on LVM, assuming that baselayout/sysvinit/openrc
on root can mount everything else.

The existing initramfs tools do not yet mount /usr regardless. I'm
sure RedHat and others pursing a read-only /usr required at time of
boot will be looking to rectify this, so Gentoo is moving along with
upstream for dracut/etc here.

> Yeah I've been using lvm for several years now, with a separate /usr and no
> initramfs, though not on root. For the last few months, I've been running
> with the tweaked udev startup scripts I mentioned before, so /usr is mounted
> before udev starts (which is possible since I don't have any requirement on
> udev-initialised hardware to mount local drives.)

That's basically my configuration as well. However, I don't really
want to set the goal of making Gentoo support my own configuration.
In any case, having a more robust initramfs solution for separate /usr
should support this and many other configurations.

>
> Regardless of the ability to backup just /usr, I'm still not convinced about
> moving every binary there. It certainly isn't necessary, in that the
> packages we install respect prefix, and there's no need to change the
> ebuilds to make packages work; further most admins already have their own
> backup scripts in-place.

I think that the path of least resistance is to move with upstream -
let's not try to force everything into /usr unless that becomes a
consensus with many other distros. However, we can at least look to
accomodate things like udev that are moving in this direction so that
it doesn't create sparks within Gentoo.

>
> I for one, would like to be able to run in single-user mode off just the
> rootfs, in case for instance, something goes wrong with lvm and /usr won't
> mount; and I don't want to duplicate all those utilities in an initramfs. If
> that's not going to be possible, fair enough: that's life.
>

I hear you. No harm in supporting this as long as seems reasonable,
but you're going to be stuck without udev from the sound of things,
and that is a lot to lose.

You should give dracut a shot - it does quite a bit automagically
though for some reason it doesn't set up my raid properly (I can do it
with a simple mdadm --assemble --scan from the dracut shell), and it
doesn't mount /usr yet. So, it isn't there, but it does quite a bit
just the same.

Rich

On Wed, 04 Jan 2012 13:06:11 +0100
Thomas Sachau <tommy@gentoo.org> wrote:

> Michał Górny schrieb:
> > On Wed, 04 Jan 2012 01:47:38 +0100
> > Thomas Sachau <tommy@gentoo.org> wrote:
> >
> >> 2. switching from udev to mdev (avoids required /usr of udev)
> >> 3. some wrapper script to mount /usr before udev starts
> >
> > These two should be really discouraged as a cheap, temporary
> > solution. We should not support hate-admining. I personally think
> > that busybox is ready to go into /usr even earlier than udev.
>
> Please give us a bit more than just your opinion.
>
> Why do you see mdev as a temporary solution?

Because we will then return to this discussion at some later point
and people will start throwing excrements at us again. So let's be done
with this at once.

> And this part was not about the movement to /usr at all, so why do you
> suggest another movement here? And while you answer that, please also
> tell us, why you want to migrate packages to a different install
> location without a need.

Because we need to finally be able to fix mistakes made in the past
by other people.

> >> For the idea of complete migration to /usr, i see no reason to go
> >> this route in advance. Just keep with our default install
> >> locations and follow upstream, if and where needed.
> >
> > What about upstreams who do not care? In other words, all those
> > packages which we hack to install into rootfs?
>
> They install and work fine, so just keep it this way. I did not see
> any argument to move packages around, that work well and have no
> issue with their current install location.

What if, say, upstream introduces pkg-config file where our hacks will
cause it to be installed into /lib/pkgconfig? Should we then expand
the hack to cover that, and something else, and then another thing...

--
Best regards,
Michał Górny

On Wed, 04 Jan 2012 13:50:45 +0000
Steven J Long <slong@rathaus.eclipse.co.uk> wrote:

> (Additionally I'd say that binaries installed to /bin that require
> libraries installed to /usr is a bug, but something that should be
> dealt with separately. Though with the direction people seem to think
> is needed, I'm not sure how much effort anyone will put into that.)

/bin/bsdcpio
libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x00007fa3c89e3000)
liblzma.so.5 => /usr/lib64/liblzma.so.5 (0x00007fa3c7be9000)
libcrypto.so.1.0.0 => /usr/lib64/libcrypto.so.1.0.0
(0x00007fa3c7418000)
/bin/expr
libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f7cf26cc000)
/bin/findmnt
libmount.so.1 => /usr/lib64/libmount.so.1 (0x00007fa23d614000)
/bin/grub2-mkfont
libfreetype.so.6 => /usr/lib64/libfreetype.so.6
(0x00007f1007a4c000)
/bin/grub2-mkimage
liblzma.so.5 => /usr/lib64/liblzma.so.5 (0x00007fa60d479000)
/bin/lowntfs-3g
libfuse.so.2 => /usr/lib64/libfuse.so.2 (0x00007f912ed48000)
/bin/mail
liblockfile.so.1 => /usr/lib64/liblockfile.so.1
(0x00007fb8e249c000)
/bin/systemctl
libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3 (0x00007f6cfdf4c000)
/sbin/audisp-remote
libcap-ng.so.0 => /usr/lib64/libcap-ng.so.0 (0x00007fbd76aae000)
/sbin/irqbalance
libcap-ng.so.0 => /usr/lib64/libcap-ng.so.0 (0x00007fc5b8eb3000)
libglib-2.0.so.0 => /usr/lib64/libglib-2.0.so.0
(0x00007fc5b8b92000)
/sbin/rpcbind
libgssglue.so.1 => /usr/lib64/libgssglue.so.1
(0x00007fb209614000)

(and I tried not to list more than one file from a single package)

The list could be longer if I enabled more USE flags, I think. And for
example, I could see benefits of having sshd on rootfs as well, if we
kept the old definition of it.

--
Best regards,
Michał Górny

On Wed, 4 Jan 2012 16:51:12 +0100
Michał Górny <mgorny@gentoo.org> wrote:
> /bin/systemctl
> libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3

Here is a prime example of why "vertical integration" should really be
called "a horrible mess of tight coupling"...

Remember how people used to make fun of Windows when it would fail to
boot if you broke Internet Explorer?

--
Ciaran McCreesh

On Wed, 4 Jan 2012 15:54:07 +0000
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Wed, 4 Jan 2012 16:51:12 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> > /bin/systemctl
> > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3

Considering that I really thought about stripping that one because
otherwise people will not even notice the other page of results.


--
Best regards,
Michał Górny

On 04-01-2012 16:37:34 +0100, Michał Górny wrote:
> > And this part was not about the movement to /usr at all, so why do you
> > suggest another movement here? And while you answer that, please also
> > tell us, why you want to migrate packages to a different install
> > location without a need.
>
> Because we need to finally be able to fix mistakes made in the past
> by other people.

What mistakes?

> > They install and work fine, so just keep it this way. I did not see
> > any argument to move packages around, that work well and have no
> > issue with their current install location.
>
> What if, say, upstream introduces pkg-config file where our hacks will
> cause it to be installed into /lib/pkgconfig? Should we then expand
> the hack to cover that, and something else, and then another thing...

Highly unlikely, but if it happens, easy to fix, so not really a
convincing issue.


--
Fabian Groffen
Gentoo on a different level

On Wed, 4 Jan 2012 17:33:15 +0100
Fabian Groffen <grobian@gentoo.org> wrote:

> On 04-01-2012 16:37:34 +0100, Michał Górny wrote:
> > > And this part was not about the movement to /usr at all, so why
> > > do you suggest another movement here? And while you answer that,
> > > please also tell us, why you want to migrate packages to a
> > > different install location without a need.
> >
> > Because we need to finally be able to fix mistakes made in the past
> > by other people.
>
> What mistakes?

The mistake of introducing a pointless separation based on a rule
of thumb which becomes more and more blurry over time, and hacking
packages just to make it work.

--
Best regards,
Michał Górny

>>>>> On Wed, 4 Jan 2012, Micha³ Górny wrote:

>> What mistakes?

> The mistake of introducing a pointless separation based on a rule of
> thumb which becomes more and more blurry over time, and hacking
> packages just to make it work.

There's really nothing pointless or blurry about this separation.
The FHS has a nice definition: "The contents of the root filesystem
must be adequate to boot, restore, recover, and/or repair the system."

Ulrich

On Wed, 2012-01-04 at 18:12 +0100, Ulrich Mueller wrote:
> >>>>> On Wed, 4 Jan 2012, Michał Górny wrote:
>
> >> What mistakes?
>
> > The mistake of introducing a pointless separation based on a rule of
> > thumb which becomes more and more blurry over time, and hacking
> > packages just to make it work.
>
> There's really nothing pointless or blurry about this separation.
> The FHS has a nice definition: "The contents of the root filesystem
> must be adequate to boot, restore, recover, and/or repair the system."

The problem is that to boot a modern system, you need a shitload of
stuff. For example, modern network filesystems often have secure
authentication and probably LDAP too, so that means we need to move ldap
and openssl into / and all the dependencies. Also, anything that
installs a udev rule needs to be in /, and the list goes on an on. Very
soon, you have almost everything in /...

This rule made sense in the 80s, but it doesn't match the modern world
anymore.

Some longer explanations:
http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
https://fedoraproject.org/wiki/Features/UsrMove

Here is a list of packages on your system that will break if you start
udev without /usr mounted:

egrep 'usb-db|pci-db|FROM_DATABASE|/usr' /*/udev/rules.d/* |cut -f 1
-d : | sort -u | xargs qfile -e


--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> On Wed, 4 Jan 2012 16:51:12 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> > /bin/systemctl
> > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
>
> Here is a prime example of why "vertical integration" should really be
> called "a horrible mess of tight coupling"...

You clearly have failed to realize that d-bus is a now the bus for
system messaging and is as much part of the system as syslog or bash.
Probably even more so, for example, in Fedora 17, you'll be able to boot
without syslog or bash, but you need d-bus.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Wed, 04 Jan 2012 12:40:10 -0500
Olivier Crête <tester@gentoo.org> wrote:
> On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> > On Wed, 4 Jan 2012 16:51:12 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > > /bin/systemctl
> > > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
> >
> > Here is a prime example of why "vertical integration" should really
> > be called "a horrible mess of tight coupling"...
>
> You clearly have failed to realize that d-bus is a now the bus for
> system messaging and is as much part of the system as syslog or bash.
> Probably even more so, for example, in Fedora 17, you'll be able to
> boot without syslog or bash, but you need d-bus.

No, I realise full well that some GnomeOS developers would like us to
think that HAL, D-BUS, network-manager, udev-extras etc are part of the
system, and are sloppily writing code that makes that assumption.
However, the question ultimately under discussion is whether Gentoo is
to be a Linux distribution or a GnomeOS distribution.

--
Ciaran McCreesh

On Wed, Jan 04, 2012 at 03:19:57PM +0000, Steven J Long wrote:
> I could swear we were told in prior discussions on this list that a separate
> /usr partition is not considered supported by upstream udev, but searching
> all I can find is that an initramfs is required.[1]
The upstream statement was more specifically that: starting udev (or
systemd) without /usr available was not considered supported.

If /usr is on a separate partition, this is forcing us down the
initramfs route (If fsck ends up on /usr, the only way to fsck /usr is
from an extra copy in the initramfs...).

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

* Olivier Crête schrieb am 04.01.12 um 18:32 Uhr:
> On Wed, 2012-01-04 at 18:12 +0100, Ulrich Mueller wrote:
> > >>>>> On Wed, 4 Jan 2012, Michał Górny wrote:
> >
> > >> What mistakes?
> >
> > > The mistake of introducing a pointless separation based on a rule of
> > > thumb which becomes more and more blurry over time, and hacking
> > > packages just to make it work.
> >
> > There's really nothing pointless or blurry about this separation.
> > The FHS has a nice definition: "The contents of the root filesystem
> > must be adequate to boot, restore, recover, and/or repair the system."
>
> The problem is that to boot a modern system, you need a shitload of
> stuff.

To boot the system on its highest level: yes. But Linux/UNIX systems
have a concept called runlevels that can perfectly cover cases where
this "shitload of stuff" is not required.

For example, to make that FHS definition be reality there are (can
be) runlevels that will only boot a system with all basic stuff
required to mount the rootfs and make root being able to login to
the local text console. These are the things that make a unixoid
system valuable over other kind of systems.

> For example, modern network filesystems often have secure
> authentication and probably LDAP too, so that means we need to move ldap
> and openssl into / and all the dependencies. Also, anything that
> installs a udev rule needs to be in /, and the list goes on an on. Very
> soon, you have almost everything in /...

You do not need everything to make a system boot some sort of
recovery-console for example.

>
> This rule made sense in the 80s, but it doesn't match the modern world
> anymore.

Why? The benefits to keep a system bootable and repairable is one of
the reasons why unix systems are more robust or can better be
repeaired than, lets say windows systems for example.

I do not like the idea to throw away all those benefits just because
so many (younger/newer) people do not know about the possibilities
an "old fashioned" unix system tends to have.

-Marc
--
8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134

2012/1/5 Ulrich Mueller <ulm@gentoo.org>
>
>>
> There's really nothing pointless or blurry about this separation.
> The FHS has a nice definition: "The contents of the root filesystem
> must be adequate to boot, restore, recover, and/or repair the system."
>

Given that these tools are being moved to /usr and/or duplicated to in
initrd , what is the point of a root filesystem anyway now? Just to
mount other things on? Just to store /etc ?

Or will /etc move to /usr too?

/usr/etc somewhat horrifies me.

And if you no longer have a suite of recovery tools on root, you
*have* to really have a copy in initrd, otherwise when /usr gets
damaged and needs repaired/recovered, you'll need a boot disk just to
solve that problem. And that I don't fancy.

And another errant thought: why not just repurpose the initrd as "the
root filesystem" if the root filesystem is just to exist for the
purpose of bolting other stuff on.

Because in my mind, the primary benefit of initrd over an actual
filesystem is the initrd is theoretically a lot harder to mess up, and
you can easily have a plethora of alternative known-good initrd's to
fall back on.

--
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

* Olivier Crête schrieb am 04.01.12 um 18:40 Uhr:
> On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> > On Wed, 4 Jan 2012 16:51:12 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> > > /bin/systemctl
> > > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
> >
> > Here is a prime example of why "vertical integration" should really be
> > called "a horrible mess of tight coupling"...
>
> You clearly have failed to realize that d-bus is a now the bus for
> system messaging and is as much part of the system as syslog or bash.
> Probably even more so, for example, in Fedora 17, you'll be able to boot
> without syslog or bash, but you need d-bus.

If this was true, we will soon have problems with linux systems that
windows had in 1995.

IMO a system should *always* be bootable without that "high level"
stuff. And by bootable I mean that you can get a root prompt at
least.

-Marc
--
8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134

On Thu, 5 Jan 2012 07:27:49 +1300
Kent Fredric <kentfredric@gmail.com> wrote:

> 2012/1/5 Ulrich Mueller <ulm@gentoo.org>
> >
> >>
> > There's really nothing pointless or blurry about this separation.
> > The FHS has a nice definition: "The contents of the root filesystem
> > must be adequate to boot, restore, recover, and/or repair the
> > system."
> >
>
> Given that these tools are being moved to /usr and/or duplicated to in
> initrd , what is the point of a root filesystem anyway now? Just to
> mount other things on? Just to store /etc ?

Well, you can either keep both /etc and /usr on a single filesystem, or
move /etc out of rootfs and just make it a tmpfs.

> And if you no longer have a suite of recovery tools on root, you
> *have* to really have a copy in initrd, otherwise when /usr gets
> damaged and needs repaired/recovered, you'll need a boot disk just to
> solve that problem. And that I don't fancy.

And if / gets damaged, keeping those tools on / doesn't help either.
If you have them on initramfs, they can fix it as well. Of course we
could go onto 'what if initramfs gets damaged?' but then you're HDD got
damaged as well...

> And another errant thought: why not just repurpose the initrd as "the
> root filesystem" if the root filesystem is just to exist for the
> purpose of bolting other stuff on.

Noone forbids you to. But then you won't get your memory back when real
system boots.

--
Best regards,
Michał Górny

Michał Górny schrieb:
> On Wed, 04 Jan 2012 13:06:11 +0100
> Thomas Sachau <tommy@gentoo.org> wrote:
>
>> Michał Górny schrieb:
>>> On Wed, 04 Jan 2012 01:47:38 +0100
>>> Thomas Sachau <tommy@gentoo.org> wrote:
>>>
>>>> 2. switching from udev to mdev (avoids required /usr of udev)
>>>> 3. some wrapper script to mount /usr before udev starts
>>>
>>> These two should be really discouraged as a cheap, temporary
>>> solution. We should not support hate-admining. I personally think
>>> that busybox is ready to go into /usr even earlier than udev.
>>
>> Please give us a bit more than just your opinion.
>>
>> Why do you see mdev as a temporary solution?
>
> Because we will then return to this discussion at some later point
> and people will start throwing excrements at us again. So let's be done
> with this at once.

Please tell me, how a replacement for udev, which in the end removes the
requirement for mounted /usr at boot time, should later require a
mounted /usr again.
And please dont tell me, that this will happen because you moved
everything to /usr. This is something you would like to do and wish to
see, but i dont see it happen.

>
>> And this part was not about the movement to /usr at all, so why do you
>> suggest another movement here? And while you answer that, please also
>> tell us, why you want to migrate packages to a different install
>> location without a need.
>
> Because we need to finally be able to fix mistakes made in the past
> by other people.

This has already been commented on by grobian and ulm, so i see no need
to dublicate their lines.

>>>> For the idea of complete migration to /usr, i see no reason to go
>>>> this route in advance. Just keep with our default install
>>>> locations and follow upstream, if and where needed.
>>>
>>> What about upstreams who do not care? In other words, all those
>>> packages which we hack to install into rootfs?
>>
>> They install and work fine, so just keep it this way. I did not see
>> any argument to move packages around, that work well and have no
>> issue with their current install location.
>
> What if, say, upstream introduces pkg-config file where our hacks will
> cause it to be installed into /lib/pkgconfig? Should we then expand
> the hack to cover that, and something else, and then another thing...

Defining a prefix is no "hack", it is an option you can use.

Anyway, we both have probably enough packages with such a "hack"
installed, but i cannot find a single file in /lib/pkgconfig, not even
that dir does exist. Is it different on your system?
If not, then please tell me, why you create some theory about possible
issues, which dont even exist. Dont you have better arguments for your
suggested move to /usr?


--

Thomas Sachau
Gentoo Linux Developer

On Wed, 4 Jan 2012 18:12:18 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Wed, 4 Jan 2012, Michał Górny wrote:
>
> >> What mistakes?
>
> > The mistake of introducing a pointless separation based on a rule of
> > thumb which becomes more and more blurry over time, and hacking
> > packages just to make it work.
>
> There's really nothing pointless or blurry about this separation.
> The FHS has a nice definition: "The contents of the root filesystem
> must be adequate to boot, restore, recover, and/or repair the system."

Why don't we have sshd there then? I don't really feel like repairing
remote system without fallback sshd.

And a compiler. If I mess up some important system component, I'd really
use one. And package manager. And backup system libraries...

--
Best regards,
Michał Górny

On Thu, 2012-01-05 at 07:27 +1300, Kent Fredric wrote:
> 2012/1/5 Ulrich Mueller <ulm@gentoo.org>
> >
> >>
> > There's really nothing pointless or blurry about this separation.
> > The FHS has a nice definition: "The contents of the root filesystem
> > must be adequate to boot, restore, recover, and/or repair the system."
> >
>
> Given that these tools are being moved to /usr and/or duplicated to in
> initrd , what is the point of a root filesystem anyway now? Just to
> mount other things on? Just to store /etc ?
>
> Or will /etc move to /usr too?
>
> /usr/etc somewhat horrifies me.

No no no, the idea is that once all binaries are in /usr, you can easily
share /usr between different systems and do updates in a sane way.. You
can also mount /usr read-only, but still have / be read-write.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Wed, 2012-01-04 at 19:30 +0100, Marc Schiffbauer wrote:
> * Olivier Crête schrieb am 04.01.12 um 18:40 Uhr:
> > On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> > > On Wed, 4 Jan 2012 16:51:12 +0100
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > /bin/systemctl
> > > > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
> > >
> > > Here is a prime example of why "vertical integration" should really be
> > > called "a horrible mess of tight coupling"...
> >
> > You clearly have failed to realize that d-bus is a now the bus for
> > system messaging and is as much part of the system as syslog or bash.
> > Probably even more so, for example, in Fedora 17, you'll be able to boot
> > without syslog or bash, but you need d-bus.
>
> If this was true, we will soon have problems with linux systems that
> windows had in 1995.
>
> IMO a system should *always* be bootable without that "high level"
> stuff. And by bootable I mean that you can get a root prompt at
> least.

d-bus is not high-level stuff... For example, you can't use bluetooth
without d-bus. Even Android has it..

That said, in the new systemd world, bash is.. Since it's only a "UI"
tools (just like gnome-shell for example). Since you don't need it to
boot.


--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Wed, Jan 4, 2012 at 1:27 PM, Kent Fredric <kentfredric@gmail.com> wrote:
> Given that these tools are being moved to /usr and/or duplicated to in
> initrd , what is the point of a root filesystem anyway now? Just to
> mount other things on? Just to store /etc ?
>
> Or will /etc move to /usr too?

I'd recommend reading the fedora docs. Their plan is to make /usr
read-only so that it contains all elements of the system managed by
the distro. In the future rpm world config files exist half on /usr,
with overriding content in /etc (they don't have etc-update, and
etc-update isn't always perfect either).

But yes, the trend is towards making rootfs a bit more "virtual."

I can see some of the benefits of this arrangement, but by the time we
get that all worked out btrfs might be practical, and its subvolumes
actually solve many of the problems that lvm and many partitions are
used to solve today. With btrfs you can make /usr a subvolume and
snapshot it at will, or set up a quota just for it. That doesn't
cover all the use cases, but it does cover most of the desktop-y ones.

As far as repairing the system from rootfs goes - I think that greatly
depends on your circumstances. If everything is on root anyway then
it is a moot point. If everything isn't on root then your ability to
recover is inversely proportional to the complexity of your systems.
As others have pointed out, there is always something that you won't
have, and to be honest it isn't all that hard to just boot a liveDVD
that has everything and the kitchen sink available anyway.

Rich

On 04-01-2012 19:50:24 +0100, Michał Górny wrote:
> On Wed, 4 Jan 2012 18:12:18 +0100
> Ulrich Mueller <ulm@gentoo.org> wrote:
>
> > >>>>> On Wed, 4 Jan 2012, Michał Górny wrote:
> >
> > >> What mistakes?
> >
> > > The mistake of introducing a pointless separation based on a rule of
> > > thumb which becomes more and more blurry over time, and hacking
> > > packages just to make it work.
> >
> > There's really nothing pointless or blurry about this separation.
> > The FHS has a nice definition: "The contents of the root filesystem
> > must be adequate to boot, restore, recover, and/or repair the system."
>
> Why don't we have sshd there then? I don't really feel like repairing
> remote system without fallback sshd.

Network isn't typically in that bootlevel. You'd just attach through
the console (netmgt, ipmi, keyboard/vga) instead.

> And a compiler. If I mess up some important system component, I'd really
> use one. And package manager. And backup system libraries...

Time for your PXE boot from net to just bring back a sane image or so.


--
Fabian Groffen
Gentoo on a different level

On 04-01-2012 13:51:26 -0500, Olivier Crête wrote:
> No no no, the idea is that once all binaries are in /usr, you can easily
> share /usr between different systems and do updates in a sane way.. You
> can also mount /usr read-only, but still have / be read-write.

http://article.gmane.org/gmane.linux.debian.devel.general/165891


--
Fabian Groffen
Gentoo on a different level

On Wed, 04 Jan 2012 19:48:03 +0100
Thomas Sachau <tommy@gentoo.org> wrote:

> Defining a prefix is no "hack", it is an option you can use.
>
> Anyway, we both have probably enough packages with such a "hack"
> installed, but i cannot find a single file in /lib/pkgconfig, not even
> that dir does exist. Is it different on your system?

Defining a prefix is no hack. Defining a prefix would result in
existence of such a file, and installation of static libraries in /lib.

We use hacks to move shared libraries to rootfs, and then create one
more hack to not confuse the linker with different locations of static
and shared libraries.

--
Best regards,
Michał Górny

On Wed, 4 Jan 2012 20:00:51 +0100
Fabian Groffen <grobian@gentoo.org> wrote:

> On 04-01-2012 19:50:24 +0100, Michał Górny wrote:
> > On Wed, 4 Jan 2012 18:12:18 +0100
> > Ulrich Mueller <ulm@gentoo.org> wrote:
> >
> > > >>>>> On Wed, 4 Jan 2012, Michał Górny wrote:
> > >
> > > >> What mistakes?
> > >
> > > > The mistake of introducing a pointless separation based on a
> > > > rule of thumb which becomes more and more blurry over time, and
> > > > hacking packages just to make it work.
> > >
> > > There's really nothing pointless or blurry about this separation.
> > > The FHS has a nice definition: "The contents of the root
> > > filesystem must be adequate to boot, restore, recover, and/or
> > > repair the system."
> >
> > Why don't we have sshd there then? I don't really feel like
> > repairing remote system without fallback sshd.
>
> Network isn't typically in that bootlevel. You'd just attach through
> the console (netmgt, ipmi, keyboard/vga) instead.
>
> > And a compiler. If I mess up some important system component, I'd
> > really use one. And package manager. And backup system libraries...
>
> Time for your PXE boot from net to just bring back a sane image or so.

My PXE boot from net won't happen because possible /usr-over-NFS relies
on random files from other rootfs, and they just failed to be in sync
between two of my systems.

--
Best regards,
Michał Górny

On 04-01-2012 20:28:01 +0100, Michał Górny wrote:
> > > And a compiler. If I mess up some important system component, I'd
> > > really use one. And package manager. And backup system libraries...
> >
> > Time for your PXE boot from net to just bring back a sane image or so.
>
> My PXE boot from net won't happen because possible /usr-over-NFS relies
> on random files from other rootfs, and they just failed to be in sync
> between two of my systems.

Seems like you've got a situation where you'd just shove in a livecd
then.


--
Fabian Groffen
Gentoo on a different level

On 04-01-2012 20:26:27 +0100, Michał Górny wrote:
> We use hacks to move shared libraries to rootfs, and then create one
> more hack to not confuse the linker with different locations of static
> and shared libraries.

So your point is that the reasons why this was originally done are now
no longer valid, because...?


--
Fabian Groffen
Gentoo on a different level

On Wed, Jan 04, 2012 at 07:26:05PM +0100, Marc Schiffbauer wrote:
> For example, to make that FHS definition be reality there are (can
> be) runlevels that will only boot a system with all basic stuff
> required to mount the rootfs and make root being able to login to
> the local text console. These are the things that make a unixoid
> system valuable over other kind of systems.

There are benefits to the proposed changes, especially for rpm based
distros and especially for non-server settings. Are they good enough?
No, I don't think they are. But since forking all those packages is
not a realistic proposition, we will have to follow along. We should
try and not break existing installations when we do though.

> I do not like the idea to throw away all those benefits just because
> so many (younger/newer) people do not know about the possibilities
> an "old fashioned" unix system tends to have.

Hey, this is web 2.0 era. Being mostly right most of the time is good
enough.

--
Eray Aslan <eras@gentoo.org>

* Olivier Crête schrieb am 04.01.12 um 19:53 Uhr:
> On Wed, 2012-01-04 at 19:30 +0100, Marc Schiffbauer wrote:
> > * Olivier Crête schrieb am 04.01.12 um 18:40 Uhr:
> > > On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> > > > On Wed, 4 Jan 2012 16:51:12 +0100
> > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > > /bin/systemctl
> > > > > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
> > > >
> > > > Here is a prime example of why "vertical integration" should really be
> > > > called "a horrible mess of tight coupling"...
> > >
> > > You clearly have failed to realize that d-bus is a now the bus for
> > > system messaging and is as much part of the system as syslog or bash.
> > > Probably even more so, for example, in Fedora 17, you'll be able to boot
> > > without syslog or bash, but you need d-bus.
> >
> > If this was true, we will soon have problems with linux systems that
> > windows had in 1995.
> >
> > IMO a system should *always* be bootable without that "high level"
> > stuff. And by bootable I mean that you can get a root prompt at
> > least.
>
> d-bus is not high-level stuff... For example, you can't use bluetooth
> without d-bus. Even Android has it..

And you need bluetooth to boot your stable datacenter server
systems?

> That said, in the new systemd world, bash is.. Since it's only a "UI"
> tools (just like gnome-shell for example). Since you don't need it to
> boot.

Yeah right. Having dbus for bluetooth is much more important than
having a shell.

Please remember that there are *way* more server systems running linux
without any graphical desktop at all than desktop systems.

-Marc
--
8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134

On Wed, 2012-01-04 at 21:45 +0100, Marc Schiffbauer wrote:
>
> > That said, in the new systemd world, bash is.. Since it's only a
> "UI"
> > tools (just like gnome-shell for example). Since you don't need it
> to
> > boot.
>
> Yeah right. Having dbus for bluetooth is much more important than
> having a shell.
>
> Please remember that there are *way* more server systems running
> linux
> without any graphical desktop at all than desktop systems.

Please remember that servers and desktops are dwarfed by the number of
embedded systems running Linux. Probably more devices are sold running
Linux in a single day than the total number of servers in the world...

But well, this isn't a number's game. D-Bus is the system bus and
bluetooth is just one example of a system level component that uses it.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On 01/04/2012 09:32 AM, Olivier Crête wrote:
> On Wed, 2012-01-04 at 18:12 +0100, Ulrich Mueller wrote:
>>>>>>> On Wed, 4 Jan 2012, Michał Górny wrote:
>>
>>>> What mistakes?
>>
>>> The mistake of introducing a pointless separation based on a rule of
>>> thumb which becomes more and more blurry over time, and hacking
>>> packages just to make it work.
>>
>> There's really nothing pointless or blurry about this separation.
>> The FHS has a nice definition: "The contents of the root filesystem
>> must be adequate to boot, restore, recover, and/or repair the system."
>
> The problem is that to boot a modern system, you need a shitload of
> stuff. For example, modern network filesystems often have secure
> authentication and probably LDAP too, so that means we need to move ldap
> and openssl into / and all the dependencies. Also, anything that
> installs a udev rule needs to be in /, and the list goes on an on. Very
> soon, you have almost everything in /...
>
> This rule made sense in the 80s, but it doesn't match the modern world
> anymore.
>
> Some longer explanations:
> http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
> https://fedoraproject.org/wiki/Features/UsrMove

The FHS notion of "root filesystem as a recovery partition" existed long
before the relatively modern development of things like busybox and
initramfs made it more practical to use an initramfs as a recovery
partition. Anyone who wouldn't prefer to use an initramfs for their
"recover partition" probably just doesn't realize how well suited an
initramfs is for the job. It's so well suited for the job that it makes
the old FHS notion of "root filesystem as a recovery partition" seem quaint.
--
Thanks,
Zac

On Sun, 01 Jan 2012 16:49:42 -0500
Olivier Crête <tester@gentoo.org> wrote:

> > > That's why you have dracut to do it for you.

> > Which is keyworded at this point. Stable users do what?

It's keyworded for only two arches.

> This is a discussion about the future... Changing keywords is trivial
> if we care.

Oh, let's quickly drop the notion of arch testing/stabilisation. :)


jer

* Olivier Crête schrieb am 04.01.12 um 22:55 Uhr:
> On Wed, 2012-01-04 at 21:45 +0100, Marc Schiffbauer wrote:
> >
> > > That said, in the new systemd world, bash is.. Since it's only a
> > "UI"
> > > tools (just like gnome-shell for example). Since you don't need it
> > to
> > > boot.
> >
> > Yeah right. Having dbus for bluetooth is much more important than
> > having a shell.
> >
> > Please remember that there are *way* more server systems running
> > linux
> > without any graphical desktop at all than desktop systems.
>
> Please remember that servers and desktops are dwarfed by the number of
> embedded systems running Linux. Probably more devices are sold running
> Linux in a single day than the total number of servers in the world...

Yes, but these do most propably never run some stock distro.

> But well, this isn't a number's game. D-Bus is the system bus and
> bluetooth is just one example of a system level component that uses it.

... which is not really required to boot a system.

-Marc
--
8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134

Jeroen Roovers wrote:
> On Sun, 01 Jan 2012 16:49:42 -0500
> Olivier Crête<tester@gentoo.org> wrote:
>
>>>> That's why you have dracut to do it for you.
>>> Which is keyworded at this point. Stable users do what?
> It's keyworded for only two arches.
>
>

And amd64 is one of them. I'd say it is a fairly popular arch too. ;-)

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Miss the compile output? Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Marc Schiffbauer posted on Wed, 04 Jan 2012 21:45:35 +0100 as excerpted:

> Please remember that there are *way* more server systems running linux
> without any graphical desktop at all than desktop systems.

So with Google activating ~800k android Linux systems a day last I heard,
how do the number of (android) Linux systems (which we've already
established as having dbus) compare to the number of server Linux systems?

If traditional gnu-linux isn't a minority in its own community already,
it soon will be.

I sympathize with the sentiment behind the argument, but the numbers game
really doesn't cut it, or we'd all be running some binary distribution or
other instead of from-source Gentoo and we'd not be having this
discussion as it would have already been had for us. (Yeah, there IS
rather a lot to be read between THOSE lines!)

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Wed, 4 Jan 2012 19:30:07 +0100
Marc Schiffbauer <mschiff@gentoo.org> wrote:

> * Olivier Crête schrieb am 04.01.12 um 18:40 Uhr:
> > On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> > > On Wed, 4 Jan 2012 16:51:12 +0100
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > /bin/systemctl
> > > > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
> > >
> > > Here is a prime example of why "vertical integration" should
> > > really be called "a horrible mess of tight coupling"...
> >
> > You clearly have failed to realize that d-bus is a now the bus for
> > system messaging and is as much part of the system as syslog or
> > bash. Probably even more so, for example, in Fedora 17, you'll be
> > able to boot without syslog or bash, but you need d-bus.
>
> IMO a system should *always* be bootable without that "high level"
> stuff. And by bootable I mean that you can get a root prompt at
> least.

And why do you consider D-Bus being high-level? Just because things
used to reinvent the wheel before in a much worse fashion?

--
Best regards,
Michał Górny

* Michał Górny schrieb am 05.01.12 um 09:26 Uhr:
> On Wed, 4 Jan 2012 19:30:07 +0100
> Marc Schiffbauer <mschiff@gentoo.org> wrote:
>
> > * Olivier Crête schrieb am 04.01.12 um 18:40 Uhr:
> > > On Wed, 2012-01-04 at 15:54 +0000, Ciaran McCreesh wrote:
> > > > On Wed, 4 Jan 2012 16:51:12 +0100
> > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > > /bin/systemctl
> > > > > libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3
> > > >
> > > > Here is a prime example of why "vertical integration" should
> > > > really be called "a horrible mess of tight coupling"...
> > >
> > > You clearly have failed to realize that d-bus is a now the bus for
> > > system messaging and is as much part of the system as syslog or
> > > bash. Probably even more so, for example, in Fedora 17, you'll be
> > > able to boot without syslog or bash, but you need d-bus.
> >
> > IMO a system should *always* be bootable without that "high level"
> > stuff. And by bootable I mean that you can get a root prompt at
> > least.
>
> And why do you consider D-Bus being high-level? Just because things
> used to reinvent the wheel before in a much worse fashion?

I meant "hight-level" only in a way that it is not really needed to
boot the very basic things of a system so that I can get a root
prompt at the console at least. E.g. you do not need dbus to find
and mount the rootfs, fire a getty and shell.

-Marc
--
8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134

On Thu, 2012-01-05 at 12:08 +0100, Marc Schiffbauer wrote:
>
> I meant "hight-level" only in a way that it is not really needed to
> boot the very basic things of a system so that I can get a root
> prompt at the console at least. E.g. you do not need dbus to find
> and mount the rootfs, fire a getty and shell.

Obviously, you can do init=/bin/sh, that's doesn't help you much. I
think we're all speaking of a minimually useful system here.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

On Thu, 2012-01-05 at 12:08 +0100, Marc Schiffbauer wrote:
>
> I meant "hight-level" only in a way that it is not really needed to
> boot the very basic things of a system so that I can get a root
> prompt at the console at least. E.g. you do not need dbus to find
> and mount the rootfs, fire a getty and shell.

Obviously, you can do init=/bin/sh, that's doesn't help you much. I
think we're all speaking of a minimally useful system here.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer

Olivier Crête posted on Thu, 05 Jan 2012 09:31:07 -0500 as excerpted:

> On Thu, 2012-01-05 at 12:08 +0100, Marc Schiffbauer wrote:
>>
>> I meant "hight-level" only in a way that it is not really needed to
>> boot the very basic things of a system so that I can get a root prompt
>> at the console at least. E.g. you do not need dbus to find and mount
>> the rootfs, fire a getty and shell.
>
> Obviously, you can do init=/bin/sh, that's doesn't help you much. I
> think we're all speaking of a minimally useful system here.

But init=/bin/sh (or /bin/bash as I use here) DOES help in a surprising
number of cases as long as the necessary storage and input drivers and
filesystem modules are builtin. And a lot of us have strong ideas about
wanting to keep it that way, being able to use init=/bin/sh on the kernel
command line itself, from grub or whatever.

Some of us even tried lvm and dumped it for precisely that reason: it
requires userspace and thus an initr* if root is on lvm, and without an
lvm managing root, its usefulness is diminished to the point where it's
more trouble than it's worth, especially since md/raid has handled
partitioned RAID very well for quite some time now (a big use case for lvm
originally, since md/raid didn't handle partitioned mds directly, back in
the day), AND unlike lvm, it can be configured on the kernel command line
directly, allowing one to actually get to that init=/bin/sh if necessary.

That's low level. Tell me when init=/usr/bin/dbus-whatever works from
the kernel command line. Until then, system-bus or no-system-bus, it's
not even in the same ball park, or even on the same planet, come to think
of it, level-wise.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Thu, 5 Jan 2012 17:12:26 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> But init=/bin/sh (or /bin/bash as I use here) DOES help in a
> surprising number of cases as long as the necessary storage and input
> drivers and filesystem modules are builtin. And a lot of us have
> strong ideas about wanting to keep it that way, being able to use
> init=/bin/sh on the kernel command line itself, from grub or whatever.

[...]

> That's low level.

Looking at your definition of 'low level', it seems that OpenRC is high
level as well.

--
Best regards,
Michał Górny

On Thu, Jan 05, 2012 at 07:27:49AM +1300, Kent Fredric wrote:
> 2012/1/5 Ulrich Mueller <ulm@gentoo.org>
> >
> >>
> > There's really nothing pointless or blurry about this separation.
> > The FHS has a nice definition: "The contents of the root filesystem
> > must be adequate to boot, restore, recover, and/or repair the system."
> >
>
> Given that these tools are being moved to /usr and/or duplicated to in
> initrd , what is the point of a root filesystem anyway now? Just to
> mount other things on? Just to store /etc ?
>
> Or will /etc move to /usr too?

No, /etc isn't going anywhere.

William

On Thu, 5 Jan 2012 13:30:24 -0600
William Hubbs <williamh@gentoo.org> wrote:
> > Or will /etc move to /usr too?
>
> No, /etc isn't going anywhere.

Are you sure? I heard a rumour that systemd will soon require you to
put /etc inside your initrd (since / can't be mounted without it).
Obviously, you'd have to reboot if you made any changes to your config
files, but that's OK since you can't safely restart daemons anyway.

--
Ciaran McCreesh

On Thu, Jan 5, 2012 at 3:08 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> Are you sure? I heard a rumour that systemd will soon require you to
> put /etc inside your initrd (since / can't be mounted without it).

While I can't speak to your comments about being unable to restart
daemons with systemd (hope this isn't the case, obviously), dracut
does in fact include a copy of some files in /etc like mdadm.conf.
So, if you reconfigure your raid it might be beneficial to rebuild
your initramfs.

As you might expect that is optional - mdadm can more-or-less work
without mdadm.conf, but in some cases you could have your raids change
name and such. If you mount root by UUID that won't prevent you from
booting, but it might mess up your own scripts if you refer to md
devices by number.

Rich

On Thu, Jan 05, 2012 at 08:08:44PM +0000, Ciaran McCreesh wrote:
> > > Or will /etc move to /usr too?
> >
> > No, /etc isn't going anywhere.
>
> Are you sure? I heard a rumour that systemd will soon require you to
> put /etc inside your initrd (since / can't be mounted without it).
> Obviously, you'd have to reboot if you made any changes to your config
> files, but that's OK since you can't safely restart daemons anyway.

They've thought of that, and will make
- kexec mandatory so that reboots are not needed for those times you
need to switch kernels
- make hibernation mandatory for the other times

On Thu, 2012-01-05 at 20:08 +0000, Ciaran McCreesh wrote:
> On Thu, 5 Jan 2012 13:30:24 -0600
> William Hubbs <williamh@gentoo.org> wrote:
> > > Or will /etc move to /usr too?
> >
> > No, /etc isn't going anywhere.
>
> Are you sure? I heard a rumour that systemd will soon require you to
> put /etc inside your initrd (since / can't be mounted without it).
> Obviously, you'd have to reboot if you made any changes to your config
> files, but that's OK since you can't safely restart daemons anyway.

Dude, the systemd people are not crazy. You should try to understand
what they do before criticizing.

--
Olivier Crête
tester@gentoo.org
Gentoo Developer