On Sun, 13 May 2018 23:29:49 -0500, you wrote: >On Sun, May 13, 2018 at 10:06 PM Stephen Worthington <
>> On Sun, 13 May 2018 18:56:42 -0500, you wrote:
>> >On Sun, May 13, 2018 at 11:55 AM Stephen Worthington <
>> >firstname.lastname@example.org> wrote:
>> >> On Sun, 13 May 2018 10:47:45 -0500, you wrote:
>> >> >I see "can't write to" issue and some "AutoExpire: CalcParams(): Max
>> >> >required Free Space: 1.0 GB w/freq: 15 min" thing which when I search
>> >> seems
>> >> >to be some TV Tuner issue
>> >> You can ignore the "Autoexpire:" messages - they are normal. It just
>> >> means that mythbackend has looked at all the storage groups and
>> >> checked that they have enough free space. If they do not, it will
>> >> choose some recordings to expire to create enough free space. Always
>> >> make sure your storage groups have enough free space unless you want
>> >> old recordings to expire. I leave a minimum of 20 Gibytes free at all
>> >> times.
>> >> The "can't write to" messages come about by mythbackend writing a zero
>> >> length file ".test" to each storage group directory on startup. If it
>> >> fails to create the file, or fails to be able to delete it, it will
>> >> give the "can't write to" message and will not use that storage group
>> >> directory for writing until the next time it is started.
>> >> So, first, take a look and see if your storage group directories
>> >> contain .test files, and what their ownership and permissions look
>> >> like. Then manually delete them if they are there. If you have had a
>> >> permissions problem that has caused the .test files to be left behind
>> >> undeleted, then I think the test fails on creating new ones, hence the
>> >> need to manually delete them after you fix the permissions.
>> >> Then, try creating the .test files yourself as the user that
>> >> mythbackend runs as (mythtv). Log in as root and do this:
>> >> su - mythtv -c touch "<storage group path>/.test"
>> >> And try deleting them again:
>> >> su - mythtv -c "rm <storage group path>/.test"
>> >> If either command does not work, then you need to fix the ownership
>> >> and permissions until it does work.
>> >so i am slightly confused. Here's what I have:
>> >BE user - myth (no "tv"), mythconverg user: mythtv
>> >FE user - mythtv
>> >In ubuntu BE, i don't have "root". so i always log in as "myth" and when I
>> >do, it let's me do the touch and rm the .test
>> So does the backend user "mythtv" exist and have "mythtv" group? That
>> is important. So what does this command show:
>> su - mythtv -c "groups"
>> This is what I get:
>> root@mypvr:~# su - mythtv -c "groups"
>> mythtv dialout cdrom audio video
>I don't know the password for username "mythtv"
>During installation, I tried to name my user as "mythtv" for consistency
>sake, but Ubuntu didn't allow me saying it's reserved so I had to create
>"myth" as a user
It is quite likely that the "mythtv" user does not have a password. It
is possible for package installers to create a user without a
password, and the "mythtv" user is created by the packages. The
normal way to access such a user is via su or sudo (or root). Using a
different name for your username on the backend box was the correct
thing to do. The "myth" user should be a member of the "mythtv"
group, but it will be a normal user capable of running a desktop and
running mythtv-setup and mythfrontend. >I do get mythtv as a username - quick search gave me a command:
>myth@ScorpioOne:~$ cut -d: -f1 /etc/passwd
>> I am not sure why it has "dialout", but the other groups are
>> >When we test permissions, do we test the "mythtv" because the FE as well
>> >the DB user is "mythtv" or do we test "myth" because that's the BE user?
>> Mythbackend runs as user mythtv on the backend box. Mythfrontend runs
>> as whatever user you run your desktop under on the frontend box. Both
>> of those users normally have "mythtv" as an additional group on their
>> respective boxes. The group IDs of the "mythtv" groups on the two
>> different boxes will not usually be the same, hence the are not the
>> same group.
>> >I gather the DB user "mythtv" comes in play later once FE talks to the BE,
>> >but the file permissions AND ownership is what confuses me.
>> >My share "/usr/local/media/metadata" is currently showing as
>> >myth@ScorpioOne:~$ ls -ahl /usr/local/media/
>> >total 208K
>> >drwxrwsr-x 9 nobody 4294967294 91 May 7 01:07 .
>> >drwxr-xr-x 11 root root 127 May 11 14:24 ..
>> >drwxrwsr-x 4 root mythtv 35 May 2 08:43 Data
>> >drwxrwsr-x 2 root mythtv 6 May 7 01:07 live-tv
>> >drwxrwsr-x 2 nobody 4294967294 128K May 13 2018 metadata
>> >drwxrwsr-x 185 nobody 4294967294 20K May 12 23:11 Music
>> >drwxrwsr-x 103 nobody 4294967294 4.0K Jan 1 09:26 Pics
>> >drwxrwsr-x 4 root mythtv 38 May 7 01:34 TV
>> >drwxrwsr-x 8 root mythtv 108 Apr 13 12:47 Videos
>> For storage groups, the ownership and permissions of the mythbackend
>> user on the backend box are what matters. Mythbackend normally runs
>> as user "mythtv" and group "mythtv" in Ubuntu. Mythfrontend talks to
>> mythbackend for access to the storage group files, so for access to
>> the recordings, its ownership and permissions do not matter as all
>> that traffic happens over TCP/IP connections between mythbackend and
>> mythfrontend. The normal setup for any storage group directory that
>> mythbackend has to access is to be ownership mythtv:mythtv with user
>> and group access both set to execute (=directory access), read and
>> write (chmod u=xrw,g=xrw).
>> However, it is possible to access things other than through storage
>> groups. Both mythmusic (which is a plugin that runs entirely in
>> mythfrontend) and mythvideo (which is a builtin part of mythfrontend)
>> have settings for paths they can use to directly access things via the
>> normal access paths of the frontend box they are running on. These
>> paths are in addition to the storage group settings they also have. So
>> it depends on how you have them set up - if you have them access
>> things via their storage groups, then the same rules apply as for any
>> storage group - access is via mythbackend and the ownership and
>> permissions of mythbackend are what matter, and access happens from
>> the backend box. But if you are using the local system paths, access
>> to things via those paths uses mythfrontend's ownership and
>> permissions, and happens from the frontend box. The local paths are
>> also different for each frontend - the frontend settings are stored in
>> the "settings" table with a different hostname field value for each
>> frontend. So each frontend can access the videos or music in the
>> storage groups via the backend, and also potentially different videos
>> and music stored locally via the frontend.
>> I think mythgallery and its replacement (mythimage?) work the same way
>> as mythmusic and mythvideo.
>> Storage directories are set up in mythtv-setup on the backend box. The
>> local directories for a frontend are set up in mythfrontend on each
>> frontend box.
>> So, I find your /usr/local/media ownership a bit strange. For a
>> start, the listing shows that the metadata, Music and Pics directories
>> have an unknown group (group ID 4294967294). That normally only
>> happens if they are network mounted from a different PC, where the
>> group IDs and user IDs do not match those on the local PC.
>You are correct - sorry I thought i had mentioned earlier - these are NFS
OK, I must have missed that. >
>> If they
>> were local they would normally have been created using existing local
>> users and groups and would be displayed with the name rather than a
>> number. Is that what is going on? That really can cause problems!
>> And having your recording directories mounted on other boxes means
>> that the bandwidth to them is limited by the Ethernet connection to
>> the other boxes, and that can cause failed recordings when the traffic
>> is too large.
>I have a gig ethernet connection to the boxes, both at my home.
>> And even more strange is that the /usr/local/media/ directory itself
>> is nobody:4294967294. So is that directory a network mount on a
>> different box?
>Yes the storage are NFS mounted for everything
> - Video
> - TV
> - metadata
> - Music
> - Pics
> - live-tv
>> The other directories (Data, live-tv, TV and Videos) all have root
>> ownership. That is not necessarily fatal, as they do have mythtv
>> group ownership, but it is not normal. They ought to be mythtv user
>> And all the files under the storage group directories also should be
>> mythtv:mythtv ownership.
>> It would also be a good idea to check that mythbackend is running as
>> user mythtv. Is this something like what "ps -ef | grep mythba" shows
>> for you?
>> root@mypvr:~# ps -ef | grep mythba
>> mythtv 3480 1 8 May11 ? 06:22:36 /usr/bin/mythbackend
>> --quiet --syslog local7 -v record,dvbcam
>> root 18746 7172 0 14:39 pts/2 00:00:00 grep --color=auto
>This is what it shows for me:
>myth@ScorpioOne:~$ ps -ef | grep mythba
>myth 4668 2188 0 23:29 pts/18 00:00:00 grep --color=auto mythba
>mythtv 25292 1 0 May12 ? 00:06:01 /usr/bin/mythbackend
>--quiet --syslog local7
That is good - mythbackend is running as the mythtv user.
I have not used NFS much, but it looks like it has the same problem as
SAMBA/CIFS - the user IDs and group IDs (apart from root = 0) do not
match between machines. That makes it difficult to get ownership to
work, unless NFS has some mechanism to map the user IDs and group IDs
to get them to match. So you probably will need to use wide open
permissions instead. As in everybody having access (chmod a=rwx). But
have a look as the NFS documentation and do a web search to see what
your options are for getting correct ownership.
Using network drives is fine for read-only things, such as videos,
pictures and music. And there should be no problems for the small
writeable files for things like the metadata and artwork. But the
higher bandwidth, and time critical, writing of recordings can be a
problem. Looking at the numbers for the bandwidth of one HD DVB-T
recording from my channels, I seem to need about 10 Mbytes/s or 80
Mbit/s for one recording. So being generous and allocating 100
Mbit/s, you would think that a 1 Gbit/s Ethernet would be fine for up
to 10 recordings at once. But that ignores other traffic, and unless
the NFS recording traffic has its packets flagged as high priority and
the switch between the backend PC and the NFS box does QoS and gives
those packets priority over other traffic, then you will have a
problem with just one recording if something else is using all the
bandwidth it can get. Just you copying a file to the NFS box from
another device can use all the available bandwidth on the NFS box's
one Ethernet port, and has the potential to cause gaps in your
recordings. Doing backups to the NFS box (a common use for such a
box) is very likely to cause trouble unless the backup software has
options to be set to use lower bandwidth, or can be set to only do
backups when there will never be any recordings happening.
So I would suggest doing some serious testing to see if it is OK to
record directly to the NFS box before setting it up that way. How
many recordings at once do think you will be doing? How many tuners
do you have? How many multirec recordings will each tuner do at once?
An alternative is to record to a local drive on the mythbackend box,
and when each recording completes (or later), move it over to the NFS
box. To make that work, you set up another storage group (other than
"Default") and put the NFS recording directories in that storage
group. I have one like that called "Archives", for my 8 Tbyte
shingled drives that can not be used for recordings as they can just
stop for several seconds while they do a shingle re-write operation.
Then you put the local hard drive directories in the "Default" storage
group, and if you never create any recording rules that use the
"Archives" storage group, nothing will ever be recorded to that
storage group, but anything you move into the "Archives" storage group
will be visible to mythbackend and can be played and deleted.
I have written myself some Python for moving files to my archive
drives. It keeps a check on MythTV activity and stops moving files
when a recording starts or just before the next one is scheduled to
start. It is probably not quite what you would want for moving files
from your backend local drive(s) to NFS, but I would be happy to
modify it to do what you need if you decide to go that way.
Even with a local recording drive, if it is spinning rust I use a rule
of thumb that there should be no more than three recordings going to a
drive at once. The problem is about the time taken for head movements
between the recording files, and also to the directory areas when the
file needs to be expanded. And it is even worse if your system and
database are on the same drive. Then I would do no more than two
recordings at once. Modern hard drives have very fast write speeds,
but the head movement has not sped up nearly as much as the on-track
write speed has, so it is easy to get fooled by the fast write speed
in the specification.
Of course, if the local drive is an SSD, then you can usually write to
many more files at once as there is no head movement. All you have to
watch out for there is the problem of the erase times. If the spare
space on an SSD has already been erased, its write times are very
fast. But if the drive is using a weekly TRIM operation to do the
erasing (which is the default in Ubuntu), then the large file sizes of
recording files can cause it to run out of erased blocks and have to
do a block erase before each block write. That makes an SSD write
very slowly. So if you record to SSD you should either be using the
"discard" option on the SSD when it is mounted from fstab, or running
the fstrim cron job much more often. I run my fstrim cron job daily,
but I do not record to the SSD - it just gets a lot of database
mythtv-users mailing list
email@example.com http://lists.mythtv.org/mailman/listinfo/mythtv-users http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org