Mailing List Archive

controlling disk space on "/" on mythtv backend
On my backend my boot device and "/" directory is a 16GB Flash drive. I
did something stupid and filled up the drive.  So after deleting the
file I moved to the wrong place I'm back to the normal free space.

But it brings up a question.  Does mythbackend use temporary disk space
somewhere on "/" that I need to redirect to one of my 2TB hard drives? 
I have tons of free disk space just not on "/"

My first guess is all the /var/lib/mythtv/* files to see which grow
large over time.  Obviously, Default and LiveTV are on the separate hard
drives.

Jim A


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 11 January 2018 9:49:39 pm Jim Abernathy <jfabernathy@outlook.com> wrote:

> On my backend my boot device and "/" directory is a 16GB Flash drive. I
> did something stupid and filled up the drive.  So after deleting the
> file I moved to the wrong place I'm back to the normal free space.
>
> But it brings up a question.  Does mythbackend use temporary disk space
> somewhere on "/" that I need to redirect to one of my 2TB hard drives? 
> I have tons of free disk space just not on "/"
>
> My first guess is all the /var/lib/mythtv/* files to see which grow
> large over time.  Obviously, Default and LiveTV are on the separate hard
> drives.
>
> Jim A
>
>
>

~/.mythtv/cache/
Is the first one that comes to mind for me. I have seen previous reports of
it causing disk space issues on this list.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 01/11/2018 06:55 AM, Mark Perkins wrote:
> On 11 January 2018 9:49:39 pm Jim Abernathy <jfabernathy@outlook.com> wrote:
>
>> On my backend my boot device and "/" directory is a 16GB Flash drive. I
>> did something stupid and filled up the drive.  So after deleting the
>> file I moved to the wrong place I'm back to the normal free space.
>>
>> But it brings up a question.  Does mythbackend use temporary disk space
>> somewhere on "/" that I need to redirect to one of my 2TB hard drives?
>> I have tons of free disk space just not on "/"
>>
>> My first guess is all the /var/lib/mythtv/* files to see which grow
>> large over time.  Obviously, Default and LiveTV are on the separate hard
>> drives.
>>
>> Jim A
>>
>>
>>
> ~/.mythtv/cache/
> Is the first one that comes to mind for me. I have seen previous reports of
> it causing disk space issues on this list.
>
If this system never ran mythfrontend, would ~/.mythtv/cache/ be a problem?

Jim A

> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Thu, 11 Jan 2018 11:12:52 +0000, you wrote:

>On my backend my boot device and "/" directory is a 16GB Flash drive. I
>did something stupid and filled up the drive.? So after deleting the
>file I moved to the wrong place I'm back to the normal free space.
>
>But it brings up a question.? Does mythbackend use temporary disk space
>somewhere on "/" that I need to redirect to one of my 2TB hard drives??
>I have tons of free disk space just not on "/"
>
>My first guess is all the /var/lib/mythtv/* files to see which grow
>large over time.? Obviously, Default and LiveTV are on the separate hard
>drives.
>
>Jim A

Yes,you are living dangerously with a drive that small. Depending on
how many recordings you have, the caches under:

/home<mythfrontend user>/.mythtv

can get very large - multiple gigabytes. Those caches can be deleted
at any time and mythfrontend will re-create them as necessary. But
you may want to move them to another drive and replace them with links
to the new locations.

If you have a cron job doing automatic database backups, you need
plenty of space to do that as it creates an ASCII .sql version of the
entire mythconverg database, and then compresses that using gzip. I
believe it stores the .sql file in /tmp.

I have a modified version of mythconverg_backup.pl on my web server
that adds a new option (--tmpdir) that allows you to set the location
for the temporary .sqlfile. You can get it from here:

http://www.jsw.gen.nz/mythtv/mythconverg_backup_jsw.pl

The database backups default to being stored here:

/var/lib/mythtv/db_backups

so you should change the location for that - I think you can set it in
the storage groups setup, and also on the mythconverg_backup.pl
command line in the cron job.

Anything that still gets stored under /var/lib/mythtv (eg artwork)
should be moved elsewhere, as the storage used grows with time and the
number of recordings. I would suggest shutting down mythbackend, then
moving the /var/lib/mythtv directory to a drive with lots of space.
Then you can create /var/lib/mythtv as a link to the new location.
That ensures that any default usage of /var/lib/mythtv that you have
not set up to use a different location will not go to the SSD. But be
aware that if the drive you use is also used for recordings, you may
be increasing the head movements on that drive, and hence decreasing
the number of recordings the drive can have happening on it at the
same time without losing some recording data.

And if you are running the optimise and repair database cron job
(optimize_mythdb), that requires space to make a temporary copy of
each database table as it does the optimising. So you need room for a
temporary copy of the largest table in your database (normally
recordedseek) and its index file. So see what this command shows you:

du -hc /var/lib/mysql/mythconverg/recordedseek.*

and you always need at least that much free space to avoid a database
crash during optimising. Here is mine (I have over 20,000
recordings):

root@mypvr:~# du -hc /var/lib/mysql/mythconverg/recordedseek.*
4.0K /var/lib/mysql/mythconverg/recordedseek.frm
5.6G /var/lib/mysql/mythconverg/recordedseek.MYD
5.1G /var/lib/mysql/mythconverg/recordedseek.MYI
11G total

I have at some time had a problem with all of the above causing low
disk space on / - which I have now solved (at least for the moment) by
having / as a 100 Gibyte partition on my new NVMe SSD. It is
currently using 45 Gibytes.

Whenever you manage to have / run out of space, you should always make
sure you run:

/usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl

to repair and optimise the database before running mythbackend again,
as the database may well have one or more tables crashed due to the
lack of space. Fortunately MySQL and MariaDB are very forgiving about
what happens when the space runs out, so a simple repair normally
fixes everything. Unfortunately, mythbackend can appear to be running
normally when tables are crashed, but it will in fact be unable to
write to the crashed tables. There will normally be log messages
about SQL problems, but mythbackend will not otherwise notify you that
it has a problem.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Thu, 11 Jan 2018 12:31:40 +0000, you wrote:

>
>
>On 01/11/2018 06:55 AM, Mark Perkins wrote:
>> On 11 January 2018 9:49:39 pm Jim Abernathy <jfabernathy@outlook.com> wrote:
>>
>>> On my backend my boot device and "/" directory is a 16GB Flash drive. I
>>> did something stupid and filled up the drive.? So after deleting the
>>> file I moved to the wrong place I'm back to the normal free space.
>>>
>>> But it brings up a question.? Does mythbackend use temporary disk space
>>> somewhere on "/" that I need to redirect to one of my 2TB hard drives?
>>> I have tons of free disk space just not on "/"
>>>
>>> My first guess is all the /var/lib/mythtv/* files to see which grow
>>> large over time.? Obviously, Default and LiveTV are on the separate hard
>>> drives.
>>>
>>> Jim A
>>>
>>>
>>>
>> ~/.mythtv/cache/
>> Is the first one that comes to mind for me. I have seen previous reports of
>> it causing disk space issues on this list.
>>
>If this system never ran mythfrontend, would ~/.mythtv/cache/ be a problem?
>
>Jim A

No.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
Hoi Stephen,

Thursday, January 11, 2018, 2:44:15 PM, you wrote:

> On Thu, 11 Jan 2018 12:31:40 +0000, you wrote:

>>
>>
>>On 01/11/2018 06:55 AM, Mark Perkins wrote:
>>> On 11 January 2018 9:49:39 pm Jim Abernathy <jfabernathy@outlook.com> wrote:
>>>
>>>> On my backend my boot device and "/" directory is a 16GB Flash drive. I
>>>> did something stupid and filled up the drive.? So after deleting the
>>>> file I moved to the wrong place I'm back to the normal free space.
>>>>
>>>> But it brings up a question.? Does mythbackend use temporary disk space
>>>> somewhere on "/" that I need to redirect to one of my 2TB hard drives?
>>>> I have tons of free disk space just not on "/"
>>>>
>>>> My first guess is all the /var/lib/mythtv/* files to see which grow
>>>> large over time.? Obviously, Default and LiveTV are on the separate hard
>>>> drives.
>>>>
>>>> Jim A
>>>>
>>>>
>>>>
>>> ~/.mythtv/cache/
>>> Is the first one that comes to mind for me. I have seen previous reports of
>>> it causing disk space issues on this list.
>>>
>>If this system never ran mythfrontend, would ~/.mythtv/cache/ be a problem?
>>
>>Jim A

> No.
> _______________________________________________

You should mount /tmp and /var/tmp elsewhere as they on occasion can
get filled up by any process and on /var are more risky directories
like for instance /var/log

Tot mails,
Hika mailto:hikavdh@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Jan 11, 2018, at 10:17 AM, Hika van den Hoven <hikavdh@gmail.com<mailto:hikavdh@gmail.com>> wrote:

Hoi Stephen,

Thursday, January 11, 2018, 2:44:15 PM, you wrote:

On Thu, 11 Jan 2018 12:31:40 +0000, you wrote:



On 01/11/2018 06:55 AM, Mark Perkins wrote:
On 11 January 2018 9:49:39 pm Jim Abernathy <jfabernathy@outlook.com<mailto:jfabernathy@outlook.com>> wrote:

On my backend my boot device and "/" directory is a 16GB Flash drive. I
did something stupid and filled up the drive. So after deleting the
file I moved to the wrong place I'm back to the normal free space.

But it brings up a question. Does mythbackend use temporary disk space
somewhere on "/" that I need to redirect to one of my 2TB hard drives?
I have tons of free disk space just not on "/"

My first guess is all the /var/lib/mythtv/* files to see which grow
large over time. Obviously, Default and LiveTV are on the separate hard
drives.

Jim A



~/.mythtv/cache/
Is the first one that comes to mind for me. I have seen previous reports of
it causing disk space issues on this list.

If this system never ran mythfrontend, would ~/.mythtv/cache/ be a problem?

Jim A

No.
_______________________________________________

You should mount /tmp and /var/tmp elsewhere as they on occasion can
get filled up by any process and on /var are more risky directories
like for instance /var/log

Tot mails,
Hika mailto:hikavdh@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens


I think i have moved most of my /var/lib/mythtv directory usage to other drives via the Storage Directories setup in mythtv-setup. But I was thinking about for short term fixes if I could just use Clonezilla to copy everything on my 16GB boot Flash drive to a 64GB flash drive. Would it really be that easy??

Jim A


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org<mailto:mythtv-users@mythtv.org>
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org<https://forum.mythtv.org/>
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Thu, Jan 11, 2018 at 11:28 AM, Jim Abernathy <jfabernathy@outlook.com>
wrote:


> I think i have moved most of my /var/lib/mythtv directory usage to other
> drives via the Storage Directories setup in mythtv-setup. But I was
> thinking about for short term fixes if I could just use Clonezilla to copy
> everything on my 16GB boot Flash drive to a 64GB flash drive. Would it
> really be that easy??
>
>
It should be. You have to be careful to expand the file system to the size
of your hard drive with Clonezilla. There is definitely an option for that
but you can miss it. That's the simplest choice. Clonezilla will fix the
bootloader install for you.

You could use this script here:
http://www.linuxjournal.com/magazine/hack-and-migrate-new-hard-drive

I have used that script many, many times to migrate hard drives. At the
end you'll have to reinstall GRUB or your bootloader of choice. That's the
only tricky bit but it's not that hard. You have to chroot into the new
drive and run one command. Otherwise, the drive will not boot.

You will also have to fix the UUIDs most likely in /etc/fstab. Cut and
paste is your friend here.

Chroot details here: http://bremford.org/tips/MigrateDiskInUbuntu.html and
also another migration recipe (but I use the first link).

Doing it the harder way pays off in the end. You learn by doing.

No matter what, BE CAREFUL to not mix up the drives using Clonezilla or a
recipe. Using the recipe, I mount the drives as /mnt/source and
/mnt/target to be safer and more clear. And I do an 'ls' to check
beforehand that I haven't mixed them up. It can happen.
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Thu, Jan 11, 2018 at 11:48 AM, Jerry <mythtv@hambone.e4ward.com> wrote:

> On Thu, Jan 11, 2018 at 11:28 AM, Jim Abernathy <jfabernathy@outlook.com>
> wrote:
>
>
>> I think i have moved most of my /var/lib/mythtv directory usage to other
>> drives via the Storage Directories setup in mythtv-setup. But I was
>> thinking about for short term fixes if I could just use Clonezilla to copy
>> everything on my 16GB boot Flash drive to a 64GB flash drive. Would it
>> really be that easy??
>>
>>
> It should be. You have to be careful to expand the file system to the
> size of your hard drive with Clonezilla. There is definitely an option for
> that but you can miss it. That's the simplest choice. Clonezilla will fix
> the bootloader install for you.
>
> You could use this script here: http://www.linuxjournal.com/ma
> gazine/hack-and-migrate-new-hard-drive
>
> I have used that script many, many times to migrate hard drives. At the
> end you'll have to reinstall GRUB or your bootloader of choice. That's the
> only tricky bit but it's not that hard. You have to chroot into the new
> drive and run one command. Otherwise, the drive will not boot.
>
> You will also have to fix the UUIDs most likely in /etc/fstab. Cut and
> paste is your friend here.
>
> Chroot details here: http://bremford.org/tips/MigrateDiskInUbuntu.html
> and also another migration recipe (but I use the first link).
>
> Doing it the harder way pays off in the end. You learn by doing.
>
> No matter what, BE CAREFUL to not mix up the drives using Clonezilla or a
> recipe. Using the recipe, I mount the drives as /mnt/source and
> /mnt/target to be safer and more clear. And I do an 'ls' to check
> beforehand that I haven't mixed them up. It can happen.
>

One other thing here: I boot into a Live CD/DVD environment. It's
definitely better than doing it with a live file system. You can use your
Ubuntu Installation CD/DVD of choice. Just boot with it, say you want to
try out Ubuntu, open up a terminal and get cranking. Or plug the USB
drives into a different independent computer and skip the Live CD/DVD.

And another thing: if you plug them in, Ubuntu will most likely mount them
itself. You'll have to umount the drives and mount them somewhere else, or
use the paths it creates. In Fedora, USB drives mount under
/run/media/username. It may be the same in Ubuntu. So just:

<boot -- plug in drives>

$ sudo umount /run/media/jim/UUID_STRING_1/moretext1
/run/media/jim/UUID_STRING_2/moretext2
$ sudo mkdir /mnt/source /mnt/target

$ dmesg

<page through that - if you just plugged in the drives the info will be
near the end with disk sizes>

$ sudo mount /dev/sd<whatever> /mnt/source
$ sudo mount /dev/sd<next_one> /mnt/target

<resume script here>
<finish script>
<boot new usb drive>
<fix alcoholic drink of choice (beer for me)>
<profit>
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 01/11/2018 12:14 PM, Jerry wrote:
On Thu, Jan 11, 2018 at 11:48 AM, Jerry <mythtv@hambone.e4ward.com<mailto:mythtv@hambone.e4ward.com>> wrote:
On Thu, Jan 11, 2018 at 11:28 AM, Jim Abernathy <jfabernathy@outlook.com<mailto:jfabernathy@outlook.com>> wrote:

I think i have moved most of my /var/lib/mythtv directory usage to other drives via the Storage Directories setup in mythtv-setup. But I was thinking about for short term fixes if I could just use Clonezilla to copy everything on my 16GB boot Flash drive to a 64GB flash drive. Would it really be that easy??


It should be. You have to be careful to expand the file system to the size of your hard drive with Clonezilla. There is definitely an option for that but you can miss it. That's the simplest choice. Clonezilla will fix the bootloader install for you.

You could use this script here: http://www.linuxjournal.com/magazine/hack-and-migrate-new-hard-drive

I have used that script many, many times to migrate hard drives. At the end you'll have to reinstall GRUB or your bootloader of choice. That's the only tricky bit but it's not that hard. You have to chroot into the new drive and run one command. Otherwise, the drive will not boot.

You will also have to fix the UUIDs most likely in /etc/fstab. Cut and paste is your friend here.

Chroot details here: http://bremford.org/tips/MigrateDiskInUbuntu.html and also another migration recipe (but I use the first link).

Doing it the harder way pays off in the end. You learn by doing.

No matter what, BE CAREFUL to not mix up the drives using Clonezilla or a recipe. Using the recipe, I mount the drives as /mnt/source and /mnt/target to be safer and more clear. And I do an 'ls' to check beforehand that I haven't mixed them up. It can happen.

One other thing here: I boot into a Live CD/DVD environment. It's definitely better than doing it with a live file system. You can use your Ubuntu Installation CD/DVD of choice. Just boot with it, say you want to try out Ubuntu, open up a terminal and get cranking. Or plug the USB drives into a different independent computer and skip the Live CD/DVD.

And another thing: if you plug them in, Ubuntu will most likely mount them itself. You'll have to umount the drives and mount them somewhere else, or use the paths it creates. In Fedora, USB drives mount under /run/media/username. It may be the same in Ubuntu. So just:

<boot -- plug in drives>

$ sudo umount /run/media/jim/UUID_STRING_1/moretext1 /run/media/jim/UUID_STRING_2/moretext2
$ sudo mkdir /mnt/source /mnt/target

$ dmesg

<page through that - if you just plugged in the drives the info will be near the end with disk sizes>

$ sudo mount /dev/sd<whatever> /mnt/source
$ sudo mount /dev/sd<next_one> /mnt/target

<resume script here>
<finish script>
<boot new usb drive>
<fix alcoholic drink of choice (beer for me)>
<profit>


Thanks for the information!
Here's what I did and I have it working.
1. First I created a bootable USB Flash with Clonezilla.
2. I booted that on my myth backend server and cloned disk to disk the USB boot flash to the new larger USB flash. I figured that wouldn't do any damage.
3. I then removed the boot flash and saved it in a desk draw.
4. I took the new larger USB Flash to another Ubuntu desktop and use gparted to delete the swap and expand the EXT4 partition to within 8GB of the end and then created a new swap partition there.
5. I took the new larger flash and put it in the server and rebooted. I forgot to fix the UUID in the /etc/fstab for the swap file so it timed out on the swap at boot. Once booted I fixed the UUID problem for swap and reboot.

Done and working so far.

Thanks all,

Jim A


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org<mailto:mythtv-users@mythtv.org>
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 01/11/2018 08:43 AM, Stephen Worthington wrote:

On Thu, 11 Jan 2018 11:12:52 +0000, you wrote:



On my backend my boot device and "/" directory is a 16GB Flash drive. I
did something stupid and filled up the drive. So after deleting the
file I moved to the wrong place I'm back to the normal free space.

But it brings up a question. Does mythbackend use temporary disk space
somewhere on "/" that I need to redirect to one of my 2TB hard drives?
I have tons of free disk space just not on "/"

My first guess is all the /var/lib/mythtv/* files to see which grow
large over time. Obviously, Default and LiveTV are on the separate hard
drives.

Jim A


.....

If you have a cron job doing automatic database backups, you need
plenty of space to do that as it creates an ASCII .sql version of the
entire mythconverg database, and then compresses that using gzip. I
believe it stores the .sql file in /tmp.


Question on the cron job for db backup. Is this setup on install of mythtv? I have not created a cron job to do this. But there are a few backups in /var/lib/mythtv/db_backups and I have not run the backup manually yet. I just now ran:
/usr/share/mythtv/mythconverg_backup.pl
It puts the backup in the directory that I specified in Storage Directories in setup. I just don't know if it's running automatically.

Jim A


__________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org<mailto:mythtv-users@mythtv.org>
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 01/11/2018 01:11 PM, Jim Abernathy wrote:
> Question on the cron job for db backup. Is this setup on install of mythtv? I have not created a cron job to do this. But there are a few backups in /var/lib/mythtv/db_backups and I have not run the backup manually yet. I just now ran:
> /usr/share/mythtv/mythconverg_backup.pl
> It puts the backup in the directory that I specified in Storage Directories in setup. I just don't know if it's running automatically.

Try this: find /etc/cron* -name "*myth*"

I suspect you'll find something like this: /etc/cron.weekly/mythtv-database
If you look in the file you may find mythbuntu footprints.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 12 January 2018 5:49:42 am Jim Abernathy <jfabernathy@outlook.com> wrote:

>
>
> Question on the cron job for db backup. Is this setup on install of mythtv? I have not created a cron job to do this. But there are a few backups in /var/lib/mythtv/db_backups and I have not run the backup manually yet. I just now ran:
> /usr/share/mythtv/mythconverg_backup.pl
> It puts the backup in the directory that I specified in Storage Directories in setup. I just don't know if it's running automatically.
>
> Jim A
>
>
As I understand it, no the cron backup is not created automatically by mythtv. However when it does database upgrades it will try very hard to run a backup before doing the upgrade. That may be the few files you see.
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 01/11/2018 03:08 PM, Mark Perkins wrote:
> As I understand it, no the cron backup is not created automatically by mythtv.

I agree, but mythbuntu did appear to do it. Mario Limonciello's name is in
the script on my production host, although it could be leftover from 2010
and no longer provided. I build from source now, so can't really tell.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
> On Jan 11, 2018, at 2:49 PM, Bill Meek <keemllib@gmail.com> wrote:
>
> On 01/11/2018 01:11 PM, Jim Abernathy wrote:
>> Question on the cron job for db backup. Is this setup on install of mythtv? I have not created a cron job to do this. But there are a few backups in /var/lib/mythtv/db_backups and I have not run the backup manually yet. I just now ran:
>> /usr/share/mythtv/mythconverg_backup.pl
>> It puts the backup in the directory that I specified in Storage Directories in setup. I just don't know if it's running automatically.
>
> Try this: find /etc/cron* -name "*myth*"
>
> I suspect you'll find something like this: /etc/cron.weekly/mythtv-database
> If you look in the file you may find mythbuntu footprints.
>
> --

Yes, it’s in the weekly section. Thanks

Jim A

> Bill
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Thu, 11 Jan 2018 21:08:11 +0000, you wrote:

>On 12 January 2018 5:49:42 am Jim Abernathy <jfabernathy@outlook.com> wrote:
>
>>
>>
>> Question on the cron job for db backup. Is this setup on install of mythtv? I have not created a cron job to do this. But there are a few backups in /var/lib/mythtv/db_backups and I have not run the backup manually yet. I just now ran:
>> /usr/share/mythtv/mythconverg_backup.pl
>> It puts the backup in the directory that I specified in Storage Directories in setup. I just don't know if it's running automatically.
>>
>> Jim A
>>
>>
>As I understand it, no the cron backup is not created automatically by mythtv. However when it does database upgrades it will try very hard to run a backup before doing the upgrade. That may be the few files you see.

I believe the cron job for optimise/repair was created by Mythbuntu
Control Centre. So if you have used that in the past, you are likely
to have set it up that way. But as MCC is no longer supported, anyone
now doing a new install will need to set it up manually.

To install the optimise/repair job on Ubuntu, all you have to do is
this (as root or using sudo):

cp -a
/usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
/etc/cron.daily/optimize_mythdb

For the database backup job, here is the standard
/etc/cron.weekly/mythtv-database file, which is installed by the
mythtv-database package:

#!/bin/sh
# /etc/cron.weekly/mythtv-database script - check and backup
mythconverg tables
# Copyright 2005/12/02 2006/10/08 Paul Andreassen
# 2010 Mario Limonciello

set -e -u

DBNAME="mythconverg"
DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"

/usr/bin/mysqlcheck $DEBIAN -s $DBNAME

/usr/share/mythtv/mythconverg_backup.pl

/usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
backed up."

# End of file.

I prefer to do my database backups daily, and direct them to be stored
on my Windows box over the network. So I have his in
/etc/cron.daily/mythtv-backup:

#!/bin/sh
# /etc/cron.weekly/mythtv-database script - check and backup
mythconverg tables
# Copyright 2005/12/02 2006/10/08 Paul Andreassen
# 2010 Mario Limonciello

# JSW Modified for /etc/cron.daily backups to a network drive.

set -e -u

DBNAME="mythconverg"
DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"

# Debug:
/usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
cron.daily checking started."

/usr/bin/mysqlcheck $DEBIAN -s $DBNAME

# Debug:
/usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
cron.daily checking finished, backup starting."

/usr/bin/nice -n 2 /usr/local/bin/mythconverg_backup_jsw.pl
--directory "/mnt/savaidh/ldrive/Backups/MythTV_db_backup/mypvr"
--tempdir "/mnt/ssd1/tmp"

/usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
backed up."

# End of file.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 01/11/2018 08:14 PM, Stephen Worthington wrote:
> On Thu, 11 Jan 2018 21:08:11 +0000, you wrote:
>
>> On 12 January 2018 5:49:42 am Jim Abernathy <jfabernathy@outlook.com> wrote:
>>
>>>
>>> Question on the cron job for db backup. Is this setup on install of mythtv? I have not created a cron job to do this. But there are a few backups in /var/lib/mythtv/db_backups and I have not run the backup manually yet. I just now ran:
>>> /usr/share/mythtv/mythconverg_backup.pl
>>> It puts the backup in the directory that I specified in Storage Directories in setup. I just don't know if it's running automatically.
>>>
>>> Jim A
>>>
>>>
>> As I understand it, no the cron backup is not created automatically by mythtv. However when it does database upgrades it will try very hard to run a backup before doing the upgrade. That may be the few files you see.
> I believe the cron job for optimise/repair was created by Mythbuntu
> Control Centre. So if you have used that in the past, you are likely
> to have set it up that way. But as MCC is no longer supported, anyone
> now doing a new install will need to set it up manually.
>
> To install the optimise/repair job on Ubuntu, all you have to do is
> this (as root or using sudo):
>
> cp -a
> /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
> /etc/cron.daily/optimize_mythdb
I must not have something installed I have no directory:
/usr/share/doc/mythtv-backend
And I can't find the file optimize_mythdb.pl

Jim A

> For the database backup job, here is the standard
> /etc/cron.weekly/mythtv-database file, which is installed by the
> mythtv-database package:
>
> #!/bin/sh
> # /etc/cron.weekly/mythtv-database script - check and backup
> mythconverg tables
> # Copyright 2005/12/02 2006/10/08 Paul Andreassen
> # 2010 Mario Limonciello
>
> set -e -u
>
> DBNAME="mythconverg"
> DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"
>
> /usr/bin/mysqlcheck $DEBIAN -s $DBNAME
>
> /usr/share/mythtv/mythconverg_backup.pl
>
> /usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
> backed up."
>
> # End of file.
>
> I prefer to do my database backups daily, and direct them to be stored
> on my Windows box over the network. So I have his in
> /etc/cron.daily/mythtv-backup:
>
> #!/bin/sh
> # /etc/cron.weekly/mythtv-database script - check and backup
> mythconverg tables
> # Copyright 2005/12/02 2006/10/08 Paul Andreassen
> # 2010 Mario Limonciello
>
> # JSW Modified for /etc/cron.daily backups to a network drive.
>
> set -e -u
>
> DBNAME="mythconverg"
> DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"
>
> # Debug:
> /usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
> cron.daily checking started."
>
> /usr/bin/mysqlcheck $DEBIAN -s $DBNAME
>
> # Debug:
> /usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
> cron.daily checking finished, backup starting."
>
> /usr/bin/nice -n 2 /usr/local/bin/mythconverg_backup_jsw.pl
> --directory "/mnt/savaidh/ldrive/Backups/MythTV_db_backup/mypvr"
> --tempdir "/mnt/ssd1/tmp"
>
> /usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
> backed up."
>
> # End of file.
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
>> I believe the cron job for optimise/repair was created by Mythbuntu
>> Control Centre. So if you have used that in the past, you are likely
>> to have set it up that way. But as MCC is no longer supported, anyone
>> now doing a new install will need to set it up manually.
>>
>> To install the optimise/repair job on Ubuntu, all you have to do is
>> this (as root or using sudo):
>>
>> cp -a
>> /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
>> /etc/cron.daily/optimize_mythdb
> I must not have something installed I have no directory:
> /usr/share/doc/mythtv-backend
> And I can't find the file optimize_mythdb.pl
>
> Jim A

Never mind, I found it. I was looking on a frontend and not the
backend.  too many windows open.

Jim A

>> For the database backup job, here is the standard
>> /etc/cron.weekly/mythtv-database file, which is installed by the
>> mythtv-database package:
>>
>> #!/bin/sh
>> # /etc/cron.weekly/mythtv-database script - check and backup
>> mythconverg tables
>> # Copyright 2005/12/02 2006/10/08 Paul Andreassen
>> # 2010 Mario Limonciello
>>
>> set -e -u
>>
>> DBNAME="mythconverg"
>> DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"
>>
>> /usr/bin/mysqlcheck $DEBIAN -s $DBNAME
>>
>> /usr/share/mythtv/mythconverg_backup.pl
>>
>> /usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
>> backed up."
>>
>> # End of file.
>>
>> I prefer to do my database backups daily, and direct them to be stored
>> on my Windows box over the network. So I have his in
>> /etc/cron.daily/mythtv-backup:
>>
>> #!/bin/sh
>> # /etc/cron.weekly/mythtv-database script - check and backup
>> mythconverg tables
>> # Copyright 2005/12/02 2006/10/08 Paul Andreassen
>> # 2010 Mario Limonciello
>>
>> # JSW Modified for /etc/cron.daily backups to a network drive.
>>
>> set -e -u
>>
>> DBNAME="mythconverg"
>> DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"
>>
>> # Debug:
>> /usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
>> cron.daily checking started."
>>
>> /usr/bin/mysqlcheck $DEBIAN -s $DBNAME
>>
>> # Debug:
>> /usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
>> cron.daily checking finished, backup starting."
>>
>> /usr/bin/nice -n 2 /usr/local/bin/mythconverg_backup_jsw.pl
>> --directory "/mnt/savaidh/ldrive/Backups/MythTV_db_backup/mypvr"
>> --tempdir "/mnt/ssd1/tmp"
>>
>> /usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
>> backed up."
>>
>> # End of file.
>> _______________________________________________
>> mythtv-users mailing list
>> mythtv-users@mythtv.org
>> http://lists.mythtv.org/mailman/listinfo/mythtv-users
>> http://wiki.mythtv.org/Mailing_List_etiquette
>> MythTV Forums: https://forum.mythtv.org
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
> I believe the cron job for optimise/repair was created by Mythbuntu
> Control Centre. So if you have used that in the past, you are likely
> to have set it up that way. But as MCC is no longer supported, anyone
> now doing a new install will need to set it up manually.
>
> To install the optimise/repair job on Ubuntu, all you have to do is
> this (as root or using sudo):
>
> cp -a
> /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
> /etc/cron.daily/optimize_mythdb
>
> For the database backup job, here is the standard
> /etc/cron.weekly/mythtv-database file, which is installed by the
> mythtv-database package:
>
> #!/bin/sh
> # /etc/cron.weekly/mythtv-database script - check and backup
> mythconverg tables
> # Copyright 2005/12/02 2006/10/08 Paul Andreassen
> # 2010 Mario Limonciello
>
> set -e -u
>
> DBNAME="mythconverg"
> DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"
>
> /usr/bin/mysqlcheck $DEBIAN -s $DBNAME
>
> /usr/share/mythtv/mythconverg_backup.pl
>
> /usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
> backed up."
>
> # End of file.
>
> I prefer to do my database backups daily, and direct them to be stored
> on my Windows box over the network. So I have his in
> /etc/cron.daily/mythtv-backup:
>
> #!/bin/sh
> # /etc/cron.weekly/mythtv-database script - check and backup
> mythconverg tables
> # Copyright 2005/12/02 2006/10/08 Paul Andreassen
> # 2010 Mario Limonciello
>
> # JSW Modified for /etc/cron.daily backups to a network drive.
>
> set -e -u
>
> DBNAME="mythconverg"
> DEBIAN="--defaults-extra-file=/etc/mysql/debian.cnf"
>
> # Debug:
> /usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
> cron.daily checking started."
>
> /usr/bin/mysqlcheck $DEBIAN -s $DBNAME
>
> # Debug:
> /usr/bin/logger -p daemon.info -i -t${0##*/} "Debug: $DBNAME
> cron.daily checking finished, backup starting."
>
> /usr/bin/nice -n 2 /usr/local/bin/mythconverg_backup_jsw.pl
> --directory "/mnt/savaidh/ldrive/Backups/MythTV_db_backup/mypvr"
> --tempdir "/mnt/ssd1/tmp"
>
> /usr/bin/logger -p daemon.info -i -t${0##*/} "$DBNAME checked and
> backed up."
>
> # End of file.

Thanks, I've left the weekly backup the system installed as is.  I added
the daily and changed the backup locations and temp directories.

So we'll see what happens.

Thanks again.

Jim A

> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
Whenever you manage to have / run out of space, you should always make
sure you run:

/usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl

to repair and optimise the database before running mythbackend again,
as the database may well have one or more tables crashed due to the
lack of space. Fortunately MySQL and MariaDB are very forgiving about
what happens when the space runs out, so a simple repair normally
fixes everything. Unfortunately, mythbackend can appear to be running
normally when tables are crashed, but it will in fact be unable to
write to the crashed tables. There will normally be log messages
about SQL problems, but mythbackend will not otherwise notify you that
it has a problem.


Is the optimize_mythdb.pl script something that should be run regularly??

Jim A
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On 13/01/18 12:28, Jim Abernathy wrote:
>> Whenever you manage to have / run out of space, you should always make
>> sure you run:
>>
>> /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
>>
>> to repair and optimise the database before running mythbackend again,
>> as the database may well have one or more tables crashed due to the
>> lack of space. Fortunately MySQL and MariaDB are very forgiving about
>> what happens when the space runs out, so a simple repair normally
>> fixes everything. Unfortunately, mythbackend can appear to be running
>> normally when tables are crashed, but it will in fact be unable to
>> write to the crashed tables. There will normally be log messages
>> about SQL problems, but mythbackend will not otherwise notify you that
>> it has a problem.
> Is the optimize_mythdb.pl script something that should be run regularly??
>
> Jim A
>
https://www.mythtv.org/wiki/Optimize_mythdb.pl


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Sat, 13 Jan 2018 12:28:18 +0000, you wrote:

>
>Whenever you manage to have / run out of space, you should always make
>sure you run:
>
>/usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
>
>to repair and optimise the database before running mythbackend again,
>as the database may well have one or more tables crashed due to the
>lack of space. Fortunately MySQL and MariaDB are very forgiving about
>what happens when the space runs out, so a simple repair normally
>fixes everything. Unfortunately, mythbackend can appear to be running
>normally when tables are crashed, but it will in fact be unable to
>write to the crashed tables. There will normally be log messages
>about SQL problems, but mythbackend will not otherwise notify you that
>it has a problem.
>
>
>Is the optimize_mythdb.pl script something that should be run regularly??
>
>Jim A

Absolutely - run it daily. If you do not regularly do a check and
repair on mysql databases, you risk losing your tables to corruption.
The way it seem to work is that if there is a simple problem with a
table, the check and repair process can just fix it for you, with no
loss of data. But if a problem remains unfixed, and another problem
happens, that can then make the table unrepairable.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Jan 13, 2018, at 7:56 AM, Stephen Worthington <stephen_agent@jsw.gen.nz<mailto:stephen_agent@jsw.gen.nz>> wrote:

On Sat, 13 Jan 2018 12:28:18 +0000, you wrote:


Whenever you manage to have / run out of space, you should always make
sure you run:

/usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl

to repair and optimise the database before running mythbackend again,
as the database may well have one or more tables crashed due to the
lack of space. Fortunately MySQL and MariaDB are very forgiving about
what happens when the space runs out, so a simple repair normally
fixes everything. Unfortunately, mythbackend can appear to be running
normally when tables are crashed, but it will in fact be unable to
write to the crashed tables. There will normally be log messages
about SQL problems, but mythbackend will not otherwise notify you that
it has a problem.


Is the optimize_mythdb.pl script something that should be run regularly??

Jim A

Absolutely - run it daily. If you do not regularly do a check and
repair on mysql databases, you risk losing your tables to corruption.
The way it seem to work is that if there is a simple problem with a
table, the check and repair process can just fix it for you, with no
loss of data. But if a problem remains unfixed, and another problem
happens, that can then make the table unrepairable.

Thanks all,

I have setup cron.daily jobs for optimizing and backing up the database. I’ve tested them by manually running them:

sudo bash /etc/cron.daily/optimize_mythdb
sudo bash /etc/cron.daily/mythtv-backup

They do what I expect.

Following a lot of what has been said I decided to get rid of my USB key boot drive and cloned it to a 500GB Hard drive I had laying around.

Now I’ll be long dead before the daily logs fill up the 500GBs.

Thanks again, all

Jim A


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org<mailto:mythtv-users@mythtv.org>
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org<https://forum.mythtv.org/>
Re: controlling disk space on "/" on mythtv backend [ In reply to ]
On Sat, 13 Jan 2018 15:21:38 +0000, you wrote:

>Now I?ll be long dead before the daily logs fill up the 500GBs.

Sorry, no, there are all sorts of ways logs files can get out of hand
and grow until all the space is used up. It just takes a bit longer
with a bigger partition - hours instead of minutes.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org