Mailing List Archive

FFmpeg plans
To David and other devs

I have separated the MythTV configure from the FFmpeg configure so that
they no longer need to be merged when bringing in new FFmpeg versions.
see ticket https://code.mythtv.org/trac/ticket/13259 . I will commit
this to master shortly, once I see that no compile errors caused by this
come from the build slaves and my testing is all successful.

I would like to propose that we incorporate FFmpeg master branch into
MythTV instead of periodically merging release branches. Also I would
like to merge the FFmpeg git into ours, so that bringing in FFmpeg
updates becomes a git pull operation. I am not a git expert but I think
this should work:

1. Delete mythtv/external/FFmpeg in mythtv
From git root:
git rm -r mythtv/external/FFmpeg

2. Create a subtree of FFmpeg master
From git root:
git subtree add -P mythtv/external/FFmpeg
https://github.com/FFmpeg/FFmpeg.git master [--squash]

3. Apply our FFmpeg customizations and commit them.

4. Later, when we want to pull subsequent FFmpeg updates, run this
From git root
git subtree pull -P mythtv/external/FFmpeg
https://github.com/FFmpeg/FFmpeg.git master [--squash]

I will test this in the ffmpeg-resync branch first to make sure it all
works.

Notes: Many articles recommend the --squash flag to suppress importing
the history of the other project. Does anybody have opinions about this?

See
https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
and man git-subtree.

It is recommended but not necessary to separate commits to FFmpeg from
commits to the rest of the system. This is mainly in case you need to
submit updates back to the main FFmpeg repository, which we would not
likely be doing.

Please let me have your opinions or comments on this. Also, let me know
if I have missed something in git that may prevent this from working
correctly.

Peter


_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
What about the mpeg-ts demuxer?

Get Firefox on Android
________________________________
From: Peter Bennett
Sent: Saturday, 28 April 2018 22:08
To: David Engel; Development of MythTV
Subject: [mythtv] FFmpeg plans

To David and other devs

I have separated the MythTV configure from the FFmpeg configure so that
they no longer need to be merged when bringing in new FFmpeg versions.
see ticket https://code.mythtv.org/trac/ticket/13259 . I will commit
this to master shortly, once I see that no compile errors caused by this
come from the build slaves and my testing is all successful.

I would like to propose that we incorporate FFmpeg master branch into
MythTV instead of periodically merging release branches. Also I would
like to merge the FFmpeg git into ours, so that bringing in FFmpeg
updates becomes a git pull operation. I am not a git expert but I think
this should work:

1. Delete mythtv/external/FFmpeg in mythtv
From git root:
git rm -r mythtv/external/FFmpeg

2. Create a subtree of FFmpeg master
From git root:
git subtree add -P mythtv/external/FFmpeg
https://github.com/FFmpeg/FFmpeg.git master [--squash]

3. Apply our FFmpeg customizations and commit them.

4. Later, when we want to pull subsequent FFmpeg updates, run this
From git root
git subtree pull -P mythtv/external/FFmpeg
https://github.com/FFmpeg/FFmpeg.git master [--squash]

I will test this in the ffmpeg-resync branch first to make sure it all
works.

Notes: Many articles recommend the --squash flag to suppress importing
the history of the other project. Does anybody have opinions about this?

See
https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
and man git-subtree.

It is recommended but not necessary to separate commits to FFmpeg from
commits to the rest of the system. This is mainly in case you need to
submit updates back to the main FFmpeg repository, which we would not
likely be doing.

Please let me have your opinions or comments on this. Also, let me know
if I have missed something in git that may prevent this from working
correctly.

Peter


_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 04/28/2018 05:16 PM, Jean-Yves Avenard wrote:
> What about the mpeg-ts demuxer?
>
That is part of step 3. (Apply our FFmpeg customizations and commit
them). It includes adding files that were added to our copy of FFmpeg.
This is really the FFmpeg merge as I did before (without the configure
merge) but hopefully now for the last time because going forward we can
just pull from FFmpeg.

Do you see a problem with this?

Peter



> Get Firefox on Android
> ------------------------------------------------------------------------
> *From:* Peter Bennett
> *Sent:* Saturday, 28 April 2018 22:08
> *To:* David Engel; Development of MythTV
> *Subject:* [mythtv] FFmpeg plans
>
> To David and other devs
>
> I have separated the MythTV configure from the FFmpeg configure so that
> they no longer need to be merged when bringing in new FFmpeg versions.
> see ticket https://code.mythtv.org/trac/ticket/13259 . I will commit
> this to master shortly, once I see that no compile errors caused by this
> come from the build slaves and my testing is all successful.
>
> I would like to propose that we incorporate FFmpeg master branch into
> MythTV instead of periodically merging release branches. Also I would
> like to merge the FFmpeg git into ours, so that bringing in FFmpeg
> updates becomes a git pull operation. I am not a git expert but I think
> this should work:
>
> 1. Delete mythtv/external/FFmpeg in mythtv
> From git root:
> git rm -r mythtv/external/FFmpeg
>
> 2. Create a subtree of FFmpeg master
> From git root:
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
> 3. Apply our FFmpeg customizations and commit them.
>
> 4. Later, when we want to pull subsequent FFmpeg updates, run this
> From git root
> git subtree pull -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
> I will test this in the ffmpeg-resync branch first to make sure it all
> works.
>
> Notes: Many articles recommend the --squash flag to suppress importing
> the history of the other project. Does anybody have opinions about this?
>
> See
> https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
>
> and man git-subtree.
>
> It is recommended but not necessary to separate commits to FFmpeg from
> commits to the rest of the system. This is mainly in case you need to
> submit updates back to the main FFmpeg repository, which we would not
> likely be doing.
>
> Please let me have your opinions or comments on this. Also, let me know
> if I have missed something in git that may prevent this from working
> correctly.
>
> Peter
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
________________________________
From: Peter Bennett
Sent: Saturday, 28 April 2018 23:35
To: mythtv-dev@mythtv.org
Subject: Re: [mythtv] FFmpeg plans



On 04/28/2018 05:16 PM, Jean-Yves Avenard wrote:
>
> What about the mpeg-ts demuxer?
>
That is part of step 3. (Apply our FFmpeg customizations and commit them). It includes adding files that were added to our copy of FFmpeg. This is really the FFmpeg merge as I did before (without the configure merge) but hopefully now for the last time because going forward we can just pull from FFmpeg.

Do you see a problem with this?

Peter

No problem at all, just from experience every resync required manual tweaks there. Don't see how that could be automated.


BTW, 4.0 is out :)
>
> Get Firefox on Android
> ________________________________
> From: Peter Bennett
> Sent: Saturday, 28 April 2018 22:08
> To: David Engel; Development of MythTV
> Subject: [mythtv] FFmpeg plans
>
> To David and other devs
>
> I have separated the MythTV configure from the FFmpeg configure so that
> they no longer need to be merged when bringing in new FFmpeg versions.
> see ticket https://code.mythtv.org/trac/ticket/13259 . I will commit
> this to master shortly, once I see that no compile errors caused by this
> come from the build slaves and my testing is all successful.
>
> I would like to propose that we incorporate FFmpeg master branch into
> MythTV instead of periodically merging release branches. Also I would
> like to merge the FFmpeg git into ours, so that bringing in FFmpeg
> updates becomes a git pull operation. I am not a git expert but I think
> this should work:
>
> 1. Delete mythtv/external/FFmpeg in mythtv
> From git root:
> git rm -r mythtv/external/FFmpeg
>
> 2. Create a subtree of FFmpeg master
> From git root:
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
> 3. Apply our FFmpeg customizations and commit them.
>
> 4. Later, when we want to pull subsequent FFmpeg updates, run this
> From git root
> git subtree pull -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
> I will test this in the ffmpeg-resync branch first to make sure it all
> works.
>
> Notes: Many articles recommend the --squash flag to suppress importing
> the history of the other project. Does anybody have opinions about this?
>
> See
> https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
> and man git-subtree.
>
> It is recommended but not necessary to separate commits to FFmpeg from
> commits to the rest of the system. This is mainly in case you need to
> submit updates back to the main FFmpeg repository, which we would not
> likely be doing.
>
> Please let me have your opinions or comments on this. Also, let me know
> if I have missed something in git that may prevent this from working
> correctly.
>
> Peter
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 04/28/2018 06:07 PM, Jean-Yves Avenard wrote:
> *From:* Peter Bennett
> *Sent:* Saturday, 28 April 2018 23:35
> *To:* mythtv-dev@mythtv.org
> *Subject:* Re: [mythtv] FFmpeg plans
>
>
>
> On 04/28/2018 05:16 PM, Jean-Yves Avenard wrote:
>> What about the mpeg-ts demuxer?
>>
> That is part of step 3. (Apply our FFmpeg customizations and commit
> them). It includes adding files that were added to our copy of FFmpeg.
> This is really the FFmpeg merge as I did before (without the configure
> merge) but hopefully now for the last time because going forward we
> can just pull from FFmpeg.
>
> Do you see a problem with this?
>
> Peter
>
> No problem at all, just from experience every resync required manual
> tweaks there. Don't see how that could be automated.
I plan to try it out by bringing in a commit of FFmpeg from a month or
two ago and do the tweaks. Then try a pull of later commit from FFmpeg
to see how well it works. There could be conflicts, as there could with
any pull, that have to be fixed.

Peter
Re: FFmpeg plans [ In reply to ]
> On Apr 28, 2018, at 4:08 PM, Peter Bennett <pb.mythtv@gmail.com> wrote:
>
> ...
> 2. Create a subtree of FFmpeg master
> From git root:
> git subtree add -P mythtv/external/FFmpeg https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
For MacPorts, I rely on GitHub producing a tarball from a commit hash. MacPorts mirrors that tarball and it ensures that our build is reproducible.[1] I could be wrong but I don’t think a subtree would be included in the tarball. If you are going to test this, please check this out. Other packagers may also be affected.

OTOH, if the MythTV project wanted to do more frequent releases and produce tarballs, that would allelivate any concern.

Craig

[1] https://trac.macports.org/wiki/ReproducibleBuilds

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 04/28/2018 08:17 PM, Craig Treleaven wrote:
>> On Apr 28, 2018, at 4:08 PM, Peter Bennett <pb.mythtv@gmail.com> wrote:
>>
>> ...
>> 2. Create a subtree of FFmpeg master
>> From git root:
>> git subtree add -P mythtv/external/FFmpeg https://github.com/FFmpeg/FFmpeg.git master [--squash]
>>
> For MacPorts, I rely on GitHub producing a tarball from a commit hash. MacPorts mirrors that tarball and it ensures that our build is reproducible.[1] I could be wrong but I don’t think a subtree would be included in the tarball. If you are going to test this, please check this out. Other packagers may also be affected.
>
> OTOH, if the MythTV project wanted to do more frequent releases and produce tarballs, that would allelivate any concern.
>
> Craig
>
> [1] https://trac.macports.org/wiki/ReproducibleBuilds
>
> _______________________________________________
>
Hi Craig

Thanks for the info. I believe it will include the subtree in the
tarball. I will make sure to test it to make sure it is included.
I see the way to get a tarball is using a URL like this
https://github.com/MythTV/mythtv/archive/devel/ffmpeg-resync.tar.gz .

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Sat, 2018-04-28 at 16:08 -0400, Peter Bennett wrote:
>
> 2. Create a subtree of FFmpeg master
> From git root:
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]

I use https://github.com/ingydotnet/git-subrepo in another project with
good success. If you do use it, be sure to use the release/0.4.0
branch.

> 3. Apply our FFmpeg customizations and commit them.

Why are these not upstreamed? Are they not generally useful?

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
On Sun, 2018-04-29 at 11:21 -0400, Peter Bennett wrote:
>
> Thanks for the info. I believe it will include the subtree in the
> tarball.

Well, to be pedantic, git sub[tree|repo] pulls a *copy* of the git
project it is subbing into the git project it is subbing into (if that
made sense). git submodule is the command that puts a "pointer" to a
sha1 of the project it is submodule'ing into the project it is subbing
into.

Think of them like "cp -r" and "ln -s" respectively.

As long as whatever makes the tarball descends the directory created by
git sub[tree|repo], the subdir should be in the tarball, and since this
is what happens currently, there should be no change there.

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
On 04/29/2018 11:55 AM, Brian J. Murrell wrote:
> On Sat, 2018-04-28 at 16:08 -0400, Peter Bennett wrote:
>> 2. Create a subtree of FFmpeg master
>> From git root:
>> git subtree add -P mythtv/external/FFmpeg
>> https://github.com/FFmpeg/FFmpeg.git master [--squash]
> I use https://github.com/ingydotnet/git-subrepo in another project with
> good success. If you do use it, be sure to use the release/0.4.0
> branch.
I looked at it and I am not too clear what advantages it offers us over
subtree. One advantage I see is it keeps commits to the main code
separate from those to the subrepo, thereby making it easier to send
back upstream changes. However we do not have the ability to push
changes back to FFmpeg. I think the developers may be reluctant to use a
third-party component like this unless there are very clear advantages.
>> 3. Apply our FFmpeg customizations and commit them.
> Why are these not upstreamed? Are they not generally useful?
These have been in place for a long time and developers who originally
created them have submitted them to FFmpeg a long time ago but they were
never accepted.

Peter

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Sun, 2018-04-29 at 12:47 -0400, Peter Bennett wrote:
>
> These have been in place for a long time and developers who
> originally
> created them have submitted them to FFmpeg a long time ago but they
> were
> never accepted.

If it was a long (enough) time ago, it might be worthwhile trying to
submit them again. With enough time passing, their general usefullness
might be more clear to upstream and/or the upstream developers may have
changed enough such that they might have a more liberal view of the
changes now, etc.

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
They won't be.

Our changes adds callback to detect things like resolution change, something ffmpeg moved away from a long time ago.

The biggest thing would be to update *their* mpeg-ts demuxer to support our extra features. And that's no trivial task as the two mpeg-ts demuxers have diverged significantly over the years, they almost have nothing in common these days.

Get Firefox on Android
________________________________
From: "Brian J. Murrell" <brian@interlinx.bc.ca>
Sent: Sunday, 29 April 2018 19:47
To: mythtv-dev@mythtv.org
Subject: Re: [mythtv] FFmpeg plans

On Sun, 2018-04-29 at 12:47 -0400, Peter Bennett wrote:
>
> These have been in place for a long time and developers who
> originally
> created them have submitted them to FFmpeg a long time ago but they
> were
> never accepted.

If it was a long (enough) time ago, it might be worthwhile trying to
submit them again.  With enough time passing, their general usefullness
might be more clear to upstream and/or the upstream developers may have
changed enough such that they might have a more liberal view of the
changes now, etc.

Cheers,
b.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Sat, Apr 28, 2018 at 04:08:19PM -0400, Peter Bennett wrote:
> To David and other devs
>
> I have separated the MythTV configure from the FFmpeg configure so that they
> no longer need to be merged when bringing in new FFmpeg versions. see ticket
> https://code.mythtv.org/trac/ticket/13259 . I will commit this to master
> shortly, once I see that no compile errors caused by this come from the
> build slaves and my testing is all successful.
>
> I would like to propose that we incorporate FFmpeg master branch into MythTV
> instead of periodically merging release branches. Also I would like to merge
> the FFmpeg git into ours, so that bringing in FFmpeg updates becomes a git
> pull operation. I am not a git expert but I think this should work:
>
> 1. Delete mythtv/external/FFmpeg in mythtv
> From git root:
> git rm -r mythtv/external/FFmpeg
>
> 2. Create a subtree of FFmpeg master
> From git root:
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
> 3. Apply our FFmpeg customizations and commit them.
>
> 4. Later, when we want to pull subsequent FFmpeg updates, run this
> From git root
> git subtree pull -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git master [--squash]
>
> I will test this in the ffmpeg-resync branch first to make sure it all
> works.
>
> Notes: Many articles recommend the --squash flag to suppress importing the
> history of the other project. Does anybody have opinions about this?
>
> See
> https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
> and man git-subtree.
>
> It is recommended but not necessary to separate commits to FFmpeg from
> commits to the rest of the system. This is mainly in case you need to submit
> updates back to the main FFmpeg repository, which we would not likely be
> doing.
>
> Please let me have your opinions or comments on this. Also, let me know if I
> have missed something in git that may prevent this from working correctly.

I'm not familiar enough with the differences between using subtree and
submodule to comment about that yet.

Likewise, I'm not sure that FFmpeg's master branch is the best choice.
How do you propose to handle post-release fixes from FFmpeg? Looking
at FFmpeg's release branch for 3.4, it saw a lot of fixes spanning 4
months after the initial releases. Perhaps Aman could shed some light
on how FFmpeg handles their release and master branches and suggest
the best one to track.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 04/29/2018 04:08 PM, David Engel wrote:
> Likewise, I'm not sure that FFmpeg's master branch is the best choice.
> How do you propose to handle post-release fixes from FFmpeg? Looking
> at FFmpeg's release branch for 3.4, it saw a lot of fixes spanning 4
> months after the initial releases. Perhaps Aman could shed some light
> on how FFmpeg handles their release and master branches and suggest
> the best one to track.
Hi David

I looked at https://ffmpeg.org/download.html#get-sources
They recommend in two places there that people compiling from source
should use the development (master) branch. They also state that fixes
are first applied to master and then some of them are cherry-picked into
release branches.

If we use a release branch, I do not know if the subtree approach will
work. When we move from one release branch to the next release branch, I
don't know if pulling from a different branch would work.

Also, regarding post-release fixes, AFAIK we currently do not take them.
I have not merged in any post-release fixes for 3.4. If we use master we
can do another pull any time to get fixes.

Peter

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Sun, 2018-04-29 at 12:11 -0400, Brian J. Murrell wrote:
> On Sun, 2018-04-29 at 11:21 -0400, Peter Bennett wrote:
> >
> > Thanks for the info. I believe it will include the subtree in the
> > tarball.
>
> Well, to be pedantic, git sub[tree|repo] pulls a *copy* of the git
> project it is subbing into the git project it is subbing into (if
> that
> made sense). git submodule is the command that puts a "pointer" to a
> sha1 of the project it is submodule'ing into the project it is
> subbing
> into.

If I recall correctly, a git submodule requires write permission on the
other repository (e.g. FFmpeg) in order to commit and changes/tags/etc.
That's probably a non-starter.

I'd love to hear more about your experience with subrepos. I've been
meaning to update many of the other external components, and haven't
done so yet because I haven't had time to fully research submodules vs.
subtrees (and now vs. subrepos). Whichever we decide to use, we should
use it consistently across all the external modules.

David



> Think of them like "cp -r" and "ln -s" respectively.
>
> As long as whatever makes the tarball descends the directory created
> by
> git sub[tree|repo], the subdir should be in the tarball, and since
> this
> is what happens currently, there should be no change there.
>
> Cheers,
> b.
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 28/04/18 23:58, Peter Bennett wrote:
>
>
> On 04/28/2018 06:07 PM, Jean-Yves Avenard wrote:
>> *From:* Peter Bennett
>> *Sent:* Saturday, 28 April 2018 23:35
>> *To:* mythtv-dev@mythtv.org
>> *Subject:* Re: [mythtv] FFmpeg plans
>>
>>
>>
>> On 04/28/2018 05:16 PM, Jean-Yves Avenard wrote:
>>> What about the mpeg-ts demuxer?
>>>
>> That is part of step 3. (Apply our FFmpeg customizations and commit
>> them). It includes adding files that were added to our copy of FFmpeg.
>> This is really the FFmpeg merge as I did before (without the configure
>> merge) but hopefully now for the last time because going forward we
>> can just pull from FFmpeg.
>>
>> Do you see a problem with this?
>>
>> Peter
>>
>> No problem at all, just from experience every resync required manual
>> tweaks there. Don't see how that could be automated.
> I plan to try it out by bringing in a commit of FFmpeg from a month or
> two ago and do the tweaks. Then try a pull of later commit from FFmpeg
> to see how well it works. There could be conflicts, as there could with
> any pull, that have to be fixed.
>

Can you apply your own patches to an external tree in git?
That would be a requirement for this to work surely?


Regards
Stuart


> Peter
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 30/04/18 09:15, Stuart Auchterlonie wrote:

> On 28/04/18 23:58, Peter Bennett wrote:
>>
>>
>> On 04/28/2018 06:07 PM, Jean-Yves Avenard wrote:
>>> *From:* Peter Bennett
>>> *Sent:* Saturday, 28 April 2018 23:35
>>> *To:* mythtv-dev@mythtv.org
>>> *Subject:* Re: [mythtv] FFmpeg plans
>>>
>>>
>>>
>>> On 04/28/2018 05:16 PM, Jean-Yves Avenard wrote:
>>>> What about the mpeg-ts demuxer?
>>>>
>>> That is part of step 3. (Apply our FFmpeg customizations and commit
>>> them). It includes adding files that were added to our copy of
>>> FFmpeg. This is really the FFmpeg merge as I did before (without the
>>> configure merge) but hopefully now for the last time because going
>>> forward we can just pull from FFmpeg.
>>>
>>> Do you see a problem with this?
>>>
>>> Peter
>>>
>>> No problem at all, just from experience every resync required manual
>>> tweaks there. Don't see how that could be automated.
>> I plan to try it out by bringing in a commit of FFmpeg from a month
>> or two ago and do the tweaks. Then try a pull of later commit from
>> FFmpeg to see how well it works. There could be conflicts, as there
>> could with any pull, that have to be fixed.
>>
>
> Can you apply your own patches to an external tree in git?
> That would be a requirement for this to work surely?
>
>
> Regards
> Stuart
>

I think that could be a problem since you need write access to the
upstream project to commit them. What I do is fork the external project
and then add my own fork as the submodule. It's easy to keep the fork in
sync and you only have to do it when it's convenient. You don't want to
pull a big update from upstream just before you do a release for example.

I'm in no way a git expert so there may well be more elegant ways of
doing it and  usually when git is involved I suspect if you ask six
developers the best way to do it they will tell you six different answers :)

Paul H.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 09:15 +0100, Stuart Auchterlonie wrote:
>
> Can you apply your own patches to an external tree in git?

As a sub{tree|repo} it's not external -- that's a git submodule. A git
sub{tree|repo} is a full copy of the external source in your own repo.
You are just using git sub{tree|repo} to manage (i.e. forking,
updating, etc.) it like you would any git repo you might not have
commit privileges with.

So just like you can fork any git repo, apply your own patches and push
those to your own fork, then periodically pull in changes since your
fork from upstream, commit those, rinse, repeat, you can do the same
with a git sub{tree|repo}.

> That would be a requirement for this to work surely?

Of course.

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 10:04 +0100, Paul Harrison wrote:
>
> I think that could be a problem since you need write access to the
> upstream project to commit them.

No. The sub{tree|repo} is a fork, not a pointer, to the upstream
module.

> What I do is fork the external project
> and then add my own fork as the submodule.

That is exactly what git sub{tree|repo} is. It's really just tooling
around that workflow, keeping track of where you last synced from so
that you can pull changes since then, etc.

> It's easy to keep the fork in
> sync and you only have to do it when it's convenient.

Right. Exactly what git sub{tree|repo} does also. You decide when to
git sub{tree|repo} pull from upstream.

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 00:14 -0400, David Hampton wrote:
>
> If I recall correctly, a git submodule requires write permission on
> the
> other repository (e.g. FFmpeg) in order to commit and
> changes/tags/etc.

Yes, that's because with git submodule, you only put a reference to a
sha1 of the repo you are setting up as a submodule. Think of this like
creating a symlink to a directory somewhere else.

With a sub{tree|repo} you actually create an entire copy of the source
of the module you are subbing in your own repo. Think of this like
creating a recursive copy of that directory somewhere else.

> That's probably a non-starter.

Of course. Particularly where the remote repository maintainers are
unwilling to merge your changes.

> I'd love to hear more about your experience with subrepos.

It's been fine. Using git subrepo (URL previously posted in this
thread), I did find I needed to switch to the 0.4.0 release branch
pretty quickly as the 0.3.0 branch was giving me problems. But I would
not entirely rule out that that could have just been learning curve
with the limitations of git subrepo (probably applies to git subtree
also) and rebasing commits for/from the subrepo.

> I've been
> meaning to update many of the other external components, and haven't
> done so yet because I haven't had time to fully research submodules
> vs.
> subtrees (and now vs. subrepos). Whichever we decide to use, we
> should
> use it consistently across all the external modules.

Agreed, well mostly. I do still have a like for submodules, where they
fit, particularly if I am the maintainer of both the parent and child
modules.

One use case for using git sub* is not just to fork and keep changes in
an external repo but to keep code that is common to many projects in
one git module. For this use-case, I tend to like git submodule better
as it truly does just mean maintaining the common code in one place and
then adding git submodule (references, not copies) to the projects that
want to use the module. Because changes *have* to be made in the
module directly, it instills a workflow that requires changes be made
to the module itself and doesn't allow consumers of the module to keep
local changes (i.e. forget to push them back to the original module).

Where you want to keep local changes and don't have commit rights to
the module you are consuming, git sub{tree|repo} are the better tool.
The one thing that does grate me the wrong with with git sub{tree|repo}
is the duplication. I'm efficiency-OCD and hate waste so duplicating
an entire source tree feels wasteful to me.

As for which of git sub{tree|repo} to use, they both achieve the same
goal. git subrepo is supposed to just be a better git subtree. From
the README there:


This command is an improvement from git-submodule and git-subtree;
two other git commands with similar goals, but various problems.

I'll let you decide if subrepo is better than subtree since it's pretty
subjective.

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
Thank you Brian Murrell for the helpful information.

This is my take on the whole submodule/subtree/subrepo discussion:

1. submodule would incorporate a reference to the FFmpeg repository in
MythTV. This would not work because we have many customizations of
FFmpeg. It could work if we created first our own FFmpeg repository. I
do not like that idea.

2. subtree includes a copy of FFmpeg's repo in our repo. We can
customize the code as we need. We can also pull newer versions from
FFmpeg and it will merge in the changes. Sometimes you may need to
resolve conflicts if they have changed code that we have customized.

3. subrepo is a third party tool
(https://github.com/ingydotnet/git-subrepo) that seems to be almost the
same as subtree. One difference is that if you make a subsequent commit
that includes changes to MythTV as well as to FFmpeg, subrepo separates
them into two commits. It also keeps history of the subtrepo separate
from the main history. My feeling about subrepo is that I would rather
stick with built-in features of git (subtree) than rely on a third party
addon that may not be supported in future.

I believe that using subtree or subrepo requires that we use the master
branch of FFmpeg. I do not see a problem with using FFmpeg master
together with MythTV master. The FFmpeg web page recommends using master
if you are compiling FFmpeg yourself.

I will test subtree in my own repository first to make sure it behaves
as expected.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Sun, Apr 29, 2018 at 04:44:53PM -0400, Peter Bennett wrote:
>
>
> On 04/29/2018 04:08 PM, David Engel wrote:
> > Likewise, I'm not sure that FFmpeg's master branch is the best choice.
> > How do you propose to handle post-release fixes from FFmpeg? Looking
> > at FFmpeg's release branch for 3.4, it saw a lot of fixes spanning 4
> > months after the initial releases. Perhaps Aman could shed some light
> > on how FFmpeg handles their release and master branches and suggest
> > the best one to track.
> Hi David
>
> I looked at https://ffmpeg.org/download.html#get-sources
> They recommend in two places there that people compiling from source should
> use the development (master) branch. They also state that fixes are first
> applied to master and then some of them are cherry-picked into release
> branches.

That recommendation seems highly unusual to me. Why do releases at
all then? It sounds like thay handle release branches in the same,
pretty standard way we do. We try to keep our master branch stable
too, but we don't recommend others run it. I wonder why they do.

David

> If we use a release branch, I do not know if the subtree approach will work.
> When we move from one release branch to the next release branch, I don't
> know if pulling from a different branch would work.
>
> Also, regarding post-release fixes, AFAIK we currently do not take them. I
> have not merged in any post-release fixes for 3.4. If we use master we can
> do another pull any time to get fixes.
>
> Peter
>

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
Hi

> On 30 Apr 2018, at 5:45 pm, David Engel <david@istwok.net> wrote:
>
> That recommendation seems highly unusual to me. Why do releases at
> all then? It sounds like thay handle release branches in the same,
> pretty standard way we do. We try to keep our master branch stable
> too, but we don't recommend others run it. I wonder why they do.
>
> David

Because some developers never ever introduce regressions and always produce bug free code… doh !
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 11:05 -0400, Peter Bennett wrote:
> Thank you Brian Murrell for the helpful information.

NP.

> 1. submodule would incorporate a reference to the FFmpeg repository
> in
> MythTV.

Correct.

> This would not work because we have many customizations of
> FFmpeg.

Correct.

> It could work if we created first our own FFmpeg repository.

Right. Which is just taking the same approach as sub{tree|repo} but
doing the work tracking yourself manually.

> I
> do not like that idea.

Indeed. There is nothing about it to like. :-)

> 2. subtree includes a copy of FFmpeg's repo in our repo.

Correct. Although I don't think it's the whole "repo", where a repo is
every object ever created in a project and so includes all branches,
tags, etc. Rather I think git subtree (and subrepo) are just taking
the content of a single branch and copying it to your subdir.

> We can
> customize the code as we need.

Correct.

> We can also pull newer versions from
> FFmpeg and it will merge in the changes.

Correct.

> Sometimes you may need to
> resolve conflicts if they have changed code that we have customized.

Correct.

> 3. subrepo is a third party tool
> (https://github.com/ingydotnet/git-subrepo) that seems to be almost
> the
> same as subtree.

Correct.

> One difference is that if you make a subsequent commit
> that includes changes to MythTV as well as to FFmpeg, subrepo
> separates
> them into two commits.

TBH, I'm not entirely sure about that. I generally use subrepo in the
submodule sense, where I never commit changes to the subrepo'd dir in a
consuming project but instead commit them upstream and then pull them
back down to the consumers.

For my use-case, in fact, I would actually be using submodule except
that that requires users of my module to know there is a submodule and
do the git submodule init/blah-blah and I don't want my users to know
they have to that and for CI systems to know they have to do it.

> My feeling about subrepo is that I would
> rather
> stick with built-in features of git (subtree) than rely on a third
> party
> addon that may not be supported in future.

That's fair enough. But given that both are just keeping a copy of the
external module in a subdir locally in your module, I don't think there
is anything stopping you from switching at some point in the future.
You might just have to do a bit of gluing of one method's metadata to
the other. Or even simpler, you just make a patch of your changes vs.
upstream, blow the subdir away and then use a different method (i.e.
converting from subrepo to subtree) to recreate the subdir and then
apply your patch and you are switched.

> I believe that using subtree or subrepo requires that we use the
> master
> branch of FFmpeg.

I'm not convinced of that. At least with subrepo, you can choose which
branch you clone and track into the subdir. If/when you want to change
the upstream tracking branch, worst case scenario (assuming the tool
doesn't have a way to change) is you do as above. Create your patch
vs. upstream, blow the subdir away, recreate the subdir using
{subtree|submodule} and then apply your patch, fixing any conflicts,
etc.

Cheers
b.
Re: FFmpeg plans [ In reply to ]
On 04/30/2018 11:45 AM, David Engel wrote:
> That recommendation seems highly unusual to me. Why do releases at
> all then? It sounds like thay handle release branches in the same,
> pretty standard way we do. We try to keep our master branch stable
> too, but we don't recommend others run it. I wonder why they do.
>
> David
We can treat their master like our master. Before we tag a release
branch make sure it has been stable with a particular pull of FFmpeg. In
the pre-release branch we can do testing and cherry-pick additional
FFmpeg commits or reverse bad FFmpeg commits as necessary, as we do with
MythTV commits, if needed to fix bugs.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 11:52 -0400, Brian J. Murrell wrote:
> 2. subtree includes a copy of FFmpeg's repo in our repo.
>
> Correct. Although I don't think it's the whole "repo", where a repo is
> every object ever created in a project and so includes all branches,
> tags, etc. Rather I think git subtree (and subrepo) are just taking
> the content of a single branch and copying it to your subdir.

AIUI technically what it does (without --squash) is take the tree-sha
of the tree (the datastructure which is the git equivalent to a
directory) pointed to by an upstream commit (perhaps/probably one on a
branch) and grafts it in directly as an entry in a sub-tree (i.e. a
subdirectory) of the tree of the downstream repo.

Ian.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 04/30/2018 11:52 AM, Brian J. Murrell wrote:
>> I believe that using subtree or subrepo requires that we use the
>> master
>> branch of FFmpeg.
> I'm not convinced of that. At least with subrepo, you can choose which
> branch you clone and track into the subdir. If/when you want to change
> the upstream tracking branch, worst case scenario (assuming the tool
> doesn't have a way to change) is you do as above. Create your patch
> vs. upstream, blow the subdir away, recreate the subdir using
> {subtree|submodule} and then apply your patch, fixing any conflicts,
> etc.
This is essentially what we do at the moment. We delete the source code,
copy in the new source, and then work at re-applying all of our changes.
I am trying to improve on that process, which is rather time-consuming.

Peter
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 12:37 -0400, Peter Bennett wrote:
>
> This is essentially what we do at the moment. We delete the source
> code,
> copy in the new source, and then work at re-applying all of our
> changes.
> I am trying to improve on that process, which is rather time-
> consuming.

Sure, but this is worst-case scenario. I have not tried to switch
branches with git subrepo (nor git subtree as I have not even used
that). Maybe it's easier than that. But it's also only something you
would ever have to do (again, in worst-case) when switching upstream
tracking (i.e. release) branches, so I think you are still better off.

At least you have git a workflow and sub{repo|tree} bookkeeping your
updating for you. Overall it should make keeping your ffmpeg subdir
up-to-date with upstream trivial, and the more frequently you refresh
(git subrepo pull) it the smaller the potential conflicts are.

If it were me, I'd set up a job daily or weekly just to refresh from
upstream to keep the potential for conflicts as entirely small as
possible.

Cheers,
b.
Re: FFmpeg plans [ In reply to ]
On 04/30/2018 01:52 PM, Brian J. Murrell wrote:
> Sure, but this is worst-case scenario. I have not tried to switch
> branches with git subrepo (nor git subtree as I have not even used
> that). Maybe it's easier than that. But it's also only something you
> would ever have to do (again, in worst-case) when switching upstream
> tracking (i.e. release) branches, so I think you are still better off.
Currently the only time we do any updating at all is when switching
upstream branches. So, no improvement.


> At least you have git a workflow and sub{repo|tree} bookkeeping your
> updating for you. Overall it should make keeping your ffmpeg subdir
> up-to-date with upstream trivial, and the more frequently you refresh
> (git subrepo pull) it the smaller the potential conflicts are.

> If it were me, I'd set up a job daily or weekly just to refresh from
> upstream to keep the potential for conflicts as entirely small as
> possible.
We need to decide a strategy. Currently we only refresh FFmpeg once a
year or so. I would not want to do any refresh shortly before a new
MythTV release because we would not want to possibly introduce
unexpected problems at the last minute.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
> On 30 Apr 2018, at 7:52 pm, Brian J. Murrell <brian@interlinx.bc.ca> wrote:
>
> On Mon, 2018-04-30 at 12:37 -0400, Peter Bennett wrote:
>>
>> This is essentially what we do at the moment. We delete the source
>> code,
>> copy in the new source, and then work at re-applying all of our
>> changes.
>> I am trying to improve on that process, which is rather time-
>> consuming.
>
> Sure, but this is worst-case scenario. I have not tried to switch
> branches with git subrepo (nor git subtree as I have not even used
> that). Maybe it's easier than that. But it's also only something you
> would ever have to do (again, in worst-case) when switching upstream
> tracking (i.e. release) branches, so I think you are still better off.


I’ve never seen a case where ffmpeg didn’t break API between version.

Updating automatically ffmpeg from a subrepo is just not going to work, there’s just no project that does what you’re stating. Every single project checkout a particular version of ffmpeg and stick to that version until the next stable release. It’s the one it’s tested against, and patched if necessary.

The only exception to this I can think of is mplayer, and ever attempted to checkout an older version of mplayer (which always pull the latest version of ffmpeg) ? Good luck. mplayer current checkout will work with ffmpeg of the day, never day to perform a bisect or attempt to lookup for a regression.

So not having to continually rebase our changes and particularly configure is one thing.

but all this talk about automated, git subtree vs subrepo and so forth, I’m sorry to say, clearly show you’ve really have never worked with ffmpeg before. Otherwise we wouldn’t have reached 24 messages in this thread.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Mon, 2018-04-30 at 20:08 +0200, Jean-Yves Avenard wrote:
>
> but all this talk about automated, git subtree vs subrepo and so
> forth, I’m sorry to say, clearly show you’ve really have never worked
> with ffmpeg before.

No I haven't. I never claimed to have and I find your condescending
and dismissing tone about offensive. But I work with lots of other
upstream projects in my real-life job even so I am not completely
talking out of my ass here.

I was providing input about git sub* tool experience. But if that
experience is not really appreciated and is going to be met with such
an attitude I will just keep it to myself, OK?

b.
Re: FFmpeg plans [ In reply to ]
There seems to be a little confusion concerning how git subtree works.
It doesn't do anything weird to the subdirectory it's applied to: the
only difference anyone would notice in the mythtv repo, if using git
subtree, is that the updates to the ffmpeg subdir would appear as merges
and some of the commits in the branches merged would mysteriously refer
to files within the ffmpeg subdir without the "external/FFmpeg" prefix.
Using git subtree would not make the mythtv repo rely on the ffmpeg
repo. It would just make them share some sha-1 hashes in a useful way.

Before git subtree existed, it was possible to have a local repo for one
project include remotes from an entirely different project. Without git
subtree it wasn't useful to do so because if you checked out on a branch
from one project all the files from the other project would disappear.
git subtree is just a useful way of using remotes from different
projects in the case that one project represents the files of a
subdirectory of another project.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
FFmpeg plans [ In reply to ]
In my own copy of MythTV repository, I tried using a subtree to include
FFmpeg. I hit up against a few problems -

I had wanted to try importing a particular commit from FFmpeg, but I was
unable to do that. I could only import the latest version of a branch.
Also I could see no way of cherry-picking FFmpeg commits into our copy.

git subtree add -P mythtv/external/FFmpeg
https://github.com/FFmpeg/FFmpeg.git 7e3a070 --squash
fatal: Couldn't find remote ref 7e3a070

I tried adding it as a remote, with the same result

git remote add ffmpeg https://github.com/FFmpeg/FFmpeg.git
git fetch ffmpeg
git subtree add -P mythtv/external/FFmpeg ffmpeg 7e3a070 --squash
fatal: Couldn't find remote ref 7e3a070

If you replace the commit id with a branch name it works

git subtree add -P mythtv/external/FFmpeg
https://github.com/FFmpeg/FFmpeg.git release/4.0 --squash
worked

I do not want to be forever constrained to only updating to the latest
commit of a branch in FFmpeg.

I have now created my own copy of FFmpeg repository and applied our
changes to that. Perhaps this is a better solution. We could maintain an
ffmpeg repository in MythTV and either copy from it as needed after
merging from the main FFmpeg, or else use a subtree based on our own
FFmpeg. We could create a mythtv-master branch in the private FFmpeg for
the subtree, and merge or cherry-pick to that as needed. This way we
would a better workfolw and accurate tracking of who contributed changes
in future. Currently the tracking of contributions to MythTV's copy of
FFmpeg is inaccurate, incorrectly showing most things having been
contributed by the various people who have done the FFmpeg resync.

We could also make a submodule instead of a subtree from our private
FFmpeg, but that means that people who access MythTV source would have
to run an explicit command to pull in the submodule data and to pull in
updates from it when needed.

On the current merge:
I am using FFmpeg master as of yesterday. The recent fixes to mediacodec
are not in the release branch. After applying the MythTV modifications
(5680 lines of patches) there were 12 files with conflicts. Most were
easy to fix, but FFmpeg has redesigned the code for registering demuxers
so I had to make some changes to the code for registering
ff_mythtv_mpegts_demuxer and ff_mythtv_mpegtsraw_demuxer.

It now has compile errors in the mythtv-mpegts demuxer code caused by
undeclared functions and macros, presumably deprecated things removed
from FFmpeg. I will have to fix these. The mythtv-mpegts demuxer code
also uses a lot of deprecated functions.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Fri, May 11, 2018 at 01:51:26PM -0400, Peter Bennett wrote:
> In my own copy of MythTV repository, I tried using a subtree to include
> FFmpeg. I hit up against a few problems -
>
> I had wanted to try importing a particular commit from FFmpeg, but I was
> unable to do that. I could only import the latest version of a branch. Also
> I could see no way of cherry-picking FFmpeg commits into our copy.
>
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git 7e3a070 --squash
> fatal: Couldn't find remote ref 7e3a070
>
> I tried adding it as a remote, with the same result
>
> git remote add ffmpeg https://github.com/FFmpeg/FFmpeg.git
> git fetch ffmpeg
> git subtree add -P mythtv/external/FFmpeg ffmpeg 7e3a070 --squash
> fatal: Couldn't find remote ref 7e3a070
>
> If you replace the commit id with a branch name it works
>
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git release/4.0 --squash
> worked
>
> I do not want to be forever constrained to only updating to the latest
> commit of a branch in FFmpeg.

I have one idea, but it's not without its own share of warts. That is
to treat the ffmpeg-master branch (and possibly others) like tracking
branches. It could go something like this.

# Remove the existing external/FFmpeg from the mythtv repo.
git rm -r external/FFmpeg

# Create an empty ffmpeg-master branch in the mythtv repo.
git co ce7a5f62 (the epoch commit with an empty repo>
git branch ffmpeg-master
git co ffmpeg-master

# Pull in ffmpeg master branch and move it to the right place.
git pull --allow-unrelated-histories https://github.com/FFmpeg/FFmpeg.git master
mkdir external/FFmpeg
git mv <list of files> external/FFmpeg

# Merge the ffmpeg-master branch into mythtv master.
git co master
git merge ffmpeg-master

One wart is that anytime ffmpeg adds new files in their root, we would
have to notice them when we pull and git mv them to external/FFmpeg.

Another wart is we would have to do the same with any other ffmpeg
branches that we might want to cherry-pick from.

> I have now created my own copy of FFmpeg repository and applied our changes
> to that. Perhaps this is a better solution. We could maintain an ffmpeg
> repository in MythTV and either copy from it as needed after merging from
> the main FFmpeg, or else use a subtree based on our own FFmpeg. We could
> create a mythtv-master branch in the private FFmpeg for the subtree, and
> merge or cherry-pick to that as needed. This way we would a better workfolw
> and accurate tracking of who contributed changes in future. Currently the
> tracking of contributions to MythTV's copy of FFmpeg is inaccurate,
> incorrectly showing most things having been contributed by the various
> people who have done the FFmpeg resync.
>
> We could also make a submodule instead of a subtree from our private FFmpeg,
> but that means that people who access MythTV source would have to run an
> explicit command to pull in the submodule data and to pull in updates from
> it when needed.
>
> On the current merge:
> I am using FFmpeg master as of yesterday. The recent fixes to mediacodec are
> not in the release branch. After applying the MythTV modifications (5680
> lines of patches) there were 12 files with conflicts. Most were easy to fix,
> but FFmpeg has redesigned the code for registering demuxers so I had to make
> some changes to the code for registering ff_mythtv_mpegts_demuxer and
> ff_mythtv_mpegtsraw_demuxer.
>
> It now has compile errors in the mythtv-mpegts demuxer code caused by
> undeclared functions and macros, presumably deprecated things removed from
> FFmpeg. I will have to fix these. The mythtv-mpegts demuxer code also uses a
> lot of deprecated functions.

FFmpeg really is a moving target, isn't it! :( Is it worth trying
again to get some of our changes accepted upstream?

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Fri, 2018-05-11 at 13:51 -0400, Peter Bennett wrote:
> In my own copy of MythTV repository, I tried using a subtree to include
> FFmpeg. I hit up against a few problems -
>
> I had wanted to try importing a particular commit from FFmpeg, but I was
> unable to do that. I could only import the latest version of a branch.
> Also I could see no way of cherry-picking FFmpeg commits into our copy.
>
> git subtree add -P mythtv/external/FFmpeg
> https://github.com/FFmpeg/FFmpeg.git 7e3a070 --squash
> fatal: Couldn't find remote ref 7e3a070
>
> I tried adding it as a remote, with the same result
>
> git remote add ffmpeg https://github.com/FFmpeg/FFmpeg.git
> git fetch ffmpeg
> git subtree add -P mythtv/external/FFmpeg ffmpeg 7e3a070 --squash
> fatal: Couldn't find remote ref 7e3a070

There are two forms of `git subtree add`:
> git subtree add -P <prefix> <commit>
> git subtree add -P <prefix> <repository> <ref>

You've tried the second one but I think you need the first, in a slight
modification to your second attempt with a remote:

ijc@dagon:mythtv-subtree-test$ git remote add ffmpeg https://github.com/FFmpeg/FFmpeg.git
ijc@dagon:mythtv-subtree-test$ git fetch -n ffmpeg
warning: no common commits
[...]
ijc@dagon:mythtv-subtree-test$ git subtree add -P mythtv/external/FFmpeg-test 7e3a070 --squash
Added dir 'mythtv/external/FFmpeg-test'
ijc@dagon:mythtv-subtree-test$ ls mythtv/external/FFmpeg-test
Changelog COPYING.GPLv2 CREDITS INSTALL.md libavformat/ libswresample/ Makefile tests/
compat/ COPYING.GPLv3 doc/ libavcodec/ libavresample/ libswscale/ presets/ tools/
configure* COPYING.LGPLv2.1 ffbuild/ libavdevice/ libavutil/ LICENSE.md README.md
CONTRIBUTING.md COPYING.LGPLv3 fftools/ libavfilter/ libpostproc/ MAINTAINERS RELEASE

(I fudged the prefix because it already exists, but I guess you have
already done "git rm -r" to get rid of the old copy in your tree, I
used -n so I wouldn't get all the ffmpeg tags in my local repo). You
could avoid the remote by just doing:

git fetch https://github.com/FFmpeg/FFmpeg.git <branch|commit>

instead of the `git remote add foo` + `git fetch -n foo`.

I've not looked at how this differs from the subtree merge command:

> git subtree merge -P <prefix> <commit>

but I guess this is the one you want going forward after the initial
add.

For cherry-picking there is a process given in https://stackoverflow.co
m/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
(second half of first answer) which looks pretty plausible to me, but
it does rely on not using `--squash` for the regular merges (although
possibly it could be adapted).

TBH I'd recommend not using squash anyway, it'll give a more accurate
picture of the history which is useful for people doing archaeology
which crosses into the ffmpeg tree (it shows the actual upstream
commits and authors instead of the subtree-merger) and I think git does
a better job of merging etc if it has more granular history to look at
(e.g. I think it can spot when the same changes appear in two commits
in different branches, perhaps due to cherry-picking, which helps merge
do the right thing more often).

Ian.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/12/2018 07:30 AM, Ian Campbell wrote:
> On Fri, 2018-05-11 at 13:51 -0400, Peter Bennett wrote:
>> In my own copy of MythTV repository, I tried using a subtree to include
>> FFmpeg. I hit up against a few problems -
>>
>> I had wanted to try importing a particular commit from FFmpeg, but I was
>> unable to do that. I could only import the latest version of a branch.
>> Also I could see no way of cherry-picking FFmpeg commits into our copy.
>>
>> git subtree add -P mythtv/external/FFmpeg
>> https://github.com/FFmpeg/FFmpeg.git 7e3a070 --squash
>> fatal: Couldn't find remote ref 7e3a070
>>
>> I tried adding it as a remote, with the same result
>>
>> git remote add ffmpeg https://github.com/FFmpeg/FFmpeg.git
>> git fetch ffmpeg
>> git subtree add -P mythtv/external/FFmpeg ffmpeg 7e3a070 --squash
>> fatal: Couldn't find remote ref 7e3a070
> There are two forms of `git subtree add`:
>> git subtree add -P <prefix> <commit>
>> git subtree add -P <prefix> <repository> <ref>
> You've tried the second one but I think you need the first, in a slight
> modification to your second attempt with a remote:
>
> ijc@dagon:mythtv-subtree-test$ git remote add ffmpeg https://github.com/FFmpeg/FFmpeg.git
> ijc@dagon:mythtv-subtree-test$ git fetch -n ffmpeg
> warning: no common commits
> [...]
> ijc@dagon:mythtv-subtree-test$ git subtree add -P mythtv/external/FFmpeg-test 7e3a070 --squash
> Added dir 'mythtv/external/FFmpeg-test'
> ijc@dagon:mythtv-subtree-test$ ls mythtv/external/FFmpeg-test
> Changelog COPYING.GPLv2 CREDITS INSTALL.md libavformat/ libswresample/ Makefile tests/
> compat/ COPYING.GPLv3 doc/ libavcodec/ libavresample/ libswscale/ presets/ tools/
> configure* COPYING.LGPLv2.1 ffbuild/ libavdevice/ libavutil/ LICENSE.md README.md
> CONTRIBUTING.md COPYING.LGPLv3 fftools/ libavfilter/ libpostproc/ MAINTAINERS RELEASE
>
> (I fudged the prefix because it already exists, but I guess you have
> already done "git rm -r" to get rid of the old copy in your tree, I
> used -n so I wouldn't get all the ffmpeg tags in my local repo). You
> could avoid the remote by just doing:
>
> git fetch https://github.com/FFmpeg/FFmpeg.git <branch|commit>
>
> instead of the `git remote add foo` + `git fetch -n foo`.
>
> I've not looked at how this differs from the subtree merge command:
>
>> git subtree merge -P <prefix> <commit>
> but I guess this is the one you want going forward after the initial
> add.
Thank you for the insight. This gives a better way of adding a subtree.
>
> For cherry-picking there is a process given in https://stackoverflow.co
> m/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
> (second half of first answer) which looks pretty plausible to me, but
> it does rely on not using `--squash` for the regular merges (although
> possibly it could be adapted).
It is a bit messy - 5 step process.
> TBH I'd recommend not using squash anyway, it'll give a more accurate
> picture of the history which is useful for people doing archaeology
> which crosses into the ffmpeg tree (it shows the actual upstream
> commits and authors instead of the subtree-merger) and I think git does
> a better job of merging etc if it has more granular history to look at
> (e.g. I think it can spot when the same changes appear in two commits
> in different branches, perhaps due to cherry-picking, which helps merge
> do the right thing more often).
The thing I was concerned about with that is that there are so many
commits in FFmpeg compared with MythTV, that they would overwhelm the
history. There are 91000 commits in the master branch of FFmpeg. I
feared that the day I pulled in the subtree without squash, it may
become very difficult to find anything useful pertaining to MythTV in
the log. Perhaps that would only affect certain views of the log.

I am currently trying an approach of having a private copy of the FFmpeg
repository. Then I can use the simpler option at the bottom of the
stackoverflow  article, that says "Another more simple option, if you
have access to the original subtree repository, is to make the cherry
pick there in a branch and then just git subtree pull that specific
branch.".

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/11/2018 05:13 PM, David Engel wrote:
> I have one idea, but it's not without its own share of warts. That is
> to treat the ffmpeg-master branch (and possibly others) like tracking
> branches. It could go something like this.
>
> # Remove the existing external/FFmpeg from the mythtv repo.
> git rm -r external/FFmpeg
>
> # Create an empty ffmpeg-master branch in the mythtv repo.
> git co ce7a5f62 (the epoch commit with an empty repo>
> git branch ffmpeg-master
> git co ffmpeg-master
>
> # Pull in ffmpeg master branch and move it to the right place.
> git pull --allow-unrelated-historieshttps://github.com/FFmpeg/FFmpeg.git master
> mkdir external/FFmpeg
> git mv <list of files> external/FFmpeg
>
> # Merge the ffmpeg-master branch into mythtv master.
> git co master
> git merge ffmpeg-master
>
> One wart is that anytime ffmpeg adds new files in their root, we would
> have to notice them when we pull and git mv them to external/FFmpeg.
>
> Another wart is we would have to do the same with any other ffmpeg
> branches that we might want to cherry-pick from.
I originally thought of this option before I found subtree. It is still
rather complicated. I don't know how it would handle files that are
moved from one ffmpeg directory to another (that was the case with a
bunch of files between 3.2 and 3.4).

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Sat, 2018-05-12 at 12:29 -0400, Peter Bennett wrote:
> > For cherry-picking there is a process given in https://stackoverflow.co
> > m/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
> > (second half of first answer) which looks pretty plausible to me, but
> > it does rely on not using `--squash` for the regular merges (although
> > possibly it could be adapted).
> It is a bit messy - 5 step process.

Yeah, it might be possible to script a bunch of it in a nice way.
Perhaps it is worth dropping a line to the git devs with a wishlist
request?

> > TBH I'd recommend not using squash anyway, it'll give a more accurate
> > picture of the history which is useful for people doing archaeology
> > which crosses into the ffmpeg tree (it shows the actual upstream
> > commits and authors instead of the subtree-merger) and I think git does
> > a better job of merging etc if it has more granular history to look at
> > (e.g. I think it can spot when the same changes appear in two commits
> > in different branches, perhaps due to cherry-picking, which helps merge
> > do the right thing more often).
> The thing I was concerned about with that is that there are so many
> commits in FFmpeg compared with MythTV, that they would overwhelm the
> history. There are 91000 commits in the master branch of FFmpeg. I
> feared that the day I pulled in the subtree without squash, it may
> become very difficult to find anything useful pertaining to MythTV in
> the log. Perhaps that would only affect certain views of the log.

`git log` orders things by commit date, so if the 91,000 ffmpeg commits
were largely historical then it wouldn't be so bad. But I suspect that
the ongoing flow of patches into ffmpeg probably outstrips mythtv by a
fair bit, in which case your concerns would be true by default.

You can use `git log --not ?branch|commit|etc?` to say "don't mention
things which are in here (i.e. `git log --not ffmpeg/master`), but
that's a bit tedious to remember to do.

> I am currently trying an approach of having a private copy of the FFmpeg
> repository. Then I can use the simpler option at the bottom of the
> stackoverflow article, that says "Another more simple option, if you
> have access to the original subtree repository, is to make the cherry
> pick there in a branch and then just git subtree pull that specific
> branch.".

That's effectively what the 5 step cherry-pick process above is doing,
it's just doing it all in the one repo using branches and bare
checkouts. It's certainly less things to keep straight in ones head to
do it in a separate repo though.

A separate repo presumably makes it easier to keep track of where
you've gotten up to (i.e. most recently merged) and lets you have
branches for fixes/30 etc and still keep cherry picking onto those
independently of master.

Ian.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/13/2018 03:32 AM, Ian Campbell wrote:
> On Sat, 2018-05-12 at 12:29 -0400, Peter Bennett wrote:
>>> For cherry-picking there is a process given in https://stackoverflow.co
>>> m/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
>>> (second half of first answer) which looks pretty plausible to me, but
>>> it does rely on not using `--squash` for the regular merges (although
>>> possibly it could be adapted).
>> It is a bit messy - 5 step process.
> Yeah, it might be possible to script a bunch of it in a nice way.
> Perhaps it is worth dropping a line to the git devs with a wishlist
> request?
>
>>> TBH I'd recommend not using squash anyway, it'll give a more accurate
>>> picture of the history which is useful for people doing archaeology
>>> which crosses into the ffmpeg tree (it shows the actual upstream
>>> commits and authors instead of the subtree-merger) and I think git does
>>> a better job of merging etc if it has more granular history to look at
>>> (e.g. I think it can spot when the same changes appear in two commits
>>> in different branches, perhaps due to cherry-picking, which helps merge
>>> do the right thing more often).
>> The thing I was concerned about with that is that there are so many
>> commits in FFmpeg compared with MythTV, that they would overwhelm the
>> history. There are 91000 commits in the master branch of FFmpeg. I
>> feared that the day I pulled in the subtree without squash, it may
>> become very difficult to find anything useful pertaining to MythTV in
>> the log. Perhaps that would only affect certain views of the log.
> `git log` orders things by commit date, so if the 91,000 ffmpeg commits
> were largely historical then it wouldn't be so bad. But I suspect that
> the ongoing flow of patches into ffmpeg probably outstrips mythtv by a
> fair bit, in which case your concerns would be true by default.
>
> You can use `git log --not «branch|commit|etc»` to say "don't mention
> things which are in here (i.e. `git log --not ffmpeg/master`), but
> that's a bit tedious to remember to do.
>
>> I am currently trying an approach of having a private copy of the FFmpeg
>> repository. Then I can use the simpler option at the bottom of the
>> stackoverflow article, that says "Another more simple option, if you
>> have access to the original subtree repository, is to make the cherry
>> pick there in a branch and then just git subtree pull that specific
>> branch.".
> That's effectively what the 5 step cherry-pick process above is doing,
> it's just doing it all in the one repo using branches and bare
> checkouts. It's certainly less things to keep straight in ones head to
> do it in a separate repo though.
>
> A separate repo presumably makes it easier to keep track of where
> you've gotten up to (i.e. most recently merged) and lets you have
> branches for fixes/30 etc and still keep cherry picking onto those
> independently of master.
>
> Ian.
>
In my private MythTV repository I added a subtree for FFmpeg (with
squash) from my private FFmpeg repository (with patches committed) and
got everything working. Then I pulled new ffmpeg commits into the
private ffmpeg repo and tried to pull the changes into the private
MythTV. That was a disaster. Every change, even 1 line changes, was
marked as a conflict. I think that is because with the squash it cannot
find a base for the merge. So that is a bust.

Now I have deleted the FFmpeg tree from my priivate MythTV and copied
all of the files from the private FFmpeg and committed that. This is
similar to our traditional approach, with the difference that I am
copying files from my own FFmpeg repo (with MythTV patches included)
instead of copying them from the main FFmpeg and re-applying patches.

I think maybe the best is to set up a FFmpeg repository under MythTV and
do it that way - all FFmpeg changes that we make should be first applied
to the FFmpeg repository and then copy the source over to MythTV.

I have the new FFmpeg working and very lightly tested with the master
MythTV. Playback works on VAAPI, VDPAU, XVIDEO. LiveTV works and LiveTV
subtitles work. That is all I have tested.

If you want to take a look it is in github under bennettpeter - MythTV
and FFmpeg repositories, master branches.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Mon, May 14, 2018 at 10:44:19AM -0400, Peter Bennett wrote:
>
>
> On 05/13/2018 03:32 AM, Ian Campbell wrote:
> > On Sat, 2018-05-12 at 12:29 -0400, Peter Bennett wrote:
> > > > For cherry-picking there is a process given in https://stackoverflow.co
> > > > m/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
> > > > (second half of first answer) which looks pretty plausible to me, but
> > > > it does rely on not using `--squash` for the regular merges (although
> > > > possibly it could be adapted).
> > > It is a bit messy - 5 step process.
> > Yeah, it might be possible to script a bunch of it in a nice way.
> > Perhaps it is worth dropping a line to the git devs with a wishlist
> > request?
> >
> > > > TBH I'd recommend not using squash anyway, it'll give a more accurate
> > > > picture of the history which is useful for people doing archaeology
> > > > which crosses into the ffmpeg tree (it shows the actual upstream
> > > > commits and authors instead of the subtree-merger) and I think git does
> > > > a better job of merging etc if it has more granular history to look at
> > > > (e.g. I think it can spot when the same changes appear in two commits
> > > > in different branches, perhaps due to cherry-picking, which helps merge
> > > > do the right thing more often).
> > > The thing I was concerned about with that is that there are so many
> > > commits in FFmpeg compared with MythTV, that they would overwhelm the
> > > history. There are 91000 commits in the master branch of FFmpeg. I
> > > feared that the day I pulled in the subtree without squash, it may
> > > become very difficult to find anything useful pertaining to MythTV in
> > > the log. Perhaps that would only affect certain views of the log.
> > `git log` orders things by commit date, so if the 91,000 ffmpeg commits
> > were largely historical then it wouldn't be so bad. But I suspect that
> > the ongoing flow of patches into ffmpeg probably outstrips mythtv by a
> > fair bit, in which case your concerns would be true by default.
> >
> > You can use `git log --not ?branch|commit|etc?` to say "don't mention
> > things which are in here (i.e. `git log --not ffmpeg/master`), but
> > that's a bit tedious to remember to do.
> >
> > > I am currently trying an approach of having a private copy of the FFmpeg
> > > repository. Then I can use the simpler option at the bottom of the
> > > stackoverflow article, that says "Another more simple option, if you
> > > have access to the original subtree repository, is to make the cherry
> > > pick there in a branch and then just git subtree pull that specific
> > > branch.".
> > That's effectively what the 5 step cherry-pick process above is doing,
> > it's just doing it all in the one repo using branches and bare
> > checkouts. It's certainly less things to keep straight in ones head to
> > do it in a separate repo though.
> >
> > A separate repo presumably makes it easier to keep track of where
> > you've gotten up to (i.e. most recently merged) and lets you have
> > branches for fixes/30 etc and still keep cherry picking onto those
> > independently of master.
> >
> > Ian.
> >
> In my private MythTV repository I added a subtree for FFmpeg (with squash)
> from my private FFmpeg repository (with patches committed) and got
> everything working. Then I pulled new ffmpeg commits into the private ffmpeg
> repo and tried to pull the changes into the private MythTV. That was a
> disaster. Every change, even 1 line changes, was marked as a conflict. I
> think that is because with the squash it cannot find a base for the merge.
> So that is a bust.

Did you try without squash? I know whatever we do will be tedious,
but what you describe below with our own, second repository sound more
so.

David

> Now I have deleted the FFmpeg tree from my priivate MythTV and copied all of
> the files from the private FFmpeg and committed that. This is similar to our
> traditional approach, with the difference that I am copying files from my
> own FFmpeg repo (with MythTV patches included) instead of copying them from
> the main FFmpeg and re-applying patches.
>
> I think maybe the best is to set up a FFmpeg repository under MythTV and do
> it that way - all FFmpeg changes that we make should be first applied to the
> FFmpeg repository and then copy the source over to MythTV.
>
> I have the new FFmpeg working and very lightly tested with the master
> MythTV. Playback works on VAAPI, VDPAU, XVIDEO. LiveTV works and LiveTV
> subtitles work. That is all I have tested.
>
> If you want to take a look it is in github under bennettpeter - MythTV and
> FFmpeg repositories, master branches.
>
> Peter

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/14/2018 02:11 PM, David Engel wrote:
> Did you try without squash? I know whatever we do will be tedious,
> but what you describe below with our own, second repository sound more
> so.
>
> David
>
I did not. I will create another branch in my repository and try that.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/14/2018 04:04 PM, Peter Bennett wrote:
>
>
> On 05/14/2018 02:11 PM, David Engel wrote:
>> Did you try without squash?  I know whatever we do will be tedious,
>> but what you describe below with our own, second repository sound more
>> so.
>>
>> David
>>
> I did not. I will create another branch in my repository and try that.
>
> Peter

Tested Without squash and without private repo:

It works fine I am able to merge updates from ffmpeg wiothout conflicts
as well as cherry pick a commit without problem, using the procedure in
https://stackoverflow.com/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree

The log is full of ffmpeg entries. We will need some filtering on "git
log" to make it useful. I have not looked into that. Look at
https://github.com/bennettpeter/mythtv and select the "ffmpeg-subtree"
branch. Notice it has 125000 commits, as opposed to the master branch,
which has 34000.

I think this is a good solution, as long as other developers are not too
unhappy about the log situation.

Below are the steps I used

git remote rename ffmpeg ffmpeg-private
git remote add ffmpeg git@github.com:FFmpeg/FFmpeg.git
git checkout 2403427
git branch ffmpeg-subtree
git checkout ffmpeg-subtree
git clean -xfd
git rm -r mythtv/external/FFmpeg
rm -r mythtv/external/FFmpeg/
git fetch ffmpeg
git subtree add -P mythtv/external/FFmpeg 6dc7963
git apply --directory=mythtv/external/FFmpeg
../patch/20180510_1945_FFmpeg_MythTV_Customization_fixed.patch
git apply --directory=mythtv/external/FFmpeg
../patch/20180511_1537_FFmpeg_compile_errors.patch
git apply ../patch/20180512_1728_additional_ffmpeg_fixes.patch
git apply ../patch/20180512_1158_fix_mythtv_compile_errors.patch --reject
git apply ../patch/20180512_1534_compile_error_and_fix_playback.patch
--reject
git push --set-upstream bennettpeter ffmpeg-subtree
#sync some ffmpeg updates
git fetch ffmpeg
git subtree merge -P mythtv/external/FFmpeg 7db022e
#successful
#cherry-pick a commit from ffmpeg
git checkout 7db022e
git cherry-pick -e -x 42a03e7
git log
#copy the commit hash -> 8a57b2fced942bc1c17ff26cfe8fa36193d95fbb
git checkout ffmpeg-subtree
# you can use HEAD@{1} instead of the hash if you did not do anything in
between
git subtree merge -P mythtv/external/FFmpeg
8a57b2fced942bc1c17ff26cfe8fa36193d95fbb

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Mon, May 14, 2018 at 07:44:11PM -0400, Peter Bennett wrote:
>
>
> On 05/14/2018 04:04 PM, Peter Bennett wrote:
> >
> >
> > On 05/14/2018 02:11 PM, David Engel wrote:
> > > Did you try without squash?? I know whatever we do will be tedious,
> > > but what you describe below with our own, second repository sound more
> > > so.
> > >
> > > David
> > >
> > I did not. I will create another branch in my repository and try that.
> >
> > Peter
>
> Tested Without squash and without private repo:
>
> It works fine I am able to merge updates from ffmpeg wiothout conflicts as
> well as cherry pick a commit without problem, using the procedure in https://stackoverflow.com/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
>
> The log is full of ffmpeg entries. We will need some filtering on "git log"
> to make it useful. I have not looked into that. Look at
> https://github.com/bennettpeter/mythtv and select the "ffmpeg-subtree"
> branch. Notice it has 125000 commits, as opposed to the master branch, which
> has 34000.
>
> I think this is a good solution, as long as other developers are not too
> unhappy about the log situation.

The log bloat is unfortunate, but I think this solution is better than
the other alternatives so far. Thanks for testing all of this.

David

> Below are the steps I used
>
> git remote rename ffmpeg ffmpeg-private
> git remote add ffmpeg git@github.com:FFmpeg/FFmpeg.git
> git checkout 2403427
> git branch ffmpeg-subtree
> git checkout ffmpeg-subtree
> git clean -xfd
> git rm -r mythtv/external/FFmpeg
> rm -r mythtv/external/FFmpeg/
> git fetch ffmpeg
> git subtree add -P mythtv/external/FFmpeg 6dc7963
> git apply --directory=mythtv/external/FFmpeg
> ../patch/20180510_1945_FFmpeg_MythTV_Customization_fixed.patch
> git apply --directory=mythtv/external/FFmpeg
> ../patch/20180511_1537_FFmpeg_compile_errors.patch
> git apply ../patch/20180512_1728_additional_ffmpeg_fixes.patch
> git apply ../patch/20180512_1158_fix_mythtv_compile_errors.patch --reject
> git apply ../patch/20180512_1534_compile_error_and_fix_playback.patch
> --reject
> git push --set-upstream bennettpeter ffmpeg-subtree
> #sync some ffmpeg updates
> git fetch ffmpeg
> git subtree merge -P mythtv/external/FFmpeg 7db022e
> #successful
> #cherry-pick a commit from ffmpeg
> git checkout 7db022e
> git cherry-pick -e -x 42a03e7
> git log
> #copy the commit hash -> 8a57b2fced942bc1c17ff26cfe8fa36193d95fbb
> git checkout ffmpeg-subtree
> # you can use HEAD@{1} instead of the hash if you did not do anything in
> between
> git subtree merge -P mythtv/external/FFmpeg
> 8a57b2fced942bc1c17ff26cfe8fa36193d95fbb
>
> Peter

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/15/2018 11:51 AM, David Engel wrote:
> The log bloat is unfortunate, but I think this solution is better than
> the other alternatives so far. Thanks for testing all of this.
>
> David
I did some testing of git log and I found that git seems to have a
bug/restriction which actually helps.

If you cd to the base directory of the repo and run "git log -- mythtv"
you only see the mythtv and mythtv subdirectory changes that were made
in this repository but not any of the ffmpeg subtree changes that were
imported via the subtree.  So we see the ffmpeg customizations that we
applied here but none of those imported via the subtree. This is exactly
what I would want to see.

I cannot find a way of seeing a log of subtree imported commits using a
path. Using mythtv/external/FFmpeg finds nothing, probably because when
they were committed the path was just libavformat/ and not
mythtv/external/FFmpeg/libavformat. However searching on path
libavformat also shows nothing. If you want to see those you either have
to use the full "git log", search on a revision range, or look in the
ffmpeg repository. This seems to be a bug but works in our favor.

Peter


_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On Tue, 2018-05-15 at 12:14 -0400, Peter Bennett wrote:
> If you cd to the base directory of the repo and run "git log -- mythtv"
> you only see the mythtv and mythtv subdirectory changes that were made
> in this repository but not any of the ffmpeg subtree changes that were
> imported via the subtree. So we see the ffmpeg customizations that we
> applied here but none of those imported via the subtree. This is exactly
> what I would want to see.

This is because git log isn't aware of subtrees, so as it walks over
commits which are actually from ffmpeg it isn't seeing
mythtv/external/ffmpeg/foo/bar.c (which would match your mythtv
argument) it sees just foo/bar.c (which doesn't match the argument).

So it relies a bit on ffmpeg not having a top-level directory called
"mythtv" but that seems like a safe bet.

It's not future proof against the git devs make "git log" aware of
subtrees -- but you would hope/imagine that in doing so they would add
useful options like --no-subtrees at the same time.

>
> I cannot find a way of seeing a log of subtree imported commits using a
> path. Using mythtv/external/FFmpeg finds nothing, probably because when
> they were committed the path was just libavformat/ and not
> mythtv/external/FFmpeg/libavformat.

Correct (I should have read ahead before writing the above! I'll leave
it now I've typed it)

Ian.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 15/05/18 16:51, David Engel wrote:

> On Mon, May 14, 2018 at 07:44:11PM -0400, Peter Bennett wrote:
>>
>> On 05/14/2018 04:04 PM, Peter Bennett wrote:
>>>
>>> On 05/14/2018 02:11 PM, David Engel wrote:
>>>> Did you try without squash?  I know whatever we do will be tedious,
>>>> but what you describe below with our own, second repository sound more
>>>> so.
>>>>
>>>> David
>>>>
>>> I did not. I will create another branch in my repository and try that.
>>>
>>> Peter
>> Tested Without squash and without private repo:
>>
>> It works fine I am able to merge updates from ffmpeg wiothout conflicts as
>> well as cherry pick a commit without problem, using the procedure in https://stackoverflow.com/questions/12978260/how-do-i-go-to-a-specific-commit-using-git-subtree
>>
>> The log is full of ffmpeg entries. We will need some filtering on "git log"
>> to make it useful. I have not looked into that. Look at
>> https://github.com/bennettpeter/mythtv and select the "ffmpeg-subtree"
>> branch. Notice it has 125000 commits, as opposed to the master branch, which
>> has 34000.
>>
>> I think this is a good solution, as long as other developers are not too
>> unhappy about the log situation.
> The log bloat is unfortunate, but I think this solution is better than
> the other alternatives so far. Thanks for testing all of this.
>
> David
>

Do we really want to see all the ffmpeg commits in our git logs? I hate
the idea :(

Paul H.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: FFmpeg plans [ In reply to ]
On 05/15/2018 01:05 PM, Paul Harrison wrote:
> Do we really want to see all the ffmpeg commits in our git logs? I
> hate the idea :(
>
> Paul H.
Paul
Did you see my later email on this? By using "git log -- mythtv" you can
exclude all the ffmpeg commits.
Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org