Mailing List Archive

Decision on FFmpeg repository
I would like to get a decision on what to do about a couple of FFmpeg
questions:

1. Use FFmpeg master or release branch? My recommendation is to use master.
- the FFmpeg website recommends using master for those who are compiling
FFmpeg themselves. It states that the release branches are for those who
rely on downloaded prebuilt packages
(https://ffmpeg.org/download.html#repositories).
- Only master has the recent mediacodec changes which I need for android.
- If we sync to master, we can use a better approach for the repository
(question 2 below).

2. What method should we use for storing and syncing with FFmpeg source?
Options are:
a. Reapply patches to the source from FFmpeg, resolve conflicts, and
copy the source over what we have in MythTV (current process). This can
be very time-consuming.

b. Use a subtree in the MythTV repository.
- This will not work if we use a release branch (question 1).
- When resyncing, we can do a git merge of the subtree from FFmpeg.
- We can cherry-pick changes needed without resyncing everything.
- We can sync to any required commit, we don't need to go all the way to
the latest commit.
- git log shows FFmpeg commits intermingled with MythTV. You can use
"git log -- mythtv" which eliminates those.
- This imports some 90,000 extra commits from FFmpeg.

You can see an example of the subtree in use at
https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that
repo and look at the ffmpeg-subtree branch.

c. Keep our own customized FFmpeg repository with patches applied.
- This will not work if we use a release branch (question 1).
- When resyncing, do a git pull or merge on the private FFmpeg
repository. Then copy the entire source over the MythTV FFmpeg source.
- We can cherry-pick changes needed without resyncing everything.
- We can sync to any required commit, we don't need to go all the way to
the latest commit.
- git log in MythTV does not show all the FFmpeg commits.
- git log in the private FFmpeg repository shows records of our patches
along with all the FFmpeg commits.
- We can perform periodic pulls into the private FFmpeg repository
without affecting MythTV. This will make the eventual copying into
MythTV quicker and easier.
- Any FFmpeg patches that we apply between copying into MythTV must be
applied in both MythTV and FFmpeg repositories, or apply them in the
private FFmpeg repository and then wait for the next copy into MythTV.

You can see an example of FFmpeg with patches applied at
https://github.com/bennettpeter/FFmpeg

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Wed, May 16, 2018 at 10:54 AM Peter Bennett <pb.mythtv@gmail.com> wrote:

> I would like to get a decision on what to do about a couple of FFmpeg
> questions:
>
> 1. Use FFmpeg master or release branch? My recommendation is to use master.
> - the FFmpeg website recommends using master for those who are compiling
> FFmpeg themselves. It states that the release branches are for those who
> rely on downloaded prebuilt packages
> (https://ffmpeg.org/download.html#repositories).
> - Only master has the recent mediacodec changes which I need for android.
> - If we sync to master, we can use a better approach for the repository
> (question 2 below).
>
> 2. What method should we use for storing and syncing with FFmpeg source?
> Options are:
> a. Reapply patches to the source from FFmpeg, resolve conflicts, and
> copy the source over what we have in MythTV (current process). This can
> be very time-consuming.
>
> b. Use a subtree in the MythTV repository.
> - This will not work if we use a release branch (question 1).
> - When resyncing, we can do a git merge of the subtree from FFmpeg.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to
> the latest commit.
> - git log shows FFmpeg commits intermingled with MythTV. You can use
> "git log -- mythtv" which eliminates those.
> - This imports some 90,000 extra commits from FFmpeg.
>
> You can see an example of the subtree in use at
> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that
> repo and look at the ffmpeg-subtree branch.
>
> c. Keep our own customized FFmpeg repository with patches applied.
> - This will not work if we use a release branch (question 1).
> - When resyncing, do a git pull or merge on the private FFmpeg
> repository. Then copy the entire source over the MythTV FFmpeg source.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to
> the latest commit.
> - git log in MythTV does not show all the FFmpeg commits.
> - git log in the private FFmpeg repository shows records of our patches
> along with all the FFmpeg commits.
> - We can perform periodic pulls into the private FFmpeg repository
> without affecting MythTV. This will make the eventual copying into
> MythTV quicker and easier.
> - Any FFmpeg patches that we apply between copying into MythTV must be
> applied in both MythTV and FFmpeg repositories, or apply them in the
> private FFmpeg repository and then wait for the next copy into MythTV.
>
> You can see an example of FFmpeg with patches applied at
> https://github.com/bennettpeter/FFmpeg
>

Since you are doing the work, Peter, I think you get a stronger vote that
the rest of us.

My personal vote would be for 2(c), but 2(b) would be okay. I don't like
2(a).

John
Re: Decision on FFmpeg repository [ In reply to ]
On Wed, 2018-05-16 at 12:45 -0400, Peter Bennett wrote:
> I would like to get a decision on what to do about a couple of
> FFmpeg
> questions:

I don't really have any skin in this game so take my opinions with a
grain of salt, however...

> - the FFmpeg website recommends using master for those who are
> compiling
> FFmpeg themselves. It states that the release branches are for those
> who
> rely on downloaded prebuilt packages
> (https://ffmpeg.org/download.html#repositories).

This doesn't really square for me. I don't see why those compiling
themselves are any different than those who get packages from packagers
(read on). Ultimately everything else being equal everyone wants
stability. But things are not usually equal so some trade off
stability for bleeding edge because they feel the value is there, for
them. But others value the stability more and so give up on bleeding
edge features.

The master branch (assuming typical branching processes) of (most) any
project is always going to be potentially less stable than release
branches that don't constantly have things being committed to them and
have been created at a time of recognized stability. And as time goes
on gain a record of stability.

So is the above document suggesting that master is good (enough) for
self-compiling people because they could (and would be willing to) be
agile enough to jump forward if instability is determined? If that is
the reasoning, does that really fit here? Isn't the MythTV project
(particularly on a "fixes" branch) more like the (not-so-agile)
packagers packaging for consumers?

Cheers,
b.
Re: Decision on FFmpeg repository [ In reply to ]
On 05/16/2018 01:20 PM, Brian J. Murrell wrote:
> On Wed, 2018-05-16 at 12:45 -0400, Peter Bennett wrote:
>> I would like to get a decision on what to do about a couple of
>> FFmpeg
>> questions:
> I don't really have any skin in this game so take my opinions with a
> grain of salt, however...
>
>> - the FFmpeg website recommends using master for those who are
>> compiling
>> FFmpeg themselves. It states that the release branches are for those
>> who
>> rely on downloaded prebuilt packages
>> (https://ffmpeg.org/download.html#repositories).
> This doesn't really square for me. I don't see why those compiling
> themselves are any different than those who get packages from packagers
> (read on). Ultimately everything else being equal everyone wants
> stability. But things are not usually equal so some trade off
> stability for bleeding edge because they feel the value is there, for
> them. But others value the stability more and so give up on bleeding
> edge features.
>
> The master branch (assuming typical branching processes) of (most) any
> project is always going to be potentially less stable than release
> branches that don't constantly have things being committed to them and
> have been created at a time of recognized stability. And as time goes
> on gain a record of stability.
>
> So is the above document suggesting that master is good (enough) for
> self-compiling people because they could (and would be willing to) be
> agile enough to jump forward if instability is determined? If that is
> the reasoning, does that really fit here? Isn't the MythTV project
> (particularly on a "fixes" branch) more like the (not-so-agile)
> packagers packaging for consumers?
>
> Cheers,
> b.
>
Hi Brian

You have valid points here. However my plan is that we will only bring
in FFmpeg updates to the MythTV master branch. Only once that is stable
would be create a new MythTV fixes branch. In our master branch, people
should be willing to put up with some instability as we iron out
problems. After bringing in a new version of FFmpeg we can, if
necessary, cherry-pick FFmpeg commits that fix bugs that fix bugs which
may affect us. We will never bring in a new FFmpeg version on a "fixes"
branch.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Wed, May 16, 2018 at 12:45:55PM -0400, Peter Bennett wrote:
> I would like to get a decision on what to do about a couple of FFmpeg
> questions:
>
> 1. Use FFmpeg master or release branch? My recommendation is to use master.
> - the FFmpeg website recommends using master for those who are compiling
> FFmpeg themselves. It states that the release branches are for those who
> rely on downloaded prebuilt packages
> (https://ffmpeg.org/download.html#repositories).
> - Only master has the recent mediacodec changes which I need for android.
> - If we sync to master, we can use a better approach for the repository
> (question 2 below).

As I've already mentioned, I'm not thrilled with using the master
branch. However, since the MediaCodec support still appears to need
it and that's the most important reason for upgrading ffmpeg right
now, I'll just have to accept it.

> 2. What method should we use for storing and syncing with FFmpeg source?
> Options are:
> a. Reapply patches to the source from FFmpeg, resolve conflicts, and copy
> the source over what we have in MythTV (current process). This can be very
> time-consuming.
>
> b. Use a subtree in the MythTV repository.
> - This will not work if we use a release branch (question 1).
> - When resyncing, we can do a git merge of the subtree from FFmpeg.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to the
> latest commit.
> - git log shows FFmpeg commits intermingled with MythTV. You can use "git
> log -- mythtv" which eliminates those.
> - This imports some 90,000 extra commits from FFmpeg.
>
> You can see an example of the subtree in use at
> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that
> repo and look at the ffmpeg-subtree branch.
>
> c. Keep our own customized FFmpeg repository with patches applied.
> - This will not work if we use a release branch (question 1).
> - When resyncing, do a git pull or merge on the private FFmpeg repository.
> Then copy the entire source over the MythTV FFmpeg source.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to the
> latest commit.
> - git log in MythTV does not show all the FFmpeg commits.
> - git log in the private FFmpeg repository shows records of our patches
> along with all the FFmpeg commits.
> - We can perform periodic pulls into the private FFmpeg repository without
> affecting MythTV. This will make the eventual copying into MythTV quicker
> and easier.
> - Any FFmpeg patches that we apply between copying into MythTV must be
> applied in both MythTV and FFmpeg repositories, or apply them in the private
> FFmpeg repository and then wait for the next copy into MythTV.

Of these, I prefer 2b. I'd reluctantly accept 2c if there are too
many objections to 2b. I think 2a would quickly become untenable if
we have to track the ffmpeg master branch frequently.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
> On 16 May 2018, at 8:48 pm, David Engel <david@istwok.net> wrote:
>
> On Wed, May 16, 2018 at 12:45:55PM -0400, Peter Bennett wrote:
>> I would like to get a decision on what to do about a couple of FFmpeg
>> questions:
>>
>> 1. Use FFmpeg master or release branch? My recommendation is to use master.
>> - the FFmpeg website recommends using master for those who are compiling
>> FFmpeg themselves. It states that the release branches are for those who
>> rely on downloaded prebuilt packages
>> (https://ffmpeg.org/download.html#repositories <https://ffmpeg.org/download.html#repositories>).
>> - Only master has the recent mediacodec changes which I need for android.
>> - If we sync to master, we can use a better approach for the repository
>> (question 2 below).
>
> As I've already mentioned, I'm not thrilled with using the master
> branch. However, since the MediaCodec support still appears to need
> it and that's the most important reason for upgrading ffmpeg right
> now, I'll just have to accept it.

IMHO, going with the master branch is a *big* mistake…
API changes, methods are constantly obsoleted, bugs are introduced to be fixed after etc…
Re: Decision on FFmpeg repository [ In reply to ]
On 5/17/2018 2:45 AM, Peter Bennett wrote:
> I would like to get a decision on what to do about a couple of FFmpeg
> questions:
>
> 1. Use FFmpeg master or release branch? My recommendation is to use
> master.
> - the FFmpeg website recommends using master for those who are
> compiling FFmpeg themselves. It states that the release branches are
> for those who rely on downloaded prebuilt packages
> (https://ffmpeg.org/download.html#repositories).
> - Only master has the recent mediacodec changes which I need for android.
> - If we sync to master, we can use a better approach for the
> repository (question 2 below).
>
> 2. What method should we use for storing and syncing with FFmpeg source?
> Options are:
> a. Reapply patches to the source from FFmpeg, resolve conflicts, and
> copy the source over what we have in MythTV (current process). This
> can be very time-consuming.
>
> b. Use a subtree in the MythTV repository.
> - This will not work if we use a release branch (question 1).
> - When resyncing, we can do a git merge of the subtree from FFmpeg.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way
> to the latest commit.
> - git log shows FFmpeg commits intermingled with MythTV. You can use
> "git log -- mythtv" which eliminates those.
> - This imports some 90,000 extra commits from FFmpeg.
>
> You can see an example of the subtree in use at
> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone
> that repo and look at the ffmpeg-subtree branch.
>
> c. Keep our own customized FFmpeg repository with patches applied.
> - This will not work if we use a release branch (question 1).
> - When resyncing, do a git pull or merge on the private FFmpeg
> repository. Then copy the entire source over the MythTV FFmpeg source.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way
> to the latest commit.
> - git log in MythTV does not show all the FFmpeg commits.
> - git log in the private FFmpeg repository shows records of our
> patches along with all the FFmpeg commits.
> - We can perform periodic pulls into the private FFmpeg repository
> without affecting MythTV. This will make the eventual copying into
> MythTV quicker and easier.
> - Any FFmpeg patches that we apply between copying into MythTV must be
> applied in both MythTV and FFmpeg repositories, or apply them in the
> private FFmpeg repository and then wait for the next copy into MythTV.
>
> You can see an example of FFmpeg with patches applied at
> https://github.com/bennettpeter/FFmpeg

I too am not comfortable using master but your justification is sound.
Is there a prerelease with mediacodec added? probably not.
You could start with master until mediacodec is released.

Im not sure if you tried this but you could use 2b in a ffmpeg-tracking
branch and then merge with squash to master, where the patches are.
Im not sure how well this would work compared with tracking ffmpeg in
master though wrt applying patches on top of ffmpeg compared with master
only, but it would remove the log issue.

Perhaps process of merge master to tracking, update subtree, fix
conflicts in tracking, merge with squash back to master, fixup any
issues in master.

HTH
Mark
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
Den 2018-05-16 kl. 18:45, skrev Peter Bennett:
> I would like to get a decision on what to do about a couple of FFmpeg questions:
>
> 1. Use FFmpeg master or release branch? My recommendation is to use master.
> - the FFmpeg website recommends using master for those who are compiling FFmpeg themselves. It states that the release branches are for those who rely on downloaded prebuilt packages (https://ffmpeg.org/download.html#repositories).
> - Only master has the recent mediacodec changes which I need for android.
> - If we sync to master, we can use a better approach for the repository (question 2 below).

Based on Jean-Yves' opinion I would not go for the master branch.

> 2. What method should we use for storing and syncing with FFmpeg source?
> Options are:
> a. Reapply patches to the source from FFmpeg, resolve conflicts, and copy the source over what we have in MythTV (current process). This can be very time-consuming.
>
> b. Use a subtree in the MythTV repository.
> - This will not work if we use a release branch (question 1).
> - When resyncing, we can do a git merge of the subtree from FFmpeg.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to the latest commit.
> - git log shows FFmpeg commits intermingled with MythTV. You can use "git log -- mythtv" which eliminates those.
> - This imports some 90,000 extra commits from FFmpeg.
>
> You can see an example of the subtree in use at https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that repo and look at the ffmpeg-subtree branch.
>
> c. Keep our own customized FFmpeg repository with patches applied.
> - This will not work if we use a release branch (question 1).
> - When resyncing, do a git pull or merge on the private FFmpeg repository. Then copy the entire source over the MythTV FFmpeg source.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to the latest commit.
> - git log in MythTV does not show all the FFmpeg commits.
> - git log in the private FFmpeg repository shows records of our patches along with all the FFmpeg commits.
> - We can perform periodic pulls into the private FFmpeg repository without affecting MythTV. This will make the eventual copying into MythTV quicker and easier.
> - Any FFmpeg patches that we apply between copying into MythTV must be applied in both MythTV and FFmpeg repositories, or apply them in the private FFmpeg repository and then wait for the next copy into MythTV.
>
> You can see an example of FFmpeg with patches applied at https://github.com/bennettpeter/FFmpeg

For 2c, would it not be easier to use a submodule instead of manually copying files from one repo to the other? I think I prefer that over 2b.

Jonatan
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Wed, May 16, 2018 at 09:43:01PM +0200, Jean-Yves Avenard wrote:
>
>
> > On 16 May 2018, at 8:48 pm, David Engel <david@istwok.net> wrote:
> >
> > On Wed, May 16, 2018 at 12:45:55PM -0400, Peter Bennett wrote:
> >> I would like to get a decision on what to do about a couple of FFmpeg
> >> questions:
> >>
> >> 1. Use FFmpeg master or release branch? My recommendation is to use master.
> >> - the FFmpeg website recommends using master for those who are compiling
> >> FFmpeg themselves. It states that the release branches are for those who
> >> rely on downloaded prebuilt packages
> >> (https://ffmpeg.org/download.html#repositories <https://ffmpeg.org/download.html#repositories>).
> >> - Only master has the recent mediacodec changes which I need for android.
> >> - If we sync to master, we can use a better approach for the repository
> >> (question 2 below).
> >
> > As I've already mentioned, I'm not thrilled with using the master
> > branch. However, since the MediaCodec support still appears to need
> > it and that's the most important reason for upgrading ffmpeg right
> > now, I'll just have to accept it.
>
> IMHO, going with the master branch is a *big* mistake…
> API changes, methods are constantly obsoleted, bugs are introduced to be fixed after etc…

Some background.

The main reason we are even considering an ffmpeg update so soon after
our last one is to support MediaCodec on Android. We must go to at
least ffmpeg version 4.0 to get that. Our likely only other
alternative for that is to write our own MediaCodec support and I
doubt anyone wants to do that.

Peter and I have had discussions regarding our MediaCodec support with
one of the ffmpeg developers. I think he is actually the main
MediaCodec person at ffmpeg. He has already pointed us to some
significant, MediaCodec changes that have been committed to master
with strong hints there are more to come.

So, we can either use the stable, ffmpeg 4.0 branch with reportedly
still deficient MediaCodec support or deal with potential instability
and evolving APIs in the ffmpeg master branch.

One other complicating factor is the ffmpeg developer has expressed
interest in helping us get some of our changes accepted upstream. I
don't know how well that would go over if we don't use master and
consequently aren't able to help test their integration.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Thu, May 17, 2018 at 08:16:26AM +1000, Mark Spieth wrote:
> On 5/17/2018 2:45 AM, Peter Bennett wrote:
> > I would like to get a decision on what to do about a couple of FFmpeg
> > questions:
> >
> > 1. Use FFmpeg master or release branch? My recommendation is to use
> > master.
> > - the FFmpeg website recommends using master for those who are compiling
> > FFmpeg themselves. It states that the release branches are for those who
> > rely on downloaded prebuilt packages
> > (https://ffmpeg.org/download.html#repositories).
> > - Only master has the recent mediacodec changes which I need for android.
> > - If we sync to master, we can use a better approach for the repository
> > (question 2 below).
> >
> > 2. What method should we use for storing and syncing with FFmpeg source?
> > Options are:
> > a. Reapply patches to the source from FFmpeg, resolve conflicts, and
> > copy the source over what we have in MythTV (current process). This can
> > be very time-consuming.
> >
> > b. Use a subtree in the MythTV repository.
> > - This will not work if we use a release branch (question 1).
> > - When resyncing, we can do a git merge of the subtree from FFmpeg.
> > - We can cherry-pick changes needed without resyncing everything.
> > - We can sync to any required commit, we don't need to go all the way to
> > the latest commit.
> > - git log shows FFmpeg commits intermingled with MythTV. You can use
> > "git log -- mythtv" which eliminates those.
> > - This imports some 90,000 extra commits from FFmpeg.
> >
> > You can see an example of the subtree in use at
> > https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that
> > repo and look at the ffmpeg-subtree branch.
> >
> > c. Keep our own customized FFmpeg repository with patches applied.
> > - This will not work if we use a release branch (question 1).
> > - When resyncing, do a git pull or merge on the private FFmpeg
> > repository. Then copy the entire source over the MythTV FFmpeg source.
> > - We can cherry-pick changes needed without resyncing everything.
> > - We can sync to any required commit, we don't need to go all the way to
> > the latest commit.
> > - git log in MythTV does not show all the FFmpeg commits.
> > - git log in the private FFmpeg repository shows records of our patches
> > along with all the FFmpeg commits.
> > - We can perform periodic pulls into the private FFmpeg repository
> > without affecting MythTV. This will make the eventual copying into
> > MythTV quicker and easier.
> > - Any FFmpeg patches that we apply between copying into MythTV must be
> > applied in both MythTV and FFmpeg repositories, or apply them in the
> > private FFmpeg repository and then wait for the next copy into MythTV.
> >
> > You can see an example of FFmpeg with patches applied at
> > https://github.com/bennettpeter/FFmpeg
>
> I too am not comfortable using master but your justification is sound.
> Is there a prerelease with mediacodec added? probably not.
> You could start with master until mediacodec is released.
>
> Im not sure if you tried this but you could use 2b in a ffmpeg-tracking
> branch and then merge with squash to master, where the patches are.
> Im not sure how well this would work compared with tracking ffmpeg in master
> though wrt applying patches on top of ffmpeg compared with master only, but
> it would remove the log issue.
>
> Perhaps process of merge master to tracking, update subtree, fix conflicts
> in tracking, merge with squash back to master, fixup any issues in master.

I think that would have the same problem Peter ran into when trying 2b
without squash. That is without the intermediate commits, git marked
everything as conflicting on subsequent updates because it didn't know
what to base them on.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On 17/05/18 08:52, David Engel wrote:
> On Thu, May 17, 2018 at 08:16:26AM +1000, Mark Spieth wrote:
>> On 5/17/2018 2:45 AM, Peter Bennett wrote:
>>> I would like to get a decision on what to do about a couple of FFmpeg
>>> questions:
>>>
>>> 1. Use FFmpeg master or release branch? My recommendation is to use
>>> master.
>>> - the FFmpeg website recommends using master for those who are compiling
>>> FFmpeg themselves. It states that the release branches are for those who
>>> rely on downloaded prebuilt packages
>>> (https://ffmpeg.org/download.html#repositories).
>>> - Only master has the recent mediacodec changes which I need for android.
>>> - If we sync to master, we can use a better approach for the repository
>>> (question 2 below).
>>>
>>> 2. What method should we use for storing and syncing with FFmpeg source?
>>> Options are:
>>> a. Reapply patches to the source from FFmpeg, resolve conflicts, and
>>> copy the source over what we have in MythTV (current process). This can
>>> be very time-consuming.
>>>
>>> b. Use a subtree in the MythTV repository.
>>> - This will not work if we use a release branch (question 1).
>>> - When resyncing, we can do a git merge of the subtree from FFmpeg.
>>> - We can cherry-pick changes needed without resyncing everything.
>>> - We can sync to any required commit, we don't need to go all the way to
>>> the latest commit.
>>> - git log shows FFmpeg commits intermingled with MythTV. You can use
>>> "git log -- mythtv" which eliminates those.
>>> - This imports some 90,000 extra commits from FFmpeg.
>>>
>>> You can see an example of the subtree in use at
>>> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that
>>> repo and look at the ffmpeg-subtree branch.
>>>
>>> c. Keep our own customized FFmpeg repository with patches applied.
>>> - This will not work if we use a release branch (question 1).
>>> - When resyncing, do a git pull or merge on the private FFmpeg
>>> repository. Then copy the entire source over the MythTV FFmpeg source.
>>> - We can cherry-pick changes needed without resyncing everything.
>>> - We can sync to any required commit, we don't need to go all the way to
>>> the latest commit.
>>> - git log in MythTV does not show all the FFmpeg commits.
>>> - git log in the private FFmpeg repository shows records of our patches
>>> along with all the FFmpeg commits.
>>> - We can perform periodic pulls into the private FFmpeg repository
>>> without affecting MythTV. This will make the eventual copying into
>>> MythTV quicker and easier.
>>> - Any FFmpeg patches that we apply between copying into MythTV must be
>>> applied in both MythTV and FFmpeg repositories, or apply them in the
>>> private FFmpeg repository and then wait for the next copy into MythTV.
>>>
>>> You can see an example of FFmpeg with patches applied at
>>> https://github.com/bennettpeter/FFmpeg
>> I too am not comfortable using master but your justification is sound.
>> Is there a prerelease with mediacodec added? probably not.
>> You could start with master until mediacodec is released.
>>
>> Im not sure if you tried this but you could use 2b in a ffmpeg-tracking
>> branch and then merge with squash to master, where the patches are.
>> Im not sure how well this would work compared with tracking ffmpeg in master
>> though wrt applying patches on top of ffmpeg compared with master only, but
>> it would remove the log issue.
>>
>> Perhaps process of merge master to tracking, update subtree, fix conflicts
>> in tracking, merge with squash back to master, fixup any issues in master.
> I think that would have the same problem Peter ran into when trying 2b
> without squash. That is without the intermediate commits, git marked
> everything as conflicting on subsequent updates because it didn't know
> what to base them on.
>
but you do get the full history in the ffmpeg-tracking branch, just not
master.
BOBW

Mark
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
Hi

> On 17 May 2018, at 12:49 am, David Engel <david@istwok.net> wrote:
>
> Some background.
>
> The main reason we are even considering an ffmpeg update so soon after
> our last one is to support MediaCodec on Android. We must go to at
> least ffmpeg version 4.0 to get that. Our likely only other
> alternative for that is to write our own MediaCodec support and I
> doubt anyone wants to do that.
>
> Peter and I have had discussions regarding our MediaCodec support with
> one of the ffmpeg developers. I think he is actually the main
> MediaCodec person at ffmpeg. He has already pointed us to some
> significant, MediaCodec changes that have been committed to master
> with strong hints there are more to come.
>
> So, we can either use the stable, ffmpeg 4.0 branch with reportedly
> still deficient MediaCodec support or deal with potential instability
> and evolving APIs in the ffmpeg master branch.
>
> One other complicating factor is the ffmpeg developer has expressed
> interest in helping us get some of our changes accepted upstream. I
> don't know how well that would go over if we don't use master and
> consequently aren't able to help test their integration.

That’s all good, but then that means that support for MediaCoded in FFmpeg is experimental anyway.. Do we want that in our release so soon?

Are we willing to make everything unstable (from an API point of view) just to have support for something experimental that we didn’t support up to now anyway?

If you ever ask a developer what version of his you should use, of course he/she’s always going to tell you to use the latest and greatest, luckily most team have a release manager :)

4.0 was a snapshot of ffmpeg done not even a month ago. By the time we’re ready with Peter’s change anyway, 4.1 would be out. FFmpeg make release frequently, when they are confident current master is in a stable state.

WIP on a particular feature rarely ever qualify.

Anyhow, not that I’m much involved these days to have much weight on any decisions...
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Thu, May 17, 2018 at 10:52:26AM +0200, Jean-Yves Avenard wrote:
> Hi
>
> > On 17 May 2018, at 12:49 am, David Engel <david@istwok.net> wrote:
> >
> > Some background.
> >
> > The main reason we are even considering an ffmpeg update so soon after
> > our last one is to support MediaCodec on Android. We must go to at
> > least ffmpeg version 4.0 to get that. Our likely only other
> > alternative for that is to write our own MediaCodec support and I
> > doubt anyone wants to do that.
> >
> > Peter and I have had discussions regarding our MediaCodec support with
> > one of the ffmpeg developers. I think he is actually the main
> > MediaCodec person at ffmpeg. He has already pointed us to some
> > significant, MediaCodec changes that have been committed to master
> > with strong hints there are more to come.
> >
> > So, we can either use the stable, ffmpeg 4.0 branch with reportedly
> > still deficient MediaCodec support or deal with potential instability
> > and evolving APIs in the ffmpeg master branch.
> >
> > One other complicating factor is the ffmpeg developer has expressed
> > interest in helping us get some of our changes accepted upstream. I
> > don't know how well that would go over if we don't use master and
> > consequently aren't able to help test their integration.
>
> That’s all good, but then that means that support for MediaCoded in FFmpeg is experimental anyway.. Do we want that in our release so soon?
>
> Are we willing to make everything unstable (from an API point of view) just to have support for something experimental that we didn’t support up to now anyway?

Of course we don't want to intentionally break other things, but in
case you haven't noticed, the Android port is the *only* significant
work being done right now. IOW, there is no other work to be
destabilized for the time being. If any other work does comes up and
is affected, we can move one effort or the other to a branch.

> If you ever ask a developer what version of his you should use, of course he/she’s always going to tell you to use the latest and greatest, luckily most team have a release manager :)
>
> 4.0 was a snapshot of ffmpeg done not even a month ago. By the time we’re ready with Peter’s change anyway, 4.1 would be out. FFmpeg make release frequently, when they are confident current master is in a stable state.

We're, well, mainly Peter, are ready to start now. If we wait for
ffmpeg 4.1 and it changes as much as you expect, wouldn't we have to
start over again? In a sense, we're trying to get ahead of the curver
and targeting for 4.1 now. Note that I don't think anyone is
proposing to closely track ffmpeg master forever. After the
MediaCodec support, I expect we'll go back to updating to newer ffmpeg
versions on an as needed bases.

David

> WIP on a particular feature rarely ever qualify.
>
> Anyhow, not that I’m much involved these days to have much weight on any decisions...
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On 17/05/18 15:51, David Engel wrote:
> On Thu, May 17, 2018 at 10:52:26AM +0200, Jean-Yves Avenard wrote:
<..>
>> That’s all good, but then that means that support for MediaCoded in FFmpeg is experimental anyway.. Do we want that in our release so soon?
>>
>> Are we willing to make everything unstable (from an API point of view) just to have support for something experimental that we didn’t support up to now anyway?
>
> Of course we don't want to intentionally break other things, but in
> case you haven't noticed, the Android port is the *only* significant
> work being done right now. IOW, there is no other work to be
> destabilized for the time being. If any other work does comes up and
> is affected, we can move one effort or the other to a branch.
>
>> If you ever ask a developer what version of his you should use, of course he/she’s always going to tell you to use the latest and greatest, luckily most team have a release manager :)
>>
>> 4.0 was a snapshot of ffmpeg done not even a month ago. By the time we’re ready with Peter’s change anyway, 4.1 would be out. FFmpeg make release frequently, when they are confident current master is in a stable state.
>
> We're, well, mainly Peter, are ready to start now. If we wait for
> ffmpeg 4.1 and it changes as much as you expect, wouldn't we have to
> start over again? In a sense, we're trying to get ahead of the curver
> and targeting for 4.1 now. Note that I don't think anyone is
> proposing to closely track ffmpeg master forever. After the
> MediaCodec support, I expect we'll go back to updating to newer ffmpeg
> versions on an as needed bases.
>

Given the planned release schedule we now have, the next "release" of
master is due in time for inclusion in ubuntu 19.10, which is around 18
months time.

Gives us plenty of time to stabilize things.

I say go for it.


Regards
Stuart Auchterlonie



_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
> On 17 May 2018, at 4:51 pm, David Engel <david@istwok.net> wrote:
>>
>> That’s all good, but then that means that support for MediaCoded in FFmpeg is experimental anyway.. Do we want that in our release so soon?
>>
>> Are we willing to make everything unstable (from an API point of view) just to have support for something experimental that we didn’t support up to now anyway?
>
> Of course we don't want to intentionally break other things, but in
> case you haven't noticed, the Android port is the *only* significant
> work being done right now. IOW, there is no other work to be
> destabilized for the time being. If any other work does comes up and
> is affected, we can move one effort or the other to a branch.
>
>> If you ever ask a developer what version of his you should use, of course he/she’s always going to tell you to use the latest and greatest, luckily most team have a release manager :)
>>
>> 4.0 was a snapshot of ffmpeg done not even a month ago. By the time we’re ready with Peter’s change anyway, 4.1 would be out. FFmpeg make release frequently, when they are confident current master is in a stable state.
>
> We're, well, mainly Peter, are ready to start now. If we wait for
> ffmpeg 4.1 and it changes as much as you expect, wouldn't we have to
> start over again? In a sense, we're trying to get ahead of the curver
> and targeting for 4.1 now. Note that I don't think anyone is
> proposing to closely track ffmpeg master forever. After the
> MediaCodec support, I expect we'll go back to updating to newer ffmpeg
> versions on an as needed bases.


If the aim is to only code support for MediaCodec, could always go with master temporarily, and switch to 4.1 once its frozen.
and then go from stable to stable
Re: Decision on FFmpeg repository [ In reply to ]
On 16/05/18 23:24, Jonatan Lindblad wrote:
> Den 2018-05-16 kl. 18:45, skrev Peter Bennett:
>> I would like to get a decision on what to do about a couple of FFmpeg questions:
>>
>> 1. Use FFmpeg master or release branch? My recommendation is to use master.
>> - the FFmpeg website recommends using master for those who are compiling FFmpeg themselves. It states that the release branches are for those who rely on downloaded prebuilt packages (https://ffmpeg.org/download.html#repositories).
>> - Only master has the recent mediacodec changes which I need for android.
>> - If we sync to master, we can use a better approach for the repository (question 2 below).
> Based on Jean-Yves' opinion I would not go for the master branch.
>
>> 2. What method should we use for storing and syncing with FFmpeg source?
>> Options are:
>> a. Reapply patches to the source from FFmpeg, resolve conflicts, and copy the source over what we have in MythTV (current process). This can be very time-consuming.
>>
>> b. Use a subtree in the MythTV repository.
>> - This will not work if we use a release branch (question 1).
>> - When resyncing, we can do a git merge of the subtree from FFmpeg.
>> - We can cherry-pick changes needed without resyncing everything.
>> - We can sync to any required commit, we don't need to go all the way to the latest commit.
>> - git log shows FFmpeg commits intermingled with MythTV. You can use "git log -- mythtv" which eliminates those.
>> - This imports some 90,000 extra commits from FFmpeg.
>>
>> You can see an example of the subtree in use at https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that repo and look at the ffmpeg-subtree branch.
>>
>> c. Keep our own customized FFmpeg repository with patches applied.
>> - This will not work if we use a release branch (question 1).
>> - When resyncing, do a git pull or merge on the private FFmpeg repository. Then copy the entire source over the MythTV FFmpeg source.
>> - We can cherry-pick changes needed without resyncing everything.
>> - We can sync to any required commit, we don't need to go all the way to the latest commit.
>> - git log in MythTV does not show all the FFmpeg commits.
>> - git log in the private FFmpeg repository shows records of our patches along with all the FFmpeg commits.
>> - We can perform periodic pulls into the private FFmpeg repository without affecting MythTV. This will make the eventual copying into MythTV quicker and easier.
>> - Any FFmpeg patches that we apply between copying into MythTV must be applied in both MythTV and FFmpeg repositories, or apply them in the private FFmpeg repository and then wait for the next copy into MythTV.
>>
>> You can see an example of FFmpeg with patches applied at https://github.com/bennettpeter/FFmpeg
> For 2c, would it not be easier to use a submodule instead of manually copying files from one repo to the other? I think I prefer that over 2b.
>
> Jonatan
>

Using a submodule would be my choice as well.

Paul H.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On 05/17/2018 01:14 PM, Paul Harrison wrote:
>>> You can see an example of FFmpeg with patches applied at
>>> https://github.com/bennettpeter/FFmpeg
>> For 2c, would it not be easier to use a submodule instead of manually
>> copying files from one repo to the other?  I think I prefer that over
>> 2b.
>>
>> Jonatan
>>
>
> Using a submodule would be my choice as well.
>
> Paul H.

My concern with submodules is

1. You have to run "git submodule init" when you first clone or when you
add a submodule.
2. You have to run "git submodule update" after "git pull" or when there
are submodule changes.
3. A commit hash may no longer identify what source code was used for a
build, because you cannot tell if they remembered to update the
submodule. I suppose we need to add a submodule hash to the version string.

The need to update the submodule will affect developers and build scripts.

Is this all correct? I have not used submodules, just read the
documentation. Any ideas on how to approach notifying everybody who may
be affected? Perhaps start 2c with the copy and give everybody some
period of time to add the necessary submodule code to build scripts,
then switch to the submodule.

Peter
Re: Decision on FFmpeg repository [ In reply to ]
> On 17 May 2018, at 9:05 pm, Peter Bennett <pb.mythtv@gmail.com> wrote:
>
> My concern with submodules is
>
> 1. You have to run "git submodule init" when you first clone or when you add a submodule.
> 2. You have to run "git submodule update" after "git pull" or when there are submodule changes.
> 3. A commit hash may no longer identify what source code was used for a build, because you cannot tell if they remembered to update the submodule. I suppose we need to add a submodule hash to the version string.
>
> The need to update the submodule will affect developers and build scripts.
>
> Is this all correct? I have not used submodules, just read the documentation. Any ideas on how to approach notifying everybody who may be affected? Perhaps start 2c with the copy and give everybody some period of time to add the necessary submodule code to build scripts, then switch to the submodule.
>

The configure script (or make file) typically ensures that the submodule is there and at the right value.
Re: Decision on FFmpeg repository [ In reply to ]
On Thu, May 17, 2018 at 05:51:43PM +0200, Jean-Yves Avenard wrote:
>
>
> > On 17 May 2018, at 4:51 pm, David Engel <david@istwok.net> wrote:
> >>
> >> That’s all good, but then that means that support for MediaCoded in FFmpeg is experimental anyway.. Do we want that in our release so soon?
> >>
> >> Are we willing to make everything unstable (from an API point of view) just to have support for something experimental that we didn’t support up to now anyway?
> >
> > Of course we don't want to intentionally break other things, but in
> > case you haven't noticed, the Android port is the *only* significant
> > work being done right now. IOW, there is no other work to be
> > destabilized for the time being. If any other work does comes up and
> > is affected, we can move one effort or the other to a branch.
> >
> >> If you ever ask a developer what version of his you should use, of course he/she’s always going to tell you to use the latest and greatest, luckily most team have a release manager :)
> >>
> >> 4.0 was a snapshot of ffmpeg done not even a month ago. By the time we’re ready with Peter’s change anyway, 4.1 would be out. FFmpeg make release frequently, when they are confident current master is in a stable state.
> >
> > We're, well, mainly Peter, are ready to start now. If we wait for
> > ffmpeg 4.1 and it changes as much as you expect, wouldn't we have to
> > start over again? In a sense, we're trying to get ahead of the curver
> > and targeting for 4.1 now. Note that I don't think anyone is
> > proposing to closely track ffmpeg master forever. After the
> > MediaCodec support, I expect we'll go back to updating to newer ffmpeg
> > versions on an as needed bases.
>
>
> If the aim is to only code support for MediaCodec, could always go with master temporarily, and switch to 4.1 once its frozen.
> and then go from stable to stable

My only current aim for ffmpeg is MediaCodec. Ater that is done, I
wouldn't mind going back to stable, ffmpeg releases. The git handling
with subtrees would probably be more complicated, but I have to
believe it can be done. In theory, it should boil down to a rebase.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
When talking to the FFmpeg devs, they always push for use of Master. They
claim it is generally stable.

One advantage of FFmpeg master, is that when they do deprecate an old API,
we would have a bit more "notice" since compiling then results in all of
the deprecated messages showing up.

A bad thing, is the mess of all those deprecated messages showing up! They
can be quite annoying.

As Peter has discovered, trying to use the FFmpeg branches is problematic.
It would be nice if MythTV master == FFmpeg master, and MythTV fixes ==
FFmpeg <latest stable>, but I don't think that is practical.

If David and Aman are willing / able to get our patches pushed to FFmpeg,
then this <hopefully> would become a non-issue, since we could just add an
external dependency on FFmpeg, instead of including it. Again, that is
probably not practical, though.

John
Re: Decision on FFmpeg repository [ In reply to ]
On May 16, 2018 10:24:09 PM EDT, Mark Spieth <mark@digivation.com.au> wrote:
>
>On 17/05/18 08:52, David Engel wrote:
>> On Thu, May 17, 2018 at 08:16:26AM +1000, Mark Spieth wrote:
>>> On 5/17/2018 2:45 AM, Peter Bennett wrote:
>>>> I would like to get a decision on what to do about a couple of
>FFmpeg
>>>> questions:
>>>>
>>>> 1. Use FFmpeg master or release branch? My recommendation is to use
>>>> master.
>>>> - the FFmpeg website recommends using master for those who are
>compiling
>>>> FFmpeg themselves. It states that the release branches are for
>those who
>>>> rely on downloaded prebuilt packages
>>>> (https://ffmpeg.org/download.html#repositories).
>>>> - Only master has the recent mediacodec changes which I need for
>android.
>>>> - If we sync to master, we can use a better approach for the
>repository
>>>> (question 2 below).
>>>>
>>>> 2. What method should we use for storing and syncing with FFmpeg
>source?
>>>> Options are:
>>>> a. Reapply patches to the source from FFmpeg, resolve conflicts,
>and
>>>> copy the source over what we have in MythTV (current process). This
>can
>>>> be very time-consuming.
>>>>
>>>> b. Use a subtree in the MythTV repository.
>>>> - This will not work if we use a release branch (question 1).
>>>> - When resyncing, we can do a git merge of the subtree from FFmpeg.
>>>> - We can cherry-pick changes needed without resyncing everything.
>>>> - We can sync to any required commit, we don't need to go all the
>way to
>>>> the latest commit.
>>>> - git log shows FFmpeg commits intermingled with MythTV. You can
>use
>>>> "git log -- mythtv" which eliminates those.
>>>> - This imports some 90,000 extra commits from FFmpeg.
>>>>
>>>> You can see an example of the subtree in use at
>>>> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone
>that
>>>> repo and look at the ffmpeg-subtree branch.
>>>>
>>>> c. Keep our own customized FFmpeg repository with patches applied.
>>>> - This will not work if we use a release branch (question 1).
>>>> - When resyncing, do a git pull or merge on the private FFmpeg
>>>> repository. Then copy the entire source over the MythTV FFmpeg
>source.
>>>> - We can cherry-pick changes needed without resyncing everything.
>>>> - We can sync to any required commit, we don't need to go all the
>way to
>>>> the latest commit.
>>>> - git log in MythTV does not show all the FFmpeg commits.
>>>> - git log in the private FFmpeg repository shows records of our
>patches
>>>> along with all the FFmpeg commits.
>>>> - We can perform periodic pulls into the private FFmpeg repository
>>>> without affecting MythTV. This will make the eventual copying into
>>>> MythTV quicker and easier.
>>>> - Any FFmpeg patches that we apply between copying into MythTV must
>be
>>>> applied in both MythTV and FFmpeg repositories, or apply them in
>the
>>>> private FFmpeg repository and then wait for the next copy into
>MythTV.
>>>>
>>>> You can see an example of FFmpeg with patches applied at
>>>> https://github.com/bennettpeter/FFmpeg
>>> I too am not comfortable using master but your justification is
>sound.
>>> Is there a prerelease with mediacodec added? probably not.
>>> You could start with master until mediacodec is released.
>>>
>>> Im not sure if you tried this but you could use 2b in a
>ffmpeg-tracking
>>> branch and then merge with squash to master, where the patches are.
>>> Im not sure how well this would work compared with tracking ffmpeg
>in master
>>> though wrt applying patches on top of ffmpeg compared with master
>only, but
>>> it would remove the log issue.
>>>
>>> Perhaps process of merge master to tracking, update subtree, fix
>conflicts
>>> in tracking, merge with squash back to master, fixup any issues in
>master.
>> I think that would have the same problem Peter ran into when trying
>2b
>> without squash. That is without the intermediate commits, git marked
>> everything as conflicting on subsequent updates because it didn't
>know
>> what to base them on.
>>
>but you do get the full history in the ffmpeg-tracking branch, just not
>
>master.
>BOBW

I'm not particularly concerned with branch vs master, or whether it's a submodule/subtree/subrepo. Whatever it is, I'd rather not have the git logs polluted with 90,000 extra entries. That's three times as many entries as exist for our main branch.

David
Re: Decision on FFmpeg repository [ In reply to ]
> On 17 May 2018, at 8:00 pm, mythtv-dev-request@mythtv.org wrote:
>
> I would like to get a decision on what to do about a couple of FFmpeg
> questions:
>
> 1. Use FFmpeg master or release branch? My recommendation is to use master.
> - the FFmpeg website recommends using master for those who are compiling
> FFmpeg themselves. It states that the release branches are for those who
> rely on downloaded prebuilt packages
> (https://ffmpeg.org/download.html#repositories).
> - Only master has the recent mediacodec changes which I need for android.
> - If we sync to master, we can use a better approach for the repository
> (question 2 below).
>
> 2. What method should we use for storing and syncing with FFmpeg source?
> Options are:
> a. Reapply patches to the source from FFmpeg, resolve conflicts, and
> copy the source over what we have in MythTV (current process). This can
> be very time-consuming.
>
> b. Use a subtree in the MythTV repository.
> - This will not work if we use a release branch (question 1).
> - When resyncing, we can do a git merge of the subtree from FFmpeg.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to
> the latest commit.
> - git log shows FFmpeg commits intermingled with MythTV. You can use
> "git log -- mythtv" which eliminates those.
> - This imports some 90,000 extra commits from FFmpeg.
>
> You can see an example of the subtree in use at
> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone that
> repo and look at the ffmpeg-subtree branch.
>
> c. Keep our own customized FFmpeg repository with patches applied.
> - This will not work if we use a release branch (question 1).
> - When resyncing, do a git pull or merge on the private FFmpeg
> repository. Then copy the entire source over the MythTV FFmpeg source.
> - We can cherry-pick changes needed without resyncing everything.
> - We can sync to any required commit, we don't need to go all the way to
> the latest commit.
> - git log in MythTV does not show all the FFmpeg commits.
> - git log in the private FFmpeg repository shows records of our patches
> along with all the FFmpeg commits.
> - We can perform periodic pulls into the private FFmpeg repository
> without affecting MythTV. This will make the eventual copying into
> MythTV quicker and easier.
> - Any FFmpeg patches that we apply between copying into MythTV must be
> applied in both MythTV and FFmpeg repositories, or apply them in the
> private FFmpeg repository and then wait for the next copy into MythTV.
>
> You can see an example of FFmpeg with patches applied at
> https://github.com/bennettpeter/FFmpeg


Peter ta very much.
It seems that if I pull mythtv and build (rather than download a distro prebuilt) then option 2c is the simplist.
James
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Thu, May 17, 2018 at 07:23:06PM -0400, David Hampton wrote:
>
>
> On May 16, 2018 10:24:09 PM EDT, Mark Spieth <mark@digivation.com.au> wrote:
> >
> >On 17/05/18 08:52, David Engel wrote:
> >> On Thu, May 17, 2018 at 08:16:26AM +1000, Mark Spieth wrote:
> >>> On 5/17/2018 2:45 AM, Peter Bennett wrote:
> >>>> I would like to get a decision on what to do about a couple of
> >FFmpeg
> >>>> questions:
> >>>>
> >>>> 1. Use FFmpeg master or release branch? My recommendation is to use
> >>>> master.
> >>>> - the FFmpeg website recommends using master for those who are
> >compiling
> >>>> FFmpeg themselves. It states that the release branches are for
> >those who
> >>>> rely on downloaded prebuilt packages
> >>>> (https://ffmpeg.org/download.html#repositories).
> >>>> - Only master has the recent mediacodec changes which I need for
> >android.
> >>>> - If we sync to master, we can use a better approach for the
> >repository
> >>>> (question 2 below).
> >>>>
> >>>> 2. What method should we use for storing and syncing with FFmpeg
> >source?
> >>>> Options are:
> >>>> a. Reapply patches to the source from FFmpeg, resolve conflicts,
> >and
> >>>> copy the source over what we have in MythTV (current process). This
> >can
> >>>> be very time-consuming.
> >>>>
> >>>> b. Use a subtree in the MythTV repository.
> >>>> - This will not work if we use a release branch (question 1).
> >>>> - When resyncing, we can do a git merge of the subtree from FFmpeg.
> >>>> - We can cherry-pick changes needed without resyncing everything.
> >>>> - We can sync to any required commit, we don't need to go all the
> >way to
> >>>> the latest commit.
> >>>> - git log shows FFmpeg commits intermingled with MythTV. You can
> >use
> >>>> "git log -- mythtv" which eliminates those.
> >>>> - This imports some 90,000 extra commits from FFmpeg.
> >>>>
> >>>> You can see an example of the subtree in use at
> >>>> https://github.com/bennettpeter/mythtv/tree/ffmpeg-subtree or clone
> >that
> >>>> repo and look at the ffmpeg-subtree branch.
> >>>>
> >>>> c. Keep our own customized FFmpeg repository with patches applied.
> >>>> - This will not work if we use a release branch (question 1).
> >>>> - When resyncing, do a git pull or merge on the private FFmpeg
> >>>> repository. Then copy the entire source over the MythTV FFmpeg
> >source.
> >>>> - We can cherry-pick changes needed without resyncing everything.
> >>>> - We can sync to any required commit, we don't need to go all the
> >way to
> >>>> the latest commit.
> >>>> - git log in MythTV does not show all the FFmpeg commits.
> >>>> - git log in the private FFmpeg repository shows records of our
> >patches
> >>>> along with all the FFmpeg commits.
> >>>> - We can perform periodic pulls into the private FFmpeg repository
> >>>> without affecting MythTV. This will make the eventual copying into
> >>>> MythTV quicker and easier.
> >>>> - Any FFmpeg patches that we apply between copying into MythTV must
> >be
> >>>> applied in both MythTV and FFmpeg repositories, or apply them in
> >the
> >>>> private FFmpeg repository and then wait for the next copy into
> >MythTV.
> >>>>
> >>>> You can see an example of FFmpeg with patches applied at
> >>>> https://github.com/bennettpeter/FFmpeg
> >>> I too am not comfortable using master but your justification is
> >sound.
> >>> Is there a prerelease with mediacodec added? probably not.
> >>> You could start with master until mediacodec is released.
> >>>
> >>> Im not sure if you tried this but you could use 2b in a
> >ffmpeg-tracking
> >>> branch and then merge with squash to master, where the patches are.
> >>> Im not sure how well this would work compared with tracking ffmpeg
> >in master
> >>> though wrt applying patches on top of ffmpeg compared with master
> >only, but
> >>> it would remove the log issue.
> >>>
> >>> Perhaps process of merge master to tracking, update subtree, fix
> >conflicts
> >>> in tracking, merge with squash back to master, fixup any issues in
> >master.
> >> I think that would have the same problem Peter ran into when trying
> >2b
> >> without squash. That is without the intermediate commits, git marked
> >> everything as conflicting on subsequent updates because it didn't
> >know
> >> what to base them on.
> >>
> >but you do get the full history in the ffmpeg-tracking branch, just not
> >
> >master.
> >BOBW
>
> I'm not particularly concerned with branch vs master, or whether it's a submodule/subtree/subrepo. Whatever it is, I'd rather not have the git logs polluted with 90,000 extra entries. That's three times as many entries as exist for our main branch.

I wonder if --no-ff would help when merging ffmpeg's changes into our
master branch. I know there are cases, like this maybe, where git
puts a single, "merge" commit on the main branch. When that happens,
I believe git requires extra log options to see all of the individual
commits that were merged.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Thu, 2018-05-17 at 15:05 -0400, Peter Bennett wrote:
> 3. A commit hash may no longer identify what source code was used for
> a build, because you cannot tell if they remembered to update the
> submodule. I suppose we need to add a submodule hash to the version
> string.

IIRC the commit of the submodule is committed to the main repo (in
.gitmodules or some such) and `git status` et al will show the repo as
dirty if the submodule is either dirty or not at the named revision.

The QEMU project uses submodules for various things so it might be
worth having a look at how they deal with it.

In the (dim and distant) past I have had a problem where my QEMU tree
was shown as dirty due to submodules and I couldn't for the life of me
reset it to be clean (and I am not a git neophyte) and resorted to
blowing away and recloning the tree. I suspect that was a bug in a long
gone version of git though rather than a fundamental flaw.

>
> The need to update the submodule will affect developers and build
> scripts.
>
> Is this all correct? I have not used submodules, just read the
> documentation. Any ideas on how to approach notifying everybody who
> may be affected? Perhaps start 2c with the copy and give everybody
> some period of time to add the necessary submodule code to build
> scripts, then switch to the submodule.
>
> Peter
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Decision on FFmpeg repository [ In reply to ]
On Fri, 2018-05-18 at 08:30 +0100, Ian Campbell wrote:
> On Thu, 2018-05-17 at 15:05 -0400, Peter Bennett wrote:
> > 3. A commit hash may no longer identify what source code was used for
> > a build, because you cannot tell if they remembered to update the
> > submodule. I suppose we need to add a submodule hash to the version
> > string.
>
> IIRC the commit of the submodule is committed to the main repo (in
> .gitmodules or some such) and `git status` et al will show the repo as
> dirty if the submodule is either dirty or not at the named revision.

It turns out IRinC(orrectly), in qemu.git/.gitmodules I see:
[submodule "roms/vgabios"]
path = roms/vgabios
url = git://git.qemu-project.org/vgabios.git/
[submodule "roms/seabios"]
path = roms/seabios
url = git://git.qemu-project.org/seabios.git/

So it seems the precise commit is not stored there. The
gitsubmodules(7) and gitmodules(5) manpages do not mention another
field which might be used to store the commit either.

I stand corrected.

Ian.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

1 2  View All