Mailing List Archive

Jitter with time stretch
Hi David

I do not use time stretch, so I do not know what is good and what is
bad. I tested time stretch with the current master version on Linux (no
patches), VAAPI and "-v playback" . With a 30fps recording it can be
speeded up to 2x with no log messages about aout of sync audio. With a
60fps recording (720p) from Comcast, it shows hundreds of errors, a
bunch of messages saying "Video is 20.0819 frames behind audio (too
slow), dropping frame to catch up." followed by a bunch of messages
saying "Video is 17.7729 frames ahead of audio, doubling video frame
interval to slow down.". This repeats.

On the shield, this type of thing also happens, including with 30 fps
interlaced content, perhaps because it is actually shown at 60 fps.

On the shield if you set your audio device to NULL it seems to be
perfectly smooth when speeding up to 2x.

I suspect what you are seeing is a pre-existing bug, not related to the
shield specifically, but showing up because of the frame doubling that
is happening on the shield.

I think the audio speedup may not be perfect, or maybe showing video at
higher rates than 60fps is problematic.

Peter

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On 7/15/2018 7:27 AM, Peter Bennett wrote:
> Hi David
>
> I do not use time stretch, so I do not know what is good and what is
> bad. I tested time stretch with the current master version on Linux
> (no patches), VAAPI and "-v playback" . With a 30fps recording it can
> be speeded up to 2x with no log messages about aout of sync audio.
> With a 60fps recording (720p) from Comcast, it shows hundreds of
> errors, a bunch of messages saying "Video is 20.0819 frames behind
> audio (too slow), dropping frame to catch up." followed by a bunch of
> messages saying "Video is 17.7729 frames ahead of audio, doubling
> video frame interval to slow down.". This repeats.
>
> On the shield, this type of thing also happens, including with 30 fps
> interlaced content, perhaps because it is actually shown at 60 fps.
>
> On the shield if you set your audio device to NULL it seems to be
> perfectly smooth when speeding up to 2x.
>
> I suspect what you are seeing is a pre-existing bug, not related to
> the shield specifically, but showing up because of the frame doubling
> that is happening on the shield.
>
> I think the audio speedup may not be perfect, or maybe showing video
> at higher rates than 60fps is problematic.
I notice with timestretch it is sensitive to the number of audio buffers
and video buffers to absorb any delta in A/V presentation in the
transport stream.
Increasing the audio buffering time helps in one direction (A precedes
V) and video buffers where V precedes A.
As a test, adjust audio sync to the point where you no longer get the
Video ahead of audio logs and that will give you an idea of how many
buffers are required.
Also with mediacodecs pipeline, this will exacerbate this V lagging
audio issue.
20 frames is about 750 ms at 30fps. so up the audio extra buffer to 1
sec should fix this.

Mark
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On Sun, Jul 15, 2018 at 10:14:35AM +1000, Mark Spieth wrote:
> On 7/15/2018 7:27 AM, Peter Bennett wrote:
> > Hi David
> >
> > I do not use time stretch, so I do not know what is good and what is
> > bad. I tested time stretch with the current master version on Linux (no
> > patches), VAAPI and "-v playback" . With a 30fps recording it can be
> > speeded up to 2x with no log messages about aout of sync audio. With a
> > 60fps recording (720p) from Comcast, it shows hundreds of errors, a
> > bunch of messages saying "Video is 20.0819 frames behind audio (too
> > slow), dropping frame to catch up." followed by a bunch of messages
> > saying "Video is 17.7729 frames ahead of audio, doubling video frame
> > interval to slow down.". This repeats.

I don't go by the playback, log messages except to try to narrow down
the problem when there is one. Rather, I use my eyeballs to see how
smooth the news ticker is on news channels. As long as the decoder
can keep up, it should be generally, pretty smooth with just a little
bit of jitter as frames need to be dropped or extended. At 2x, it
should be completely smooth as every other (60fps) frame gets dropped.

The generally, pretty smooth is exactly what I see on my main frontend
(Linux with VDPAU). That's also what I see on my development system
(Linux with VAAPI), albeit with a little more jitter.

> > On the shield, this type of thing also happens, including with 30 fps
> > interlaced content, perhaps because it is actually shown at 60 fps.

On my Shields with current master and current Android patch. 720p
starts to get a little more jittery than expected or wanted at just
below 2x. I believe it was good all the way to 2x before when there
were more video buffers. I need to retest that. 1080i has always
been more jittery than expected or wanted at any speed greater than
1x. This is *the* issue keeping from making the Shield my primary
frontend on my main TV.

> > On the shield if you set your audio device to NULL it seems to be
> > perfectly smooth when speeding up to 2x.

Did you try this with 1080i? 720p worked fine for me. With 1080i,
however, playback doesn't actually speed up for speeds greater than
1x. It stays at 1x even though MythTV reports it being faster. It
dows slow down for speeds less than 1x, though. Strange.

See below for more comments on the NULL audio test.(*)

> > I suspect what you are seeing is a pre-existing bug, not related to the
> > shield specifically, but showing up because of the frame doubling that
> > is happening on the shield.
> >
> > I think the audio speedup may not be perfect, or maybe showing video at
> > higher rates than 60fps is problematic.

I disagree about the pre-existing bug part. Once the processing gets
to the rendering stage, there is no real difference between rendering
video and audio that was originally 60fps from that which was
originally 30fps and then doubled to 60fps.

(*)I think the NULL audio test points the blame at the Shield's audio,
or more likely, our handling of audio on it. Does the audio timing
need to be adjusted in any way to compensate for the video frame rate
being doubled. Also, are we sure we are getting accureate timeing
information back from the audio subsystem on the Shield?

FWIW, I've often felt that something was a little off on the audio
timing. Sometimes it seems like my short skips ahead and back are a
few seconds longer or shorter than they should be. I realized that's
purely anecdotal, so take it with a huge grain of salt.

> I notice with timestretch it is sensitive to the number of audio buffers and
> video buffers to absorb any delta in A/V presentation in the transport
> stream.
> Increasing the audio buffering time helps in one direction (A precedes V)
> and video buffers where V precedes A.

Peter suggested increasing the audio buffering to 300ms or more a few
days ago. I did see some improvement going to 300. I didn't see much
improvement, if any, going as high as 2000ms. I have it set to 1000ms
currently.

> As a test, adjust audio sync to the point where you no longer get the Video
> ahead of audio logs and that will give you an idea of how many buffers are
> required.
> Also with mediacodecs pipeline, this will exacerbate this V lagging audio
> issue.
> 20 frames is about 750 ms at 30fps. so up the audio extra buffer to 1 sec
> should fix this.

I hope to do some more testing on this tomorrow and Monday.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On Sat, Jul 14, 2018 at 08:52:30PM -0500, David Engel wrote:
> On Sun, Jul 15, 2018 at 10:14:35AM +1000, Mark Spieth wrote:
> > On 7/15/2018 7:27 AM, Peter Bennett wrote:
> > > Hi David
> > >
> > > I do not use time stretch, so I do not know what is good and what is
> > > bad. I tested time stretch with the current master version on Linux (no
> > > patches), VAAPI and "-v playback" . With a 30fps recording it can be
> > > speeded up to 2x with no log messages about aout of sync audio. With a
> > > 60fps recording (720p) from Comcast, it shows hundreds of errors, a
> > > bunch of messages saying "Video is 20.0819 frames behind audio (too
> > > slow), dropping frame to catch up." followed by a bunch of messages
> > > saying "Video is 17.7729 frames ahead of audio, doubling video frame
> > > interval to slow down.". This repeats.
>
> I don't go by the playback, log messages except to try to narrow down
> the problem when there is one. Rather, I use my eyeballs to see how
> smooth the news ticker is on news channels. As long as the decoder
> can keep up, it should be generally, pretty smooth with just a little
> bit of jitter as frames need to be dropped or extended. At 2x, it
> should be completely smooth as every other (60fps) frame gets dropped.
>
> The generally, pretty smooth is exactly what I see on my main frontend
> (Linux with VDPAU). That's also what I see on my development system
> (Linux with VAAPI), albeit with a little more jitter.
>
> > > On the shield, this type of thing also happens, including with 30 fps
> > > interlaced content, perhaps because it is actually shown at 60 fps.
>
> On my Shields with current master and current Android patch. 720p
> starts to get a little more jittery than expected or wanted at just
> below 2x. I believe it was good all the way to 2x before when there
> were more video buffers. I need to retest that. 1080i has always
> been more jittery than expected or wanted at any speed greater than
> 1x. This is *the* issue keeping from making the Shield my primary
> frontend on my main TV.
>
> > > On the shield if you set your audio device to NULL it seems to be
> > > perfectly smooth when speeding up to 2x.
>
> Did you try this with 1080i? 720p worked fine for me. With 1080i,
> however, playback doesn't actually speed up for speeds greater than
> 1x. It stays at 1x even though MythTV reports it being faster. It
> dows slow down for speeds less than 1x, though. Strange.
>
> See below for more comments on the NULL audio test.(*)
>
> > > I suspect what you are seeing is a pre-existing bug, not related to the
> > > shield specifically, but showing up because of the frame doubling that
> > > is happening on the shield.
> > >
> > > I think the audio speedup may not be perfect, or maybe showing video at
> > > higher rates than 60fps is problematic.
>
> I disagree about the pre-existing bug part. Once the processing gets
> to the rendering stage, there is no real difference between rendering
> video and audio that was originally 60fps from that which was
> originally 30fps and then doubled to 60fps.

I found it! In MythPlayer::SetFrameInterval(), the frame_interval and
avsync_predictor_enabled are getting set wrong because frame_period
passed in is wrong. It is still set the 30fps equivalent instead of
the 60fps equivalent after the frame doubling in decoding.

After forcing frame_interval and avsync_predictor_enabled to the
correct values, I get smooth playback of my sample, 1080i content on
my 1080p TV at speed upto 1.90x. At 1.95x, a little bit of
non-predicted adjustment is needed, but it's barely noticeable. At
2.00x, it a little more noticeable, but really isn't too bad.

On my 4k TV, the results aren't quite as good. The video gets
noticeably jittry at 1.55x. It seems the scaling from 1080 to 4k
impacts how fast the decoding and deinterlacing can go. I wonder what
effect using Surface rendering would have on that.

Note that the above was all done with using the old numbers of buffers
(vbuffers.Init(31, true, 1, 12, 4, 2)). I'm going to see how far I
can reduce them without degrading the timestretching performance.

Finally, I don't have a proposed fix for the correct setting of
frame_interval and avsync_predictor_enabled yet. I'll defer to those
of you more familiar with the MythPlayer setup for the time being. It
probably needs to be tied into the code that Peter added to detect the
frame doubling during decoding.

David

> (*)I think the NULL audio test points the blame at the Shield's audio,
> or more likely, our handling of audio on it. Does the audio timing
> need to be adjusted in any way to compensate for the video frame rate
> being doubled. Also, are we sure we are getting accureate timeing
> information back from the audio subsystem on the Shield?
>
> FWIW, I've often felt that something was a little off on the audio
> timing. Sometimes it seems like my short skips ahead and back are a
> few seconds longer or shorter than they should be. I realized that's
> purely anecdotal, so take it with a huge grain of salt.
>
> > I notice with timestretch it is sensitive to the number of audio buffers and
> > video buffers to absorb any delta in A/V presentation in the transport
> > stream.
> > Increasing the audio buffering time helps in one direction (A precedes V)
> > and video buffers where V precedes A.
>
> Peter suggested increasing the audio buffering to 300ms or more a few
> days ago. I did see some improvement going to 300. I didn't see much
> improvement, if any, going as high as 2000ms. I have it set to 1000ms
> currently.
>
> > As a test, adjust audio sync to the point where you no longer get the Video
> > ahead of audio logs and that will give you an idea of how many buffers are
> > required.
> > Also with mediacodecs pipeline, this will exacerbate this V lagging audio
> > issue.
> > 20 frames is about 750 ms at 30fps. so up the audio extra buffer to 1 sec
> > should fix this.
>
> I hope to do some more testing on this tomorrow and Monday.
>
> David
> --
> David Engel
> david@istwok.net

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On 07/15/2018 04:50 PM, David Engel wrote:
> On Sat, Jul 14, 2018 at 08:52:30PM -0500, David Engel wrote:
>> On Sun, Jul 15, 2018 at 10:14:35AM +1000, Mark Spieth wrote:
>>> On 7/15/2018 7:27 AM, Peter Bennett wrote:
>>>> Hi David
>>>>
>>>> I do not use time stretch, so I do not know what is good and what is
>>>> bad. I tested time stretch with the current master version on Linux (no
>>>> patches), VAAPI and "-v playback" . With a 30fps recording it can be
>>>> speeded up to 2x with no log messages about aout of sync audio. With a
>>>> 60fps recording (720p) from Comcast, it shows hundreds of errors, a
>>>> bunch of messages saying "Video is 20.0819 frames behind audio (too
>>>> slow), dropping frame to catch up." followed by a bunch of messages
>>>> saying "Video is 17.7729 frames ahead of audio, doubling video frame
>>>> interval to slow down.". This repeats.
>> I don't go by the playback, log messages except to try to narrow down
>> the problem when there is one. Rather, I use my eyeballs to see how
>> smooth the news ticker is on news channels. As long as the decoder
>> can keep up, it should be generally, pretty smooth with just a little
>> bit of jitter as frames need to be dropped or extended. At 2x, it
>> should be completely smooth as every other (60fps) frame gets dropped.
>>
>> The generally, pretty smooth is exactly what I see on my main frontend
>> (Linux with VDPAU). That's also what I see on my development system
>> (Linux with VAAPI), albeit with a little more jitter.
>>
>>>> On the shield, this type of thing also happens, including with 30 fps
>>>> interlaced content, perhaps because it is actually shown at 60 fps.
>> On my Shields with current master and current Android patch. 720p
>> starts to get a little more jittery than expected or wanted at just
>> below 2x. I believe it was good all the way to 2x before when there
>> were more video buffers. I need to retest that. 1080i has always
>> been more jittery than expected or wanted at any speed greater than
>> 1x. This is *the* issue keeping from making the Shield my primary
>> frontend on my main TV.
>>
>>>> On the shield if you set your audio device to NULL it seems to be
>>>> perfectly smooth when speeding up to 2x.
>> Did you try this with 1080i? 720p worked fine for me. With 1080i,
>> however, playback doesn't actually speed up for speeds greater than
>> 1x. It stays at 1x even though MythTV reports it being faster. It
>> dows slow down for speeds less than 1x, though. Strange.
>>
>> See below for more comments on the NULL audio test.(*)
>>
>>>> I suspect what you are seeing is a pre-existing bug, not related to the
>>>> shield specifically, but showing up because of the frame doubling that
>>>> is happening on the shield.
>>>>
>>>> I think the audio speedup may not be perfect, or maybe showing video at
>>>> higher rates than 60fps is problematic.
>> I disagree about the pre-existing bug part. Once the processing gets
>> to the rendering stage, there is no real difference between rendering
>> video and audio that was originally 60fps from that which was
>> originally 30fps and then doubled to 60fps.
> I found it! In MythPlayer::SetFrameInterval(), the frame_interval and
> avsync_predictor_enabled are getting set wrong because frame_period
> passed in is wrong. It is still set the 30fps equivalent instead of
> the 60fps equivalent after the frame doubling in decoding.
>
> After forcing frame_interval and avsync_predictor_enabled to the
> correct values, I get smooth playback of my sample, 1080i content on
> my 1080p TV at speed upto 1.90x. At 1.95x, a little bit of
> non-predicted adjustment is needed, but it's barely noticeable. At
> 2.00x, it a little more noticeable, but really isn't too bad.
>
> On my 4k TV, the results aren't quite as good. The video gets
> noticeably jittry at 1.55x. It seems the scaling from 1080 to 4k
> impacts how fast the decoding and deinterlacing can go. I wonder what
> effect using Surface rendering would have on that.
>
> Note that the above was all done with using the old numbers of buffers
> (vbuffers.Init(31, true, 1, 12, 4, 2)). I'm going to see how far I
> can reduce them without degrading the timestretching performance.
>
> Finally, I don't have a proposed fix for the correct setting of
> frame_interval and avsync_predictor_enabled yet. I'll defer to those
> of you more familiar with the MythPlayer setup for the time being. It
> probably needs to be tied into the code that Peter added to detect the
> frame doubling during decoding.
>
> David
>
>
I agree - frame_interval needs fixing. We just need to be careful of not
using the adjusted value when seeking, because for file access it uses
the original frame number. For example in MythPlayer::UpdateFFRewSkip, I
amnot quite sure what that is doing with it. I will go through and check
all the places where frame_interval is used and see if I can get it right.

Peter
Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On Sun, Jul 15, 2018 at 07:05:21PM -0400, Peter Bennett wrote:
> On 07/15/2018 04:50 PM, David Engel wrote:
> > I found it! In MythPlayer::SetFrameInterval(), the frame_interval and
> > avsync_predictor_enabled are getting set wrong because frame_period
> > passed in is wrong. It is still set the 30fps equivalent instead of
> > the 60fps equivalent after the frame doubling in decoding.
> >
> > After forcing frame_interval and avsync_predictor_enabled to the
> > correct values, I get smooth playback of my sample, 1080i content on
> > my 1080p TV at speed upto 1.90x. At 1.95x, a little bit of
> > non-predicted adjustment is needed, but it's barely noticeable. At
> > 2.00x, it a little more noticeable, but really isn't too bad.
> >
> > On my 4k TV, the results aren't quite as good. The video gets
> > noticeably jittry at 1.55x. It seems the scaling from 1080 to 4k
> > impacts how fast the decoding and deinterlacing can go. I wonder what
> > effect using Surface rendering would have on that.
> >
> > Note that the above was all done with using the old numbers of buffers
> > (vbuffers.Init(31, true, 1, 12, 4, 2)). I'm going to see how far I
> > can reduce them without degrading the timestretching performance.

Things seem to be okay with the modified numbers of buffers
(vbuffers.Init(4, true, 1, 2, 2, 1)). I have more TV watching, I mean
testing, to do, though. :)

> > Finally, I don't have a proposed fix for the correct setting of
> > frame_interval and avsync_predictor_enabled yet. I'll defer to those
> > of you more familiar with the MythPlayer setup for the time being. It
> > probably needs to be tied into the code that Peter added to detect the
> > frame doubling during decoding.
> >
> > David
> >
> >
> I agree - frame_interval needs fixing. We just need to be careful of not
> using the adjusted value when seeking, because for file access it uses the
> original frame number. For example in MythPlayer::UpdateFFRewSkip, I amnot
> quite sure what that is doing with it. I will go through and check all the
> places where frame_interval is used and see if I can get it right.

We could possibly get by with just a new variable that flags when the
decoder doubles the frame rate. I think the modified frame_interval
is only needed in MythPlayer::SetFrameInterval() where
avsync_predictor_enabled is initialized and in MythPlayer::AVSync()
where avsync_predictor_enabled is checked and used.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On 07/15/2018 10:02 PM, David Engel wrote:
> On Sun, Jul 15, 2018 at 07:05:21PM -0400, Peter Bennett wrote:
>> On 07/15/2018 04:50 PM, David Engel wrote:
>>> I found it! In MythPlayer::SetFrameInterval(), the frame_interval and
>>> avsync_predictor_enabled are getting set wrong because frame_period
>>> passed in is wrong. It is still set the 30fps equivalent instead of
>>> the 60fps equivalent after the frame doubling in decoding.
>>>
>>> After forcing frame_interval and avsync_predictor_enabled to the
>>> correct values, I get smooth playback of my sample, 1080i content on
>>> my 1080p TV at speed upto 1.90x. At 1.95x, a little bit of
>>> non-predicted adjustment is needed, but it's barely noticeable. At
>>> 2.00x, it a little more noticeable, but really isn't too bad.
>>>
>>> On my 4k TV, the results aren't quite as good. The video gets
>>> noticeably jittry at 1.55x. It seems the scaling from 1080 to 4k
>>> impacts how fast the decoding and deinterlacing can go. I wonder what
>>> effect using Surface rendering would have on that.
>>>
>>> Note that the above was all done with using the old numbers of buffers
>>> (vbuffers.Init(31, true, 1, 12, 4, 2)). I'm going to see how far I
>>> can reduce them without degrading the timestretching performance.
> Things seem to be okay with the modified numbers of buffers
> (vbuffers.Init(4, true, 1, 2, 2, 1)). I have more TV watching, I mean
> testing, to do, though. :)
I am adding a line to only allocate the lower number of buffers if it is
mediacodec. I found that using the smaller number on Linux with OpenGL
profile turns playback into a slideshow with corrupted frames.
>>> Finally, I don't have a proposed fix for the correct setting of
>>> frame_interval and avsync_predictor_enabled yet. I'll defer to those
>>> of you more familiar with the MythPlayer setup for the time being. It
>>> probably needs to be tied into the code that Peter added to detect the
>>> frame doubling during decoding.
>>>
>>> David
>>>
>>>
>> I agree - frame_interval needs fixing. We just need to be careful of not
>> using the adjusted value when seeking, because for file access it uses the
>> original frame number. For example in MythPlayer::UpdateFFRewSkip, I amnot
>> quite sure what that is doing with it. I will go through and check all the
>> places where frame_interval is used and see if I can get it right.
> We could possibly get by with just a new variable that flags when the
> decoder doubles the frame rate. I think the modified frame_interval
> is only needed in MythPlayer::SetFrameInterval() where
> avsync_predictor_enabled is initialized and in MythPlayer::AVSync()
> where avsync_predictor_enabled is checked and used.
I already have a flag stored in the decoder and accessed from mythplayer
using "decoder->GetfpsMultiplier()" It takes values 1 or 2, 1=normal,
2=doubled frames as with mediacodec.

I am currently using fpsMultiplier only in AVSync where frame_delay is
calculated from frame_interval. I need to take that out and change the
calculation of frame_interval itself to use fpsMultiplier instead.

Peter



> David

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On Mon, Jul 16, 2018 at 10:47:53AM -0400, Peter Bennett wrote:
>
>
> On 07/15/2018 10:02 PM, David Engel wrote:
> > On Sun, Jul 15, 2018 at 07:05:21PM -0400, Peter Bennett wrote:
> > > On 07/15/2018 04:50 PM, David Engel wrote:
> > > > I found it! In MythPlayer::SetFrameInterval(), the frame_interval and
> > > > avsync_predictor_enabled are getting set wrong because frame_period
> > > > passed in is wrong. It is still set the 30fps equivalent instead of
> > > > the 60fps equivalent after the frame doubling in decoding.
> > > >
> > > > After forcing frame_interval and avsync_predictor_enabled to the
> > > > correct values, I get smooth playback of my sample, 1080i content on
> > > > my 1080p TV at speed upto 1.90x. At 1.95x, a little bit of
> > > > non-predicted adjustment is needed, but it's barely noticeable. At
> > > > 2.00x, it a little more noticeable, but really isn't too bad.
> > > >
> > > > On my 4k TV, the results aren't quite as good. The video gets
> > > > noticeably jittry at 1.55x. It seems the scaling from 1080 to 4k
> > > > impacts how fast the decoding and deinterlacing can go. I wonder what
> > > > effect using Surface rendering would have on that.
> > > >
> > > > Note that the above was all done with using the old numbers of buffers
> > > > (vbuffers.Init(31, true, 1, 12, 4, 2)). I'm going to see how far I
> > > > can reduce them without degrading the timestretching performance.
> > Things seem to be okay with the modified numbers of buffers
> > (vbuffers.Init(4, true, 1, 2, 2, 1)). I have more TV watching, I mean
> > testing, to do, though. :)
> I am adding a line to only allocate the lower number of buffers if it is
> mediacodec. I found that using the smaller number on Linux with OpenGL
> profile turns playback into a slideshow with corrupted frames.
> > > > Finally, I don't have a proposed fix for the correct setting of
> > > > frame_interval and avsync_predictor_enabled yet. I'll defer to those
> > > > of you more familiar with the MythPlayer setup for the time being. It
> > > > probably needs to be tied into the code that Peter added to detect the
> > > > frame doubling during decoding.
> > > >
> > > > David
> > > >
> > > >
> > > I agree - frame_interval needs fixing. We just need to be careful of not
> > > using the adjusted value when seeking, because for file access it uses the
> > > original frame number. For example in MythPlayer::UpdateFFRewSkip, I amnot
> > > quite sure what that is doing with it. I will go through and check all the
> > > places where frame_interval is used and see if I can get it right.
> > We could possibly get by with just a new variable that flags when the
> > decoder doubles the frame rate. I think the modified frame_interval
> > is only needed in MythPlayer::SetFrameInterval() where
> > avsync_predictor_enabled is initialized and in MythPlayer::AVSync()
> > where avsync_predictor_enabled is checked and used.
> I already have a flag stored in the decoder and accessed from mythplayer
> using "decoder->GetfpsMultiplier()" It takes values 1 or 2, 1=normal,
> 2=doubled frames as with mediacodec.

Excellent!

> I am currently using fpsMultiplier only in AVSync where frame_delay is
> calculated from frame_interval. I need to take that out and change the
> calculation of frame_interval itself to use fpsMultiplier instead.

So are you going to make the avsync_predictor_enabled change then or
do you want me to do it? I have time for it today.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On 07/16/2018 11:37 AM, David Engel wrote:
>> I already have a flag stored in the decoder and accessed from mythplayer
>> using "decoder->GetfpsMultiplier()" It takes values 1 or 2, 1=normal,
>> 2=doubled frames as with mediacodec.
> Excellent!
>
>> I am currently using fpsMultiplier only in AVSync where frame_delay is
>> calculated from frame_interval. I need to take that out and change the
>> calculation of frame_interval itself to use fpsMultiplier instead.
> So are you going to make the avsync_predictor_enabled change then or
> do you want me to do it? I have time for it today.
I have created a new patch that changes frame_interval calculation, and
attached it to the ticket. I also set the buffers to 8 instead of 4.
With 4 buffers there still was a slight jiggle in the 2x speedup
playback, that is gone with 8 buffers. See what you think.

FF seems to be working OK with this patch. The failure is rather random.
The re-init handles it with a slight pause when it does happen. Do you
think there is any point in trying to change FF to use software decoding?

I think I am almost ready to commit this. First I will do some testing
on Linux with different playback profiles, and on raspberry Pi, to make
sure nothing got broken. I want to split it into a few commits to
separate some things that were not exclusively for mediacodec, and
separate out the FFmpeg fix. Anything that is logically separate I will
make a separate commit. I see no sign that Aman plans to commit the
FFmpeg fix. That needs to be committed to the MythTV private FFmpeg
repository as well.

Peter
Re: Jitter with time stretch [ In reply to ]
On Mon, Jul 16, 2018 at 11:59:46AM -0400, Peter Bennett wrote:
> On 07/16/2018 11:37 AM, David Engel wrote:
> > > I already have a flag stored in the decoder and accessed from mythplayer
> > > using "decoder->GetfpsMultiplier()" It takes values 1 or 2, 1=normal,
> > > 2=doubled frames as with mediacodec.
> > Excellent!
> >
> > > I am currently using fpsMultiplier only in AVSync where frame_delay is
> > > calculated from frame_interval. I need to take that out and change the
> > > calculation of frame_interval itself to use fpsMultiplier instead.
> > So are you going to make the avsync_predictor_enabled change then or
> > do you want me to do it? I have time for it today.
> I have created a new patch that changes frame_interval calculation, and
> attached it to the ticket. I also set the buffers to 8 instead of 4. With 4
> buffers there still was a slight jiggle in the 2x speedup playback, that is
> gone with 8 buffers. See what you think.

I already noticed the new patch and am about to build.

> FF seems to be working OK with this patch. The failure is rather random. The
> re-init handles it with a slight pause when it does happen. Do you think
> there is any point in trying to change FF to use software decoding?

I used my Shield almost exclusviely for watching yesterday. There are
still some rough edges like the IllegalStateException and hangs when
left in the background for too long, but we're definitely very close
being able to use it full time. In fact we're close enough that I
think I'm going to pick up one or two Shields for my mom on the Prime
Day sale.

> I think I am almost ready to commit this. First I will do some testing on
> Linux with different playback profiles, and on raspberry Pi, to make sure
> nothing got broken. I want to split it into a few commits to separate some
> things that were not exclusively for mediacodec, and separate out the FFmpeg
> fix. Anything that is logically separate I will make a separate commit. I
> see no sign that Aman plans to commit the FFmpeg fix. That needs to be
> committed to the MythTV private FFmpeg repository as well.

Sounds good on the commit. Maybe give Aman a gentle reminder. If he
got busy with other things, he could have just forgotten.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On Mon, Jul 16, 2018 at 11:59:46AM -0400, Peter Bennett wrote:
> On 07/16/2018 11:37 AM, David Engel wrote:
> > > I already have a flag stored in the decoder and accessed from mythplayer
> > > using "decoder->GetfpsMultiplier()" It takes values 1 or 2, 1=normal,
> > > 2=doubled frames as with mediacodec.
> > Excellent!
> >
> > > I am currently using fpsMultiplier only in AVSync where frame_delay is
> > > calculated from frame_interval. I need to take that out and change the
> > > calculation of frame_interval itself to use fpsMultiplier instead.
> > So are you going to make the avsync_predictor_enabled change then or
> > do you want me to do it? I have time for it today.
> I have created a new patch that changes frame_interval calculation, and
> attached it to the ticket. I also set the buffers to 8 instead of 4. With 4
> buffers there still was a slight jiggle in the 2x speedup playback, that is
> gone with 8 buffers. See what you think.

Initial testing with recordings looks good. Live TV results in a
crash, though. Let me know if you can't reproduce it and I'll send
you a backtrace.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On 07/16/2018 02:52 PM, David Engel wrote:
> Initial testing with recordings looks good. Live TV results in a
> crash, though. Let me know if you can't reproduce it and I'll send
> you a backtrace.
Yes I see it. Live TV seems to have a couple of issues. I will get it
sorted out.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On 07/14/2018 09:52 PM, David Engel wrote:
> Did you try this with 1080i? 720p worked fine for me. With 1080i,
> however, playback doesn't actually speed up for speeds greater than
> 1x. It stays at 1x even though MythTV reports it being faster. It
> dows slow down for speeds less than 1x, though. Strange.
I did some timing on NULL audio and I discovered that with 1080i
content, the method VideoOutputOpenGL::ProcessFrame takes from 13 - 15
milliseconds. Also sometimes VideoOutputOpenGL::PrepareFrame takes 5
milliseconds. At 60 fps, one frame needs to be displayed every 16.6
milliseconds, so OpenGL the way we are using it cannot display more than
60 fps of 1920x1080 images. At 60fps it is running at maximum speed.
Without audio there is nothing that will drop frames to force it to
catch up. The time stretch works with software decoding and NULL audio,
presumably because that is rendered at 30 fps not 60.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Jitter with time stretch [ In reply to ]
On Tue, Jul 17, 2018 at 11:05:53AM -0400, Peter Bennett wrote:
> On 07/14/2018 09:52 PM, David Engel wrote:
> > Did you try this with 1080i? 720p worked fine for me. With 1080i,
> > however, playback doesn't actually speed up for speeds greater than
> > 1x. It stays at 1x even though MythTV reports it being faster. It
> > dows slow down for speeds less than 1x, though. Strange.
> I did some timing on NULL audio and I discovered that with 1080i content,
> the method VideoOutputOpenGL::ProcessFrame takes from 13 - 15 milliseconds.
> Also sometimes VideoOutputOpenGL::PrepareFrame takes 5 milliseconds. At 60
> fps, one frame needs to be displayed every 16.6 milliseconds, so OpenGL the
> way we are using it cannot display more than 60 fps of 1920x1080 images. At
> 60fps it is running at maximum speed. Without audio there is nothing that
> will drop frames to force it to catch up. The time stretch works with
> software decoding and NULL audio, presumably because that is rendered at 30
> fps not 60.

Okay. That's a very strange coincidence, though. Surface rendering
should remove some of that overhead.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org