Mailing List Archive

Re: [mythtv-commits] Ticket #12827: Raspberry Pi: Digital Audio does not work
On Wed, Jul 27, 2016 at 8:25 AM MythTV <noreply@mythtv.org> wrote:

> #12827: Raspberry Pi: Digital Audio does not work
> ----------------------------------+-----------------------------
> Reporter: pbennett | Owner: pbennett
> Type: Bug Report - General | Status: accepted
> Priority: minor | Milestone: unknown
> Component: Ports - rPi | Version: Master Head
> Severity: medium | Resolution:
> Keywords: | Ticket locked: 0
> ----------------------------------+-----------------------------
>
> Comment (by pbennett):
>
> Output of Dolby digital passthru audio to a surround sound system works
> and gives sound, but the audio and video are jerky. This is because the
> omx audio system seems to give incorrect and inconsistent values for audio
> latency (i.e. amount of audio in the sound card that has not yet been
> played). The result is that AVSync reports Video is behind audio (too
> slow) for a few seconds and drops some frames to fix it, then reports that
> Video is ahead of audio for a few seconds, and slows video down to
> compensate. Thus the video is alternately speeded up and slowed down every
> few seconds and the audio is paused and restarted every few seconds.
>
> The audio latency values are incorrect only in the digital passthru case.
> If playing digital stereo or analog, it works perfectly.
>
> If I set very high values for MAXDIVERGE and DIVERGELIMIT so that AVSync
> ignores the errors, the video plays very well. 5.1 channel Dolby Digital
> audio is good, the surround sound receiver reports it as Dolby Digital.
> The video is in good sync with the sound. However this is not a solution,
> it will certainly get out of sync soon.
>
> Options
>
> 1. Try to get some code that will estimate the latency based on amount of
> data passed in and elapsed time. I am afraid this would also gradually get
> worse over time. Also I am not sure how to handle pauses.
>
> 2. Try to minimize latency by passing data to audio only when it is
> needed. This is difficult because audio is in a ring buffer before going
> to the sound device and the ring buffer does not keep track of time codes.
>
> 3. Assume some fixed value for sound card latency all the time.
>
> 4. Average the sync calculation over 30, 75, 100, or 300 frames. Currently
> it is averaged over 4 frames. This assumes that on average over time the
> sound card latencies will be correct. I am afraid this may cause other
> problems, for example when using softblend and displaying OSD the AVSync
> has to compensate for the extra CPU load by dropping some frames. That
> works well but will not work with the averaging calculation.
>
> I am not sure how to proceed.
>

On first blush, I would be tempted to go with (2) and change the ringbuffer
to track time codes. I am not familiar with that code, though, so I don't
know how hard or practical that is.

John
Re: [mythtv-commits] Ticket #12827: Raspberry Pi: Digital Audio does not work [ In reply to ]
I am no dev... but it sounds like OMX is handling the video and audio sync
just fine... the real problem is mythtv trying to control the sync when it
really doesn't need to.

Is there any reason you couldn't just not check for sync at all in the case
of pass-through? What is the worst that would happen? I would think that
the audio and video are coming out of the decoder in sync, just push them
into the render pipeline in sync and all will be well?

Your MAXDIVERGE test suggests that the chipset does the right thing as long
as the buffers are full. While you seem suspicious of this, I would love
to hear the results of some long term testing of that rather than an
assumption that it will get out of sync eventually. It may actually be the
solution?


On Wed, Jul 27, 2016 at 12:24 PM John P Poet <jppoet@gmail.com> wrote:

> On Wed, Jul 27, 2016 at 8:25 AM MythTV <noreply@mythtv.org> wrote:
>
>> #12827: Raspberry Pi: Digital Audio does not work
>> ----------------------------------+-----------------------------
>> Reporter: pbennett | Owner: pbennett
>> Type: Bug Report - General | Status: accepted
>> Priority: minor | Milestone: unknown
>> Component: Ports - rPi | Version: Master Head
>> Severity: medium | Resolution:
>> Keywords: | Ticket locked: 0
>> ----------------------------------+-----------------------------
>>
>> Comment (by pbennett):
>>
>> Output of Dolby digital passthru audio to a surround sound system works
>> and gives sound, but the audio and video are jerky. This is because the
>> omx audio system seems to give incorrect and inconsistent values for
>> audio
>> latency (i.e. amount of audio in the sound card that has not yet been
>> played). The result is that AVSync reports Video is behind audio (too
>> slow) for a few seconds and drops some frames to fix it, then reports
>> that
>> Video is ahead of audio for a few seconds, and slows video down to
>> compensate. Thus the video is alternately speeded up and slowed down
>> every
>> few seconds and the audio is paused and restarted every few seconds.
>>
>> The audio latency values are incorrect only in the digital passthru case.
>> If playing digital stereo or analog, it works perfectly.
>>
>> If I set very high values for MAXDIVERGE and DIVERGELIMIT so that AVSync
>> ignores the errors, the video plays very well. 5.1 channel Dolby Digital
>> audio is good, the surround sound receiver reports it as Dolby Digital.
>> The video is in good sync with the sound. However this is not a solution,
>> it will certainly get out of sync soon.
>>
>> Options
>>
>> 1. Try to get some code that will estimate the latency based on amount of
>> data passed in and elapsed time. I am afraid this would also gradually
>> get
>> worse over time. Also I am not sure how to handle pauses.
>>
>> 2. Try to minimize latency by passing data to audio only when it is
>> needed. This is difficult because audio is in a ring buffer before going
>> to the sound device and the ring buffer does not keep track of time
>> codes.
>>
>> 3. Assume some fixed value for sound card latency all the time.
>>
>> 4. Average the sync calculation over 30, 75, 100, or 300 frames.
>> Currently
>> it is averaged over 4 frames. This assumes that on average over time the
>> sound card latencies will be correct. I am afraid this may cause other
>> problems, for example when using softblend and displaying OSD the AVSync
>> has to compensate for the extra CPU load by dropping some frames. That
>> works well but will not work with the averaging calculation.
>>
>> I am not sure how to proceed.
>>
>
> On first blush, I would be tempted to go with (2) and change the
> ringbuffer to track time codes. I am not familiar with that code, though,
> so I don't know how hard or practical that is.
>
> John
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
Re: [mythtv-commits] Ticket #12827: Raspberry Pi: Digital Audio does not work [ In reply to ]
On 07/27/2016 01:40 PM, Joseph Fry wrote:
> I am no dev... but it sounds like OMX is handling the video and audio
> sync just fine... the real problem is mythtv trying to control the
> sync when it really doesn't need to.
>
> Is there any reason you couldn't just not check for sync at all in the
> case of pass-through? What is the worst that would happen? I would
> think that the audio and video are coming out of the decoder in sync,
> just push them into the render pipeline in sync and all will be well?
>
> Your MAXDIVERGE test suggests that the chipset does the right thing as
> long as the buffers are full. While you seem suspicious of this, I
> would love to hear the results of some long term testing of that
> rather than an assumption that it will get out of sync eventually. It
> may actually be the solution?
>
There are a couple of problems with this

1. When displaying the OSD or carrying out certain operations from the
keyboard or remote, the raspberry pi becomes overloaded and compensates
by dropping frames. This is done using the AVSync. If that is disabled,
then lip-sync will start going out if you press buttons on the keyboard.

2. With broadcast video, even digital broadcast, there are small errors
and glitches that happen from time to time. If there is nothing to sync
the audio and video these can build up and after some minutes of
watching the lip sync will likely start going off.
Re: [mythtv-commits] Ticket #12827: Raspberry Pi: Digital Audio does not work [ In reply to ]
On Wed, Jul 27, 2016 at 3:45 PM Peter Bennett <pgbennett@comcast.net> wrote:

> On 07/27/2016 01:40 PM, Joseph Fry wrote:
>
> I am no dev... but it sounds like OMX is handling the video and audio sync
> just fine... the real problem is mythtv trying to control the sync when it
> really doesn't need to.
>
> Is there any reason you couldn't just not check for sync at all in the
> case of pass-through? What is the worst that would happen? I would think
> that the audio and video are coming out of the decoder in sync, just push
> them into the render pipeline in sync and all will be well?
>
> Your MAXDIVERGE test suggests that the chipset does the right thing as
> long as the buffers are full. While you seem suspicious of this, I would
> love to hear the results of some long term testing of that rather than an
> assumption that it will get out of sync eventually. It may actually be the
> solution?
>
> There are a couple of problems with this
>
> 1. When displaying the OSD or carrying out certain operations from the
> keyboard or remote, the raspberry pi becomes overloaded and compensates by
> dropping frames. This is done using the AVSync. If that is disabled, then
> lip-sync will start going out if you press buttons on the keyboard.
>
> 2. With broadcast video, even digital broadcast, there are small errors
> and glitches that happen from time to time. If there is nothing to sync the
> audio and video these can build up and after some minutes of watching the
> lip sync will likely start going off.
>

This makes sense, but it sounds like we rely on OMX to report when the data
is out of sync... can we not adjust for sync issues before the data is sent
to for rendering? Perhaps that's what he was suggesting with his option
2? I apologize for derailing the discussion