Mailing List Archive

Cannot max out CPU usage on HLS transcode
I am trying to get the HLS transcode stream to better utilise my CPU (i7 3930k) to get the transcode completed in a shorter time (to allow fast forwarding when playing back), however at most it uses 50 - 60% of the CPU capacity (an probably nearer to 40%).

I have tried setting the HTTPLiveStreamThreads configuration parameter in the database to better utilise the CPU (I can see the number of threads increased) but it doesn't matter as only 6 threads seem to be using the CPU - the rest sit at 0%.

Am I doing something wrong or is there some other limiting factor on how fast mythtv can process the transcoding process (handbreak can max out the CPU)?

Cheers,

Jim.




_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 15/10/12 08:12, Jim Downie wrote:
>
> I am trying to get the HLS transcode stream to better utilise my CPU (i7 3930k) to get the transcode completed in a shorter time (to allow fast forwarding when playing back), however at most it uses 50 - 60% of the CPU capacity (an probably nearer to 40%).
>
> I have tried setting the HTTPLiveStreamThreads configuration parameter in the database to better utilise the CPU (I can see the number of threads increased) but it doesn't matter as only 6 threads seem to be using the CPU - the rest sit at 0%.
>
> Am I doing something wrong or is there some other limiting factor on how fast mythtv can process the transcoding process (handbreak can max out the CPU)?
>
> Cheers,
>
> Jim.
>

How about disk IO? atop often shows mine near 100% when
post-processing, with a much more modest CPU.

John P


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
Do not believe it's IO bound as handbreak is able to max out CPU reading and writing to the same partition (10 disk raid 6 array) - see IO stats below.

root@masterbackend:/data$ dd if=/dev/zero of=/data/test bs=1G count=2
2+0 records in
2+0 records out
2147483648 bytes (2.1 GB) copied, 4.87213 s, 441 MB/s





_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
> How about disk IO? atop often shows mine near 100% when
> post-processing, with a much more modest CPU.
>
> John P

Do not believe it's IO bound as handbreak is able to max out CPU reading and writing to the same partition (10 disk raid 6 array) - see IO stats below.

root@masterbackend:/data$ dd if=/dev/zero of=/data/test bs=1G count=2
2+0 records in
2+0 records out
2147483648 bytes (2.1 GB) copied, 4.87213 s, 441 MB/s





_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 10/15/2012 03:12, Jim Downie wrote:
> only 6 threads seem to be using the CPU - the rest sit at 0%

That's because you only have six cores, and your CPU is running at 100%
load. Ignore the hyperthreading hiding behind the curtain, feeding you
bad statistics.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
> On 10/15/2012 03:12, Jim Downie wrote:
> > only 6 threads seem to be using the CPU - the rest sit at 0%

> That's because you only have six cores, and your CPU is running at 100% load. Ignore the hyperthreading hiding behind the curtain, feeding you bad statistics.


Nope - that's not it.

I have disabled hyper threading (so only 6 cores available to Ubuntu 12.04) and am monitoring the performance through i7z.


What this shows is that the CPU core frequencies predominantly stay low during HLS transcode (it can go up to 4.1GHz) and the thermal monitors show 40 degrees C (5 degree rise from no load) - it looks like it is using 30% of the available cpu's .


When I run handbreak to transcode a video the frequency goes to the set limit (4.1GHz) and I see the thermal monitor hit 61 degrees C (I'm OK with that).


There also seems to be a limit on HTTPLiveStreamThreads - tried setting it to 99 (didn't see number of threads expected) - believe limit is 9.


See below process table during single HLS transcode (HTTPLiveStreamThreads set at 3)

downiej@masterbackend:/data$ ps -eLf | grep trans
mythtv 6756 2405 6756 0 1 15:33 ? 00:00:00 sh -c /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6757 33 19 15:33 ? 00:00:21 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6758 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6759 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6760 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6761 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6763 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6764 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6765 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6767 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6768 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6769 28 19 15:33 ? 00:00:17 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6770 28 19 15:33 ? 00:00:17 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6771 27 19 15:33 ? 00:00:17 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6772 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6773 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6774 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6775 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6776 40 19 15:33 ? 00:00:25 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7
mythtv 6757 6756 6777 0 19 15:33 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 342 --verbose general --loglevel info --syslog local7


Same transcode with HTTPLiveStreamThreads set at 6 (cpu core got to 42 degrees).

downiej@masterbackend:/data$ ps -eLf | grep trans
mythtv 6810 2405 6810 0 1 15:38 ? 00:00:00 sh -c /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6811 38 22 15:38 ? 00:00:10 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6812 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6813 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6814 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6815 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6817 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6818 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6819 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6821 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6822 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6823 13 22 15:38 ? 00:00:03 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6824 13 22 15:38 ? 00:00:03 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6825 14 22 15:38 ? 00:00:03 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6826 14 22 15:38 ? 00:00:03 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6827 13 22 15:38 ? 00:00:03 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6828 14 22 15:38 ? 00:00:03 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6829 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6830 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6831 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6832 0 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6833 47 22 15:38 ? 00:00:12 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7
mythtv 6811 6810 6834 1 22 15:38 ? 00:00:00 /usr/bin/mythtranscode --hls --hlsstreamid 343 --verbose general --loglevel info --syslog local7

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On Mon, Oct 15, 2012 at 10:49 AM, Jim Downie
<Jim.Downie@sigmaseven.co.uk> wrote:
> I have disabled hyper threading (so only 6 cores available to Ubuntu 12.04) and am monitoring the performance through i7z.
>
>
> What this shows is that the CPU core frequencies predominantly stay low during HLS transcode (it can go up to 4.1GHz) and the thermal monitors show 40 degrees C (5 degree rise from no load) - it looks like it is using 30% of the available cpu's .

The HLS transcoding is done by mythtranscode. Since mythtranscode also
is a user job app, it could be that it's obeying the CPU load limit
setting in user job configuration screen of mythtv-setup. I seem to
recall 30% being the limit for the Low setting.


--
Ron Frazier
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 10/15/2012 10:49, Jim Downie wrote:
> What this shows is that the CPU core frequencies predominantly stay
> low during HLS transcode (it can go up to 4.1GHz) and the thermal
> monitors show 40 degrees C (5 degree rise from no load)

Actually, the full six-step turbo on the i7-3930K is only available when
four of the six cores are idle. Since the bus speed is 100MHz, six steps
raises it from 3.2GHz to only 3.8GHz. With all six cores active, it is
only capable of hitting 3.5GHz.

Something else to consider, what options are you encoding with? It could
very well be possible you are bottle-necked on single-threaded decoding
of the recording, and your encoders are just sitting there idle waiting
for data to operate on.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
> Something else to consider, what options are you encoding with? It could
> very well be possible you are bottle-necked on single-threaded decoding
> of the recording, and your encoders are just sitting there idle waiting
> for data to operate on.


Masterbackend is set to high cpu usage.


The transcode is part of the HLS stream and is converting a 1080i source (already fully written) to 704x400 size (not sure what other parameters are set on this transcode).


I have tried 2 HLS transcode simultaneously and the CPU load rises to 60 - 70% so the CPU is able to take the processing - its just that a single transcode cannot max out the CPU.

Any other ideas why this is only using 40% of the CPU for a single HLS transcode?

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 16/10/12 09:59, Jim Downie wrote:
>> Something else to consider, what options are you encoding with? It could
>> very well be possible you are bottle-necked on single-threaded decoding
>> of the recording, and your encoders are just sitting there idle waiting
>> for data to operate on.
>
>
> Masterbackend is set to high cpu usage.
>
>
> The transcode is part of the HLS stream and is converting a 1080i source (already fully written) to 704x400 size (not sure what other parameters are set on this transcode).
>
>
> I have tried 2 HLS transcode simultaneously and the CPU load rises to 60 - 70% so the CPU is able to take the processing - its just that a single transcode cannot max out the CPU.
>
> Any other ideas why this is only using 40% of the CPU for a single HLS transcode?
>

Not really, but can I ask what speeds you are getting? I recently
converted a 1080i recording from dvb-t2 to dvd format within
mytharchive, which uses ffmpeg. The transcoding step on my laptop took
about 6* real time. You have 3* cores and around 2* speed; do you get
much faster than real time?

Of course usually a similar product is available as the SD broadcast: I
just failed to record it :-)



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
> Not really, but can I ask what speeds you are getting? I recently
> converted a 1080i recording from dvb-t2 to dvd format within
> mytharchive, which uses ffmpeg. The transcoding step on my laptop took
> about 6* real time. You have 3* cores and around 2* speed; do you get
> much faster than real time?

What was your ffmpeg command line parameters and I will try it on one of my dvb-t2 recordings?

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 16/10/12 15:14, Jim Downie wrote:
>> Not really, but can I ask what speeds you are getting? I recently
>> converted a 1080i recording from dvb-t2 to dvd format within
>> mytharchive, which uses ffmpeg. The transcoding step on my laptop took
>> about 6* real time. You have 3* cores and around 2* speed; do you get
>> much faster than real time?
>
> What was your ffmpeg command line parameters and I will try it on one of my dvb-t2 recordings?
>

Here's a section of mythburn.log. The actual command starts "mythffmpeg
-threads 2" and ends "-map 0:2"

It's as generated. If I expected to use it routinely I would probably
change the audio recoding, but this worked. I'm afraid the progress.log
with the timings got deleted.

Input #0, mpegts, from '/home/john/Mythrecs/1054_20121010222600.mpg':
Duration: 00:58:50.24, start: 11020.436822, bitrate: 6978 kb/s
Stream #0.0[0x65]: Video: h264 (High), yuv420p, 1920x1080 [PAR 1:1
DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0.1[0x66](eng): Audio: aac_latm, 48000 Hz, stereo, s16
Stream #0.2[0x6a](eng): Audio: aac_latm, 48000 Hz, 2 channels (FC), s16
Stream #0.3[0x69](eng): Subtitle: dvbsub
Stream #0.4[0x6e]: Data: [0][0][0][0] / 0x0000
Stream #0.5[0x6f]: Data: [0][0][0][0] / 0x0000
Stream #0.6[0x82]: Data: [0][0][0][0] / 0x0000
/mnt/MythOnRoot/Mytharch_work/work/1/streaminfo.xml:-
<?xml version="1.0" ?>
<!DOCTYPE FILEINFO>
<file cutduration="3530" duration="3530"
filename="/home/john/Mythrecs/1054_20121010222600.mpg" type="mpegts">
<streams count="7">
<video aspectratio="1.77778" bitrate="0" codec="h264
(High)" ffmpegindex="0" fps="25" height="1080" id="101"
start_time="991.915108" streamindex="0" width="1920"/>
<audio bitrate="0" channels="2" codec="aac_latm"
ffmpegindex="1" id="102" language="eng" samplerate="48000"
start_time="991.839314" streamindex="1"/>
<audio bitrate="0" channels="2" codec="aac_latm"
ffmpegindex="2" id="106" language="eng" samplerate="48000"
start_time="991.867194" streamindex="2"/>
<subtitle codec="dvbsub" ffmpegindex="3" id="105"
language="eng" streamindex="3"/>
<data codec="Data: [0][0][0][0] / 0x0000"
streamindex="4"/>
<data codec="Data: [0][0][0][0] / 0x0000"
streamindex="5"/>
<data codec="Data: [0][0][0][0] / 0x0000"
streamindex="6"/>
</streams>
</file>
Preferred audio languages eng and und
Video id: 0x65, Audio1: [1] 0x66 (AAC_LATM, eng), Audio2: [2] - 0x6a
(AAC_LATM, eng)
Aspect ratio is 16:9
Re-encoding audio and video
Using encoder profiles from
/usr/share/mythtv/mytharchive/encoder_profiles/ffmpeg_dvd_pal.xml
Encoding profile (SP) found
mythffmpeg -threads 2 -v 1 -i
"/home/john/Mythrecs/1054_20121010222600.mpg" -r pal -target dvd -b
4771k -s 720x576 -acodec ac3 -ab 192k -ac 2 -copyts -aspect 16:9
"/mnt/MythOnRoot/Mytharch_work/work/1/newfile2.mpg" -acodec ac3 -ab 192k
-ac 2 -newaudio -map 0:0 -map 0:1 -map 0:2
FFmpeg version UNKNOWN, Copyright (c) 2000-2011 the FFmpeg developers
built on Sep 30 2012 03:05:58 with gcc 4.4.6 20120305 (Red Hat 4.4.6-4)
configuration: --compile-type=release --cpu=i686 --tune=i686
--enable-mmx --prefix=/usr --libdir-name=lib --x11-path=/usr/include
--enable-joystick-menu --enable-lirc --enable-ivtv --enable-firewire
--enable-dvb --enable-audio-oss --enable-audio-alsa --enable-audio-jack
--enable-x11 --enable-xrandr --enable-xv --enable-version3
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfftw3
--enable-opengl-video --enable-vdpau
--extra-cflags=-I/usr/include/nvidia-graphics
--extra-cxxflags=-I/usr/include/nvidia-graphics
--extra-libs=-L/usr/lib/nvidia-graphics --enable-crystalhd
--enable-vaapi --enable-ffmpeg-pthreads
--perl-config-opts='INSTALLDIRS=vendor' --enable-libmp3lame
--enable-libx264 --enable-libxvid --enable-libfaac --enable-libvpx
--enable-nonfree
libavutil 50. 39. 0 / 50. 39. 0
libavcodec 52.113. 1 / 52.113. 1
libavformat 52.101. 0 / 52.101. 0
libavdevice 52. 2. 3 / 52. 2. 3
libavfilter 1. 76. 0 / 1. 76. 0
libswscale 0. 12. 0 / 0. 12. 0
libpostproc 51. 2. 0 / 51. 2. 0
[dvbsub @ 0x8f74630] Invalid extradata, subtitle streams may be combined!
[NULL @ 0x8f74630] start time is not set in av_estimate_timings_from_pts
[NULL @ 0x8f779c0] start time is not set in av_estimate_timings_from_pts
[NULL @ 0x8f7ad40] start time is not set in av_estimate_timings_from_pts
[NULL @ 0x8f7e0c0] start time is not set in av_estimate_timings_from_pts
Input #0, mpegts, from '/home/john/Mythrecs/1054_20121010222600.mpg':
Duration: 00:58:50.24, start: 11020.436822, bitrate: 6978 kb/s
Stream #0.0[0x65]: Video: h264 (High), yuv420p, 1920x1080 [PAR 1:1
DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0.1[0x66](eng): Audio: aac_latm, 48000 Hz, stereo, s16
Stream #0.2[0x6a](eng): Audio: aac_latm, 48000 Hz, 2 channels (FC), s16
Stream #0.3[0x69](eng): Subtitle: dvbsub
Stream #0.4[0x6e]: Data: [0][0][0][0] / 0x0000
Stream #0.5[0x6f]: Data: [0][0][0][0] / 0x0000
Stream #0.6[0x82]: Data: [0][0][0][0] / 0x0000
Assuming PAL for target.
Incompatible sample format 's16' for codec 'ac3', auto-selecting format
'flt'
Last message repeated 1 times
[buffer @ 0x8f74050] w:1920 h:1080 pixfmt:yuv420p
[scale @ 0x8f5e0d0] w:1920 h:1080 fmt:yuv420p -> w:720 h:576 fmt:yuv420p
flags:0x4
[ac3 @ 0x8f5ee00] No channel layout specified. The encoder will guess
the layout, but it might be incorrect.
[ac3 @ 0x8f738d0] No channel layout specified. The encoder will guess
the layout, but it might be incorrect.
Output #0, dvd, to '/mnt/MythOnRoot/Mytharch_work/work/1/newfile2.mpg':
Metadata:
encoder : Lavf52.101.0
Stream #0.0: Video: mpeg2video, yuv420p, 720x576 [PAR 64:45 DAR
16:9], q=2-31, 4771 kb/s, 90k tbn, 25 tbc
Stream #0.1(eng): Audio: ac3, 48000 Hz, stereo, flt, 192 kb/s
Stream #0.2(eng): Audio: ac3, 48000 Hz, stereo, flt, 192 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Stream #0.2 -> #0.2
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 16/10/12 15:44, John Pilkington wrote:
> On 16/10/12 15:14, Jim Downie wrote:
>>> Not really, but can I ask what speeds you are getting? I recently
>>> converted a 1080i recording from dvb-t2 to dvd format within
>>> mytharchive, which uses ffmpeg. The transcoding step on my laptop took
>>> about 6* real time. You have 3* cores and around 2* speed; do you get
>>> much faster than real time?
>>
>> What was your ffmpeg command line parameters and I will try it on one
>> of my dvb-t2 recordings?

mythffmpeg -threads 2 -v 1 -i
"/home/john/Mythrecs/1054_20121010222600.mpg" -r pal -target dvd -b
4771k -s 720x576 -acodec ac3 -ab 192k -ac 2 -copyts -aspect 16:9
"/mnt/MythOnRoot/Mytharch_work/work/1/newfile2.mpg" -acodec ac3 -ab 192k
-ac 2 -newaudio -map 0:0 -map 0:1 -map 0:2


I just noticed that I cut the log just before this line, which may
answer your original question.

[h264 @ 0x8f5e470] Cannot parallelize deblocking type 1, decoding such
frames in sequential order
frame= 21 fps= 0 q=2.0 size= 124kB time=0.19
bitrate=5290.7kbits/s dup=20 drop=0
frame= 25 fps= 23 q=2.0 size= 124kB time=0.19
bitrate=5290.7kbits/s dup=21 drop=0
frame= 29 fps= 17 q=3.0 size= 180kB time=0.42
bitrate=3544.6kbits/s dup=21 drop=0

etc.

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
Tried ffmpeg on one of my recordings with those settings and managed to get 300fps (9 times normal speed).

It looks like it is the target container that is used is the determining factor if multiple threads are used. If I use libx264 then the frame rate goes down but my CPU is now using 90%.

Not sure if this helps.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
* On Mon Oct 15, 2012 at 11:36:42AM -0400, Ronald Frazier wrote:
> The HLS transcoding is done by mythtranscode. Since mythtranscode also
> is a user job app, it could be that it's obeying the CPU load limit
> setting in user job configuration screen of mythtv-setup. I seem to
> recall 30% being the limit for the Low setting.

The CPU load limitations for mythtranscode are applied by the JobQueue
directly, not from within mythtranscode itself. If you run mythtranscode
from the command line, or it is run for a HLS encode, it runs at full
speed.

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
* On Mon Oct 15, 2012 at 12:24:57PM -0400, Raymond Wagner wrote:
> Something else to consider, what options are you encoding with? It could
> very well be possible you are bottle-necked on single-threaded decoding
> of the recording, and your encoders are just sitting there idle waiting
> for data to operate on.

This would be my first guess. I put the decoding in its own thread, but
it's still fairly easy to starve the encoders since the lib264 is
configured to use up to 8 slices if the HTTPLiveStreamThreads setting is
set up to 8.

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
* On Tue Oct 16, 2012 at 09:59:30AM +0100, Jim Downie wrote:
> I have tried 2 HLS transcode simultaneously and the CPU load rises to
> 60 - 70% so the CPU is able to take the processing - its just that a
> single transcode cannot max out the CPU.
>
> Any other ideas why this is only using 40% of the CPU for a single
> HLS transcode?

I agree with Raymond's assessment in another reply that you may be
running into a limit due to the single threaded decoder. The encoder
will use up to 8 threads if you set the HTTPLiveStreamThreads setting,
with the default being 2 threads. I have the code set to use 8 slices
in avformatwriter.cpp which limits the thread usage to 8 even if you
set HTTPLiveStreamThreads higher.

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
> I agree with Raymond's assessment in another reply that you may be
> running into a limit due to the single threaded decoder. The encoder
> will use up to 8 threads if you set the HTTPLiveStreamThreads setting,
> with the default being 2 threads. I have the code set to use 8 slices
> in avformatwriter.cpp which limits the thread usage to 8 even if you
> set HTTPLiveStreamThreads higher.

Can this limit be increased in a future update to use more slices (if its being controlled through the HTTPLiveStreamThreads setting)?
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
On 10/17/2012 02:31, Jim Downie wrote:
>> I agree with Raymond's assessment in another reply that you may be
>> running into a limit due to the single threaded decoder. The
>> encoder will use up to 8 threads if you set the
>> HTTPLiveStreamThreads setting, with the default being 2 threads. I
>> have the code set to use 8 slices in avformatwriter.cpp which
>> limits the thread usage to 8 even if you set HTTPLiveStreamThreads
>> higher.
>
> Can this limit be increased in a future update to use more slices (if
> its being controlled through the HTTPLiveStreamThreads setting)?

It can be increased if there is value to doing so, which at this point
is limited to multi-socket servers with 12+ cores. Assuming your issue
is a decoding bottleneck, the only thing that would help would be a
future version of MythTV switching to ffmpeg-mt, which performs
pipelined decoding of multiple successive frames simultaneously.
Alternatively, you could use more intensive encoding parameters, such
that the encoding threads are actually doing something other than
sitting idle waiting for frames from the decoder.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: Cannot max out CPU usage on HLS transcode [ In reply to ]
Hi

On Tuesday, 16 October 2012, Jim Downie wrote:

>
>
> I have tried 2 HLS transcode simultaneously and the CPU load rises to 60 -
> 70% so the CPU is able to take the processing - its just that a single
> transcode cannot max out the CPU.
>
> Any other ideas why this is only using 40% of the CPU for a single HLS
> transcode?
>
>
I believe the default encoding h264 option are to use a maximum of two
threads per encoding. So with anything with more than 2 cores will not see
maxed out CPU..