Mailing List Archive

Radio Times XML Feed
Hi ,

With the impending launch of ITV Be channel in the UK , for the WAF I need
to get myself some guide data for this channel. Yes , Im aware the content
of this channel may be terrible.

I'm currently using the radio times XML feed (tv_grab_uk_rt) - but I dont
think this new channel has any data supplied by this method. Running

tv_grab_uk_rt --list-channels

and it doesn't show.

Please could someone point me in the right direction? Im guessing I need to
change data source or wait for RT to update.

Thanks in advance.
Re: Radio Times XML Feed [ In reply to ]
On 10/08/2014 06:16 PM, Adam Skinner wrote:
> Hi ,
>
> With the impending launch of ITV Be channel in the UK , for the WAF I
> need to get myself some guide data for this channel. Yes , Im aware the
> content of this channel may be terrible.
>
> I'm currently using the radio times XML feed (tv_grab_uk_rt) - but I
> dont think this new channel has any data supplied by this method. Running
>
> tv_grab_uk_rt --list-channels
>
> and it doesn't show.
>
> Please could someone point me in the right direction? Im guessing I need
> to change data source or wait for RT to update.

Adam,

You do realise that the Radio Times grabber is being deprecated and will
stop working at some point, right? So given that you'll need to make
changes anyway, now would be a good time to switch to another grabber.

The problem is not with the grabber, but rather (as I understand it)
that the provider of the guide data has implemented a new way of making
it available and will stop making it available in the way that the _rt
grabber uses.

I migrated to the Atlas grabber a little while ago. Whereas it takes a
bit of time to set up (probably less if you actually live in the UK
which is of course its primary target audience), it does work very well.
You do need to obtain an API key from Metabroadcast, but this is free of
charge and not a very high hurdle.

https://github.com/honir/tv_grab_uk_atlas
http://wiki.xmltv.org/index.php/Britain_and_Ireland

HTH, Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
Quoting Adam Skinner <kingmoffa@gmail.com>:

> Hi ,
>
> With the impending launch of ITV Be channel in the UK , for the WAF I need
> to get myself some guide data for this channel. Yes , Im aware the content
> of this channel may be terrible.
>
> I'm currently using the radio times XML feed (tv_grab_uk_rt) - but I dont
> think this new channel has any data supplied by this method. Running
>
> tv_grab_uk_rt --list-channels
>
> and it doesn't show.
>
> Please could someone point me in the right direction? Im guessing I need to
> change data source or wait for RT to update.
>
> Thanks in advance.

You need the atlas grabber. It already has program listings for ITVBe
and it is terrible. You don't need a guide for this type of channel as
you are meant to switch your brain off before watching.

Cheers,
Tim.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On 08/10/14 17:34, mythtv@phipps-hutton.freeserve.co.uk wrote:
>
> Quoting Adam Skinner <kingmoffa@gmail.com>:
>
>> Hi ,
>>
>> With the impending launch of ITV Be channel in the UK , for the WAF I
>> need
>> to get myself some guide data for this channel. Yes , Im aware the
>> content
>> of this channel may be terrible.
>>
>> I'm currently using the radio times XML feed (tv_grab_uk_rt) - but
>> I dont
>> think this new channel has any data supplied by this method. Running
>>
>> tv_grab_uk_rt --list-channels
>>
>> and it doesn't show.
>>
>> Please could someone point me in the right direction? Im guessing I
>> need to
>> change data source or wait for RT to update.
>>
>> Thanks in advance.
>
> You need the atlas grabber. It already has program listings for ITVBe
> and it is terrible. You don't need a guide for this type of channel as
> you are meant to switch your brain off before watching.
>
> Cheers,
> Tim.
>

The RT feed will be depreciated on 1st November and shutdown in January
2015. There is still a possibility the radio times grabber will be
updated to use the new API so the switch would be seemless but until
then to get data for the new channels you have to use the atlas grabber.

I switched recently to use the Atlas grabber which seems to be working OK.

These are few (possibly incomplete) notes I made that hopefully will
help anyone to try the new grabber.

1. get a key - click 'get your free api key' on
http://atlas.metabroadcast.com/#apiExplorer (you need a twitter, google
or github login)
2. create app once logged in - the web site often fails so be patient
pressing back often works
3. get latest stable xmltv 0.5.65 or later
4. start mythtv-setup
5. Select a Video Source and change the grabber to 'UK - Atlas
(atla.metabroadcast.com) (xmltv)'
6. Press the configure button and follow the instructions - this will
ask for your key and what region/channels you want and then create the
config file the grabber will use to get the correct channels for the source
7. Some of the id's may need changing in the channel editor for example
all the itv1

bbc2.bbc.co.uk -> england.bbc2.bbc.co.uk // or use one of the
other regions scotland/wales etc
granada.itv1.itv.co.uk -> granada.itv1.itv.com // same is true for all
the itv1 regions

8. If you want to test you can remove all the existing programs from the
program table by running this query in mysql

truncate program;

9. Now run mythfilldatabase to grab the new data

10. mythfilldatabase will appear to fail with an error when using the
early versions of the Atlas grabber.
That's because the grabber return 1 to indicate an error because it
can't find a supplemental file
tv_grab_uk_atlas/tv_grab_uk_atlas.user.map.conf. Later versions ignore
the error.

11. the atlas grabber doesn't support "allatonce" like the old radio
times grabber so to get similar behaviour you need to add the '--refresh
0-99' to mythfilldatabase

Paul H.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On 10/08/2014 07:30 PM, Paul Harrison wrote:
> These are few (possibly incomplete) notes I made that hopefully will
> help anyone to try the new grabber.

(...)

Another step I would recommend is to add '--days 14' to the
tv_grab_uk_atlas command line, because without it (despite the xmltv
documentation stating otherwise) the grabber will not grab all available
data without it.

HTH, Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On 08.10.2014 20:09, Jan Ceuleers wrote:
> On 10/08/2014 07:30 PM, Paul Harrison wrote:
>> These are few (possibly incomplete) notes I made that hopefully will
>> help anyone to try the new grabber.
>
> (...)
>
> Another step I would recommend is to add '--days 14' to the
> tv_grab_uk_atlas command line, because without it (despite the xmltv
> documentation stating otherwise) the grabber will not grab all available
> data without it.

That is unrelated.

If you add --refresh 0-99 to the mythfilldatabase args in setup and let
mythtv handle the invocation, then mfdb will take care of calling
tv_grab_uk_atlas once for each day from now until it doesn't get any
data anymore.

Bonus: Should PA extend their guide to 21 or 28 days you have nothing
to change later.

Regards,
Karl
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On 10/08/2014 08:15 PM, Karl Dietz wrote:
>> Another step I would recommend is to add '--days 14' to the
>> tv_grab_uk_atlas command line, because without it (despite the xmltv
>> documentation stating otherwise) the grabber will not grab all available
>> data without it.
>
> That is unrelated.
>
> If you add --refresh 0-99 to the mythfilldatabase args in setup and let
> mythtv handle the invocation, then mfdb will take care of calling
> tv_grab_uk_atlas once for each day from now until it doesn't get any
> data anymore.
>
> Bonus: Should PA extend their guide to 21 or 28 days you have nothing
> to change later.

OK, but your "if" does not hold in general. It does not hold for me, for
example, because I need to use multiple grabbers and data sources for
each video source I have (and I don't like the complexity of
tv_grab_combiner).

So I run my multiple tv_grab_whatevers in parallel, then feed the
results to mythfilldatabase --file for each relevant --sourceid.

I would therefore like grabbers to actually implement the XMLTV spec and
grab all available data without having to specify the --days explicitly.
This is a bug.

Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On 10/08/2014 08:15 PM, Karl Dietz wrote:
>> Another step I would recommend is to add '--days 14' to the
>> tv_grab_uk_atlas command line, because without it (despite the xmltv
>> documentation stating otherwise) the grabber will not grab all available
>> data without it.
>
> That is unrelated.
>
> If you add --refresh 0-99 to the mythfilldatabase args in setup and let
> mythtv handle the invocation, then mfdb will take care of calling
> tv_grab_uk_atlas once for each day from now until it doesn't get any
> data anymore.
>
> Bonus: Should PA extend their guide to 21 or 28 days you have nothing
> to change later.

OK, but your "if" does not hold in general. It does not hold for me, for
example, because I need to use multiple grabbers and data sources for
each video source I have (and I don't like the complexity of
tv_grab_combiner).

So I run my multiple tv_grab_whatevers in parallel, then feed the
results to mythfilldatabase --file for each relevant --sourceid.

I would therefore like grabbers to actually implement the XMLTV spec and
grab all available data without having to specify the --days explicitly.
This is a bug.

Jan


Thanks for the replies and most importantly instructions , I will give the
Atlas grabber a go. Its nice to have a few different options :)
Re: Radio Times XML Feed [ In reply to ]
On 08/10/2014 18:30, Paul Harrison wrote:
> The RT feed will be depreciated on 1st November and shutdown in
> January 2015. There is still a possibility the radio times grabber
> will be updated to use the new API so the switch would be seemless but
> until then to get data for the new channels you have to use the atlas
> grabber.
>
> I switched recently to use the Atlas grabber which seems to be
> working OK.
>
> These are few (possibly incomplete) notes I made that hopefully will
> help anyone to try the new grabber.
>
> 1. get a key - click 'get your free api key' on
> http://atlas.metabroadcast.com/#apiExplorer (you need a twitter,
> google or github login)
> 2. create app once logged in - the web site often fails so be patient
> pressing back often works
> 3. get latest stable xmltv 0.5.65 or later
> 4. start mythtv-setup
> 5. Select a Video Source and change the grabber to 'UK - Atlas
> (atla.metabroadcast.com) (xmltv)'
> 6. Press the configure button and follow the instructions - this will
> ask for your key and what region/channels you want and then create the
> config file the grabber will use to get the correct channels for the
> source
> 7. Some of the id's may need changing in the channel editor for
> example all the itv1
>
> bbc2.bbc.co.uk -> england.bbc2.bbc.co.uk // or use one of the
> other regions scotland/wales etc
> granada.itv1.itv.co.uk -> granada.itv1.itv.com // same is true for
> all the itv1 regions
>
> 8. If you want to test you can remove all the existing programs from
> the program table by running this query in mysql
>
> truncate program;
>
> 9. Now run mythfilldatabase to grab the new data
>
> 10. mythfilldatabase will appear to fail with an error when using the
> early versions of the Atlas grabber.
> That's because the grabber return 1 to indicate an error because it
> can't find a supplemental file
> tv_grab_uk_atlas/tv_grab_uk_atlas.user.map.conf. Later versions ignore
> the error.
>
> 11. the atlas grabber doesn't support "allatonce" like the old radio
> times grabber so to get similar behaviour you need to add the
> '--refresh 0-99' to mythfilldatabase
>
> Paul H.

Thanks for the notes. I have not yet made the change from RT to Atlas
and have some questions:

1, Does duplicate matching work OK? Or will I find previous episodes
that I've already watched being re-recorded? I guess this comes down to
whether the program IDs match, but I've never really understood where
those IDs come from.

2, What about radio programs - I understand the Atlas grabber supports
radio stations too, which I currently use EIT to populate. If I move
radio stations to Atlas, will duplicate matching work for those too?

3, I have a hand-crafted configuration file for the RT grabber, with
lines like:

channel south.bbc1.bbc.co.uk
channel south.bbc2.bbc.co.uk

with just the channels I want. Can I just copy the same file for the
Atlas grabber? I thought I read somewhere that the Atlas grabber uses
completely different channel IDs, four digit numbers I think. I don't
mind setting up a new config file if necessary, just wondering.

Cheers,

John

--
John Veness, MythTV user, UK, DVB-T

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
Quoting John Veness <John.Veness.mythtv@pelago.org.uk>:

> 1, Does duplicate matching work OK? Or will I find previous episodes
> that I've already watched being re-recorded? I guess this comes down
> to whether the program IDs match, but I've never really understood
> where those IDs come from.
>
> 2, What about radio programs - I understand the Atlas grabber
> supports radio stations too, which I currently use EIT to populate.
> If I move radio stations to Atlas, will duplicate matching work for
> those too?

ProgramIDs are set if the listing source supplies series and episode
information. The programid is a hash of the tiles + ascii-encoded
series and episode. So whether duplicate matching works depends on if
the RT data has that information and if the Atlas does. Duplicate
matching will break if a listing source changes whether it has
series/episode or not.

Another point is that the RT grabber has an extensive set of title
fix-ups (things like removing 'New: ' from titles etc.) and the Atlas
grabber does not. That will affect programids and duplicate matching
so you should expect so duplicates.

I don't know what will happen when the RT feed is switched off. It
would be a shame to lose that fix-up knowledge so hopefully either it
will move into the xmltv atlas grabber or the RT grabber will
assimilate the atlas feed.

The Atlas radio data is a good as EIT if not better but again
duplicate matching may fail if the titles etc are not exactly as in
previous recordings


> 3, I have a hand-crafted configuration file for the RT grabber, with
> lines like:
>
> channel south.bbc1.bbc.co.uk
> channel south.bbc2.bbc.co.uk
>
> with just the channels I want. Can I just copy the same file for the
> Atlas grabber? I thought I read somewhere that the Atlas grabber
> uses completely different channel IDs, four digit numbers I think. I
> don't mind setting up a new config file if necessary, just wondering.

Atlas does us weird strings for the channel id by default but you can
use human readable ones if you want. They may not be the same as the
RT grabber so best to check each one in the file
~/.xmltv/supplement/tv_grab_uk_atlas/tv_grab_uk_atlas.map.channels.conf

Run 'tv_grab_uk_atlas --list-channels | less' and look for ITVBe, it
isn't in the RT listing so it has the weird name 'cbYy', channels that
are in the R listings have the 'normal' names.

Cheers,
Tim.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On Thu, 09 Oct 2014 11:54:26 +0100, <mythtv@phipps-hutton.freeserve.co.uk>
wrote:

>
> Quoting John Veness <John.Veness.mythtv@pelago.org.uk>:
>
>> 1, Does duplicate matching work OK? Or will I find previous episodes
>> that I've already watched being re-recorded? I guess this comes down to
>> whether the program IDs match, but I've never really understood where
>> those IDs come from.
>>
>> 2, What about radio programs - I understand the Atlas grabber supports
>> radio stations too, which I currently use EIT to populate. If I move
>> radio stations to Atlas, will duplicate matching work for those too?
>
> ProgramIDs are set if the listing source supplies series and episode
> information. The programid is a hash of the tiles + ascii-encoded series
> and episode. So whether duplicate matching works depends on if the RT
> data has that information and if the Atlas does. Duplicate matching
> will break if a listing source changes whether it has series/episode or
> not.
>
> Another point is that the RT grabber has an extensive set of title
> fix-ups (things like removing 'New: ' from titles etc.) and the Atlas
> grabber does not. That will affect programids and duplicate matching so
> you should expect so duplicates.
>

Not sure what you're saying here.
Duplicate matching will work if they both supply the same category, title,
season & episode. And the ultimate source for both is the Press
Association so generally it should work. Apart from the fix-ups....

> I don't know what will happen when the RT feed is switched off. It would
> be a shame to lose that fix-up knowledge so hopefully either it will
> move into the xmltv atlas grabber or the RT grabber will assimilate the
> atlas feed.
>
> The Atlas radio data is a good as EIT if not better but again duplicate
> matching may fail if the titles etc are not exactly as in previous
> recordings
>

The EIT program numbers are completely different so your radio recording
history will effectively be reset. Atlas supplies season, episode, total
episodes for most programmes but there are still a few R4 series that are
totally unmarked - "More or Less" is one example I noticed.

0.28 now picks up episode & total episodes from EIT (it provides seasons
for very few programmes and not for any R4 radio that I've noticed). So
you may prefer to stick with EIT for radio.

>
>> 3, I have a hand-crafted configuration file for the RT grabber, with
>> lines like:
>>
>> channel south.bbc1.bbc.co.uk
>> channel south.bbc2.bbc.co.uk
>>
>> with just the channels I want. Can I just copy the same file for the
>> Atlas grabber? I thought I read somewhere that the Atlas grabber uses
>> completely different channel IDs, four digit numbers I think. I don't
>> mind setting up a new config file if necessary, just wondering.
>

Atlas uses a form of line-ups so you should find it uses your correct
region automatically.


> Atlas does us weird strings for the channel id by default but you can
> use human readable ones if you want. They may not be the same as the RT
> grabber so best to check each one in the file
> ~/.xmltv/supplement/tv_grab_uk_atlas/tv_grab_uk_atlas.map.channels.conf
>
> Run 'tv_grab_uk_atlas --list-channels | less' and look for ITVBe, it
> isn't in the RT listing so it has the weird name 'cbYy', channels that
> are in the R listings have the 'normal' names.
>

Is this because ITVBe isn't in the supplemental mappings ? I thought they
were just a snapshot packaged with the grabber, so won't have the mapping
for any new channels. I assumed this was to satisy the XMLTV spec. You
could try adding a user mapping.

> Cheers,
> Tim.
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
Paul Harrison <mythtv@sky.com> wrote:

> 2. create app once logged in - the web site often fails so be patient pressing back often works

Any hints what one is expected to enter for Application title, Application URL, and Application description ?
At http://metabroadcast.com/blog/create-and-manage-your-atlas-api-key it just says "enter the name of your application"

I'm guessing that these should be "tv_grab_uk_atlas", and "https://github.com/honir/tv_grab_uk_atlas" ? But I don't want to mess something up since the page is "Create new application" and I'm not creating an application - just trying to get a key to use an existing application.

Simon Hobson


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
On 12/10/14 10:00, Simon Hobson wrote:
> Paul Harrison <mythtv@sky.com> wrote:
>
>> 2. create app once logged in - the web site often fails so be patient pressing back often works
> Any hints what one is expected to enter for Application title, Application URL, and Application description ?
> At http://metabroadcast.com/blog/create-and-manage-your-atlas-api-key it just says "enter the name of your application"
>
> I'm guessing that these should be "tv_grab_uk_atlas", and "https://github.com/honir/tv_grab_uk_atlas" ? But I don't want to mess something up since the page is "Create new application" and I'm not creating an application - just trying to get a key to use an existing application.
>
> Simon Hobson
>
>

I don't think it really matters you can create any number of apps. I
just put something like 'MythTV', 'www.mythtv.org', 'DVR'.

Probably should have been more descriptive with the description but I
was eager to test the grabber :)

Paul H.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Radio Times XML Feed [ In reply to ]
The atlas feed is pretty good, using it with Freeguide 0.11, wish they kept
developing it though as it's a bit sparse.
don't think i'll renew my digiguide sub when it runs out.

http://www.artificialworlds.net/freeguide/Main/HomePage

got it set up for both linux and windows, had to update xmltv in windows to
get atlas support though, version installed with freeguide was too old.


--
Lee Donaghy.