Mailing List Archive

1 2 3 4 5 6 7 8  View All
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/11/2014 7:53 PM, Robert Eden wrote:
> On 10/11/2014 9:39 PM, Bill Meek wrote:
>> On 10/11/2014 09:34 PM, Robert Eden wrote:
>>> On 10/11/2014 8:19 PM, Yeechang Lee wrote:
>>>> Robert Eden <rmeden@gmail.com> says:
>>>>> If you need to edit your hosts file, use this address: 54.85.117.227
>>>>> That's a floating IP currently assigned to the server.
>>>> So is this IP preferable to whatever dd.schedulesdirect.org points to
>>>> at a given moment?
>>>>
>>> No. dd.schedulesdirect.org is preferred. I'll point that where it
>>> needs to go.
>>
>> But if Yeechang means what address is preferred for the /etc/hosts
>> solution, then 54.85.117.227 is correct. At least that's what I
>> updated the Wiki to say.
>>
> Yes, he (and you) are right then.
>
> 1. Use dd.schedulesdirect.org if you can use a hostname.
> 2. Use 54.85.117.227 if you have to hard code an address in /etc/hosts.
>
> Robert
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>
I seem to be having problems with the SD listings. When I run MFDB, I
see the following:

Resolving webservices.schedulesdirect.tmsdatadirect.com... 54.85.117.227
Connecting to
webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80...
connected.
HTTP request sent, awaiting response... 401 Authorization Required
Connecting to
webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80...
connected.
HTTP request sent, awaiting response... 402 Account Expired
2014-10-17 10:19:26 ERROR 402: Account Expired.

2014-10-17 10:19:26.981 DataDirect, Error: Failed to save DD cache!
redownloading data...
--2014-10-17 10:19:26--
http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService

My account is NOT expired (confirmed on the SD website). I have not
seen this happen with the previous service. Looks like something odd
going on.

Later in the same MFDB run, I see:

2014-10-17 10:25:37.828 Main temp tables populated.
2014-10-17 10:25:37.829 Did not find any new program data.
2014-10-17 10:25:37.865 Failed to fetch some program info
2014-10-17 10:25:37.865 Adjusting program database end times.
2014-10-17 10:25:38.888 0 replacements made
2014-10-17 10:25:38.888 Marking generic episodes.
2014-10-17 10:25:39.456 Found 14949
2014-10-17 10:25:39.457 Marking repeats.
2014-10-17 10:25:40.432 Found 41959
2014-10-17 10:25:40.432 Unmarking new episode rebroadcast repeats.
2014-10-17 10:25:40.863 Found 0
2014-10-17 10:25:41.538 Marking episode first showings.
2014-10-17 10:25:43.847 Found 10595
2014-10-17 10:25:43.847 Marking episode last showings.
2014-10-17 10:25:46.272 Found 10595
2014-10-17 10:25:46.286

The "Failed to fetch some program info" I think is a result of some of
the data fetches reporting Account Expired.
Jay
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/17/2014 12:29 PM, Jay Foster wrote:
...
> I seem to be having problems with the SD listings. When I run MFDB, I see the following:
>
> Resolving webservices.schedulesdirect.tmsdatadirect.com... 54.85.117.227
> Connecting to webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80... connected.
> HTTP request sent, awaiting response... 401 Authorization Required
> Connecting to webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80... connected.
> HTTP request sent, awaiting response... 402 Account Expired
> 2014-10-17 10:19:26 ERROR 402: Account Expired.
>
> 2014-10-17 10:19:26.981 DataDirect, Error: Failed to save DD cache! redownloading data...
> --2014-10-17 10:19:26-- http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService
>
> My account is NOT expired (confirmed on the SD website). I have not seen this happen with the previous service. Looks like something odd going
> on.
...

I believe you were/are running 0.21.

Note that in 0.25 (Apr. 2012), MythTV began using the internal
MythDownloadManager. It no longer uses the wget command to
download SD files (and many others.) There's also discussions
on the SD forum about similar problems:

http://forums.schedulesdirect.org/viewtopic.php?f=8&t=2599&start=20#p7888

Maybe trying: man wget and searching for --http-passwd will explain
what type of authentication is supported. My wget uses --http-password
and will encode the response in either digest or basic format depending
on the challenge issued. Currently, the report is that only basic is
being used.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/17/2014 08:43 PM, Bill Meek wrote:
> On 10/17/2014 12:29 PM, Jay Foster wrote:
> ...
> > I seem to be having problems with the SD listings. When I run MFDB,
> I see the following:
>>
>> Resolving webservices.schedulesdirect.tmsdatadirect.com... 54.85.117.227
>> Connecting to
>> webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80...
>> connected.
>> HTTP request sent, awaiting response... 401 Authorization Required
>> Connecting to
>> webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80...
>> connected.
>> HTTP request sent, awaiting response... 402 Account Expired
>> 2014-10-17 10:19:26 ERROR 402: Account Expired.
>>
>> 2014-10-17 10:19:26.981 DataDirect, Error: Failed to save DD cache!
>> redownloading data...
>> --2014-10-17 10:19:26--
>> http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService
>>
>> My account is NOT expired (confirmed on the SD website). I have not
>> seen this happen with the previous service. Looks like something odd
>> going
>> on.
> ...
>
> I believe you were/are running 0.21.
>
> Note that in 0.25 (Apr. 2012), MythTV began using the internal
> MythDownloadManager. It no longer uses the wget command to
> download SD files (and many others.) There's also discussions
> on the SD forum about similar problems:
>
> http://forums.schedulesdirect.org/viewtopic.php?f=8&t=2599&start=20#p7888
>
> Maybe trying: man wget and searching for --http-passwd will explain
> what type of authentication is supported. My wget uses --http-password
> and will encode the response in either digest or basic format depending
> on the challenge issued. Currently, the report is that only basic is
> being used.
>
I think I was not clear (my bad). I am not ALWAYS getting 402 Account
Expired, just sometimes within the same MFDB run. Most MFDB runs
authenticate each connection successfully.

I am sometimes seeing a different issue too. Most MFDB connections
fetch about 200K bytes of data. For example:

HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/xml]
Saving to: `STDOUT'

0K .......... .......... .......... .......... .......... 120K
50K .......... .......... .......... .......... .......... 135K
100K .......... .......... .......... .......... .......... 39.9K
150K .......... .......... .......... .......... .......... 96.8K
200K .......... .......... ...... 92.1K=2.8s

2014-10-14 02:33:18 (79.6 KB/s) - `-' saved [231515]

2014-10-14 02:33:18.089 DataDirect: Your subscription expires on
03/22/2015 01:59:05 PM
2014-10-14 02:33:20.477 Grab complete. Actual data from Sat Oct 25
07:00:00 2014 to Sun Oct 26 07:00:00 2014 (UTC)

I have seen the following 500 Internal Server Error too:

HTTP request sent, awaiting response... 401 Authorization Required
Connecting to
webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80...
connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2014-10-14 02:33:26 ERROR 500: Internal Server Error.

2014-10-14 02:33:26.547 DataDirect, Error: Failed to save DD cache!
redownloading data...
--2014-10-14 02:33:26--
http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService

But sometimes (usually the last connection?) it gets something strange,
such as:

HTTP request sent, awaiting response... 200 OK
Length: 5022 (4.9K) [text/xml]
Saving to: `STDOUT'

0K .... 100% 343M=0s

2014-10-14 02:33:35 (343 MB/s) - `-' saved [5022/5022]

2014-10-14 02:33:35.114 DataDirect: Your subscription expires on
03/22/2015 01:59:05 PM
2014-10-14 02:33:35.158 Grab complete. Actual data from Mon Oct 27
07:00:00 2014 to Tue Oct 28 07:00:00 2014 (UTC)
2014-10-14 02:33:35.172 Main temp tables populated.
2014-10-14 02:33:35.173 Did not find any new program data.

This may also be a reason for the "Failed to fetch some program info"
message at the end. Note that this download HAS a reported Length field
(5022) in the HTTP header (seems a bit short for listings data).

I see that most of the HTTP responses do not have a Length field in the
header (Length: unspecified [text/xml]). I seem to recall RobertE
stating that he needed to gather the listings data together first in
order to fill in the Length header, and that is why there was an
additional delay with the replacement service. Has that changed? I AM
getting listings data okay, or as far as I can tell. Just pointing out
some anomalies going on behind the scenes in the logs.

Jay
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/18/2014 3:43 PM, Jay Foster wrote:
> On 10/17/2014 08:43 PM, Bill Meek wrote:
>> On 10/17/2014 12:29 PM, Jay Foster wrote:
>> ...
>> > I seem to be having problems with the SD listings. When I run MFDB, I see the following:
>>>
>>> Resolving webservices.schedulesdirect.tmsdatadirect.com... 54.85.117.227
>>> Connecting to webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80... connected.
>>> HTTP request sent, awaiting response... 401 Authorization Required
>>> Connecting to webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80... connected.
>>> HTTP request sent, awaiting response... 402 Account Expired
>>> 2014-10-17 10:19:26 ERROR 402: Account Expired.
>>>
>>> 2014-10-17 10:19:26.981 DataDirect, Error: Failed to save DD cache! redownloading data...
>>> --2014-10-17 10:19:26-- http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService
>>>
>>> My account is NOT expired (confirmed on the SD website). I have not seen this happen with the previous service. Looks like something odd going
>>> on.
>> ...
>>
>> I believe you were/are running 0.21.
>>
>> Note that in 0.25 (Apr. 2012), MythTV began using the internal
>> MythDownloadManager. It no longer uses the wget command to
>> download SD files (and many others.) There's also discussions
>> on the SD forum about similar problems:
>>
>> http://forums.schedulesdirect.org/viewtopic.php?f=8&t=2599&start=20#p7888
>>
>> Maybe trying: man wget and searching for --http-passwd will explain
>> what type of authentication is supported. My wget uses --http-password
>> and will encode the response in either digest or basic format depending
>> on the challenge issued. Currently, the report is that only basic is
>> being used.
>>
> I think I was not clear (my bad). I am not ALWAYS getting 402 Account Expired, just sometimes within the same MFDB run. Most MFDB runs
> authenticate each connection successfully.
>
> I am sometimes seeing a different issue too. Most MFDB connections fetch about 200K bytes of data. For example:
>
> HTTP request sent, awaiting response... 200 OK
> Length: unspecified [text/xml]
> Saving to: `STDOUT'
>
> 0K .......... .......... .......... .......... .......... 120K
> 50K .......... .......... .......... .......... .......... 135K
> 100K .......... .......... .......... .......... .......... 39.9K
> 150K .......... .......... .......... .......... .......... 96.8K
> 200K .......... .......... ...... 92.1K=2.8s
>
> 2014-10-14 02:33:18 (79.6 KB/s) - `-' saved [231515]
>
> 2014-10-14 02:33:18.089 DataDirect: Your subscription expires on 03/22/2015 01:59:05 PM
> 2014-10-14 02:33:20.477 Grab complete. Actual data from Sat Oct 25 07:00:00 2014 to Sun Oct 26 07:00:00 2014 (UTC)
>
> I have seen the following 500 Internal Server Error too:
>
> HTTP request sent, awaiting response... 401 Authorization Required
> Connecting to webservices.schedulesdirect.tmsdatadirect.com|54.85.117.227|:80... connected.
> HTTP request sent, awaiting response... 500 Internal Server Error
> 2014-10-14 02:33:26 ERROR 500: Internal Server Error.
>
> 2014-10-14 02:33:26.547 DataDirect, Error: Failed to save DD cache! redownloading data...
> --2014-10-14 02:33:26-- http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService
>
> But sometimes (usually the last connection?) it gets something strange, such as:
>
> HTTP request sent, awaiting response... 200 OK
> Length: 5022 (4.9K) [text/xml]
> Saving to: `STDOUT'
>
> 0K .... 100% 343M=0s
>
> 2014-10-14 02:33:35 (343 MB/s) - `-' saved [5022/5022]
>
> 2014-10-14 02:33:35.114 DataDirect: Your subscription expires on 03/22/2015 01:59:05 PM
> 2014-10-14 02:33:35.158 Grab complete. Actual data from Mon Oct 27 07:00:00 2014 to Tue Oct 28 07:00:00 2014 (UTC)
> 2014-10-14 02:33:35.172 Main temp tables populated.
> 2014-10-14 02:33:35.173 Did not find any new program data.
>
> This may also be a reason for the "Failed to fetch some program info" message at the end. Note that this download HAS a reported Length field
> (5022) in the HTTP header (seems a bit short for listings data).
>
> I see that most of the HTTP responses do not have a Length field in the header (Length: unspecified [text/xml]). I seem to recall RobertE stating
> that he needed to gather the listings data together first in order to fill in the Length header, and that is why there was an additional delay with
> the replacement service. Has that changed? I AM getting listings data okay, or as far as I can tell. Just pointing out some anomalies going on
> behind the scenes in the logs.

I squished a pretty big authentication bug last night. Hopefully that solves the problem.
I don't actually set the headers directly, I let CGI do it, but I don't see how the length header could be set because it deson't know how long the
data will be when the headers have to go out.

Robert

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
I've switched to the new server using the /etc/hosts method using IP
54.85.117.227. I'm using MythTV 0.24. I'm running mfd with this command
line:
mythfilldatabase --dd-grab-all -v file,network >
/tmp/mythfilldatabase-newserver.log 2>&1

I can see in the log that it authenticates and downloads the data just
fine, but as it's loading up its tables, there a a ton of errors like this:

2014-10-23 14:54:28.363 DB Error (Inserting into dd_program):
Query was:
INSERT INTO dd_program ( programid, title,
subtitle, description, showtype, category_type,
mpaarating, starrating, stars,
runtime, year, seriesid, colorcode,
syndicatedepisodenumber, originalairdate) VALUES (?, ?,
?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?)
Bindings were:
:CATTYPE=series, :COLORCODE=, :DESCRIPTION=, :MPAARATING=,
:ORIGAIRDATE=2009-01-01, :PROGRAMID=EP000009935605, :RUNTIME=,
:SERIESID=EP00000993, :SHOWTYPE=Series, :STARRATING=, :STARS=0,
:SUBTITLE=Teams TBA, :SYNDNUM=TBA, :TITLE=College Football, :YEAR=
Driver error was [2/1062]:
QMYSQL3: Unable to execute statement
Database error was:
Duplicate entry 'EP000009935605' for key 'PRIMARY'

That's the only kind of error that appears (inserting into dd_program), but
it's there several hundred times with different programs each time. When I
run mfd with the /etc/hosts line commented out, there aren't any errors
like that. The end of the log appears to be successful; it finds generic
episodes, repeats, etc and tells the backend to reschedule.
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/23/2014 03:14 PM, Anthony Brummett wrote:
> I've switched to the new server using the /etc/hosts method using IP
> 54.85.117.227. I'm using MythTV 0.24. I'm running mfd with this command
> line:
> mythfilldatabase --dd-grab-all -v file,network >
> /tmp/mythfilldatabase-newserver.log 2>&1
>
> I can see in the log that it authenticates and downloads the data just
> fine, but as it's loading up its tables, there a a ton of errors like this:
>
> 2014-10-23 14:54:28.363 DB Error (Inserting into dd_program):
> Query was:
> INSERT INTO dd_program ( programid, title,
> subtitle, description, showtype, category_type,
> mpaarating, starrating, stars,
> runtime, year, seriesid, colorcode,
> syndicatedepisodenumber, originalairdate) VALUES (?, ?,
> ?, ?, ?, ?, ?, ?, ?,
> ?, ?, ?, ?, ?, ?)
> Bindings were:
> :CATTYPE=series, :COLORCODE=, :DESCRIPTION=, :MPAARATING=,
> :ORIGAIRDATE=2009-01-01, :PROGRAMID=EP000009935605, :RUNTIME=,
> :SERIESID=EP00000993, :SHOWTYPE=Series, :STARRATING=, :STARS=0,
> :SUBTITLE=Teams TBA, :SYNDNUM=TBA, :TITLE=College Football, :YEAR=
> Driver error was [2/1062]:
> QMYSQL3: Unable to execute statement
> Database error was:
> Duplicate entry 'EP000009935605' for key 'PRIMARY'
>
> That's the only kind of error that appears (inserting into dd_program), but
> it's there several hundred times with different programs each time. When I
> run mfd with the /etc/hosts line commented out, there aren't any errors
> like that. The end of the log appears to be successful; it finds generic
> episodes, repeats, etc and tells the backend to reschedule.

Hi,

That's some great data collection work. The timestamp says you ran it
today, so this *shouldn't* be the cause:

http://forums.schedulesdirect.org/viewtopic.php?f=6&t=2599&start=70#p7985

I'm just wondering if you've got some stale data and if it would
be worth running again tomorrow before jumping in to trouble-
shooting this one.

I did a run just now for comparison and here's what I see in the raw
download, note the multiple entries for this show (and I don't get any
errors.):

http://pastebin.com/Qn14FPXV

You can look for the line: ... Writing to temporary file: [/tmp/... in
your mfdb log and copy the file to a safe place before it gets deleted.
Then you can see if there are more entries than mine if you want to try
to work on it now. Sounds like extra data coming from SD though. You'll
have about a minute to copy if, so no super rush. Wait 'til you see the
line saying ... Uncompressed to xxxxxx bytes 1st.

dd_program is a table in the temporary DB created and removed when mfdb
is run.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On Thu, Oct 23, 2014 at 4:55 PM, Bill Meek <keemllib@gmail.com> wrote:

> On 10/23/2014 03:14 PM, Anthony Brummett wrote:
>
>> I've switched to the new server using the /etc/hosts method using IP
>> 54.85.117.227. I'm using MythTV 0.24. I'm running mfd with this command
>> line:
>> mythfilldatabase --dd-grab-all -v file,network >
>> /tmp/mythfilldatabase-newserver.log 2>&1
>>
>

> That's some great data collection work. The timestamp says you ran it
> today, so this *shouldn't* be the cause:
>
> http://forums.schedulesdirect.org/viewtopic.php?f=6&t=2599&
> start=70#p7985
>
> I'm just wondering if you've got some stale data and if it would
> be worth running again tomorrow before jumping in to trouble-
> shooting this one.
>

Probably not. I've been trying from time to time all week, and it was
failing in the same way. I ran it again today, and it's still failing with
the same kind of errors. Just looking at the first failure:
2014-10-24 11:16:05.054 DB Error (Inserting into dd_program):
Query was:
INSERT INTO dd_program ( programid, title,
subtitle, description, showtype, category_type,
mpaara
ting, starrating, stars, runtime, year,
seriesid, colorcode, syndicatedepisodenumber, ori
ginalairdate) VALUES (?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?
, ?, ?)
Bindings were:
:CATTYPE=series, :COLORCODE=, :DESCRIPTION=, :MPAARATING=,
:ORIGAIRDATE=2009-01-01, :PROGRAMID=EP000009935605, :RUNTIME=,
:SERIESID=EP00000993, :SHOWTYPE=Series, :STARRATING=, :STARS=0,
:SUBTITLE=Teams TBA, :SYNDNUM=TBA, :TITLE=College Football, :YEAR=
Driver error was [2/1062]:
QMYSQL3: Unable to execute statement
Database error was:
Duplicate entry 'EP000009935605' for key 'PRIMARY'

When I grep for 'EP000009935605' in the download data, there are 104
'schedule' tags (many, like yours). There also 2 'program' tags and 2
'programGenre' tags, where yours only has one of each. When I download
from the old server, it also has only a single 'program' and 'programGenre'
tags with that ID/program.

Here's the log from mfd taking to the new server with only a few of the 683
insert failures included:
http://pastebin.com/RK86fjzc

Here's the lines from the download data for 'EP000009935605' for the new
server:
http://pastebin.com/cJbpFADL

Here's the lines from the download data for 'EP000009935605' from the old
server:
http://pastebin.com/zMN0Xw07

Let me know if you need any more information.
Thanks,
-- Tony
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/24/2014 2:40 PM, Anthony Brummett wrote:
> On Thu, Oct 23, 2014 at 4:55 PM, Bill Meek <keemllib@gmail.com <mailto:keemllib@gmail.com>> wrote:
>
> On 10/23/2014 03:14 PM, Anthony Brummett wrote:
>
> I've switched to the new server using the /etc/hosts method using IP
> 54.85.117.227. I'm using MythTV 0.24. I'm running mfd with this command
> line:
> mythfilldatabase --dd-grab-all -v file,network >
> /tmp/mythfilldatabase-newserver.log 2>&1
>
> That's some great data collection work. The timestamp says you ran it
> today, so this *shouldn't* be the cause:
>
> http://forums.schedulesdirect.org/viewtopic.php?f=6&t=2599&start=70#p7985
>
> I'm just wondering if you've got some stale data and if it would
> be worth running again tomorrow before jumping in to trouble-
> shooting this one.
>
>
> Probably not. I've been trying from time to time all week, and it was failing in the same way. I ran it again today, and it's still failing with
> the same kind of errors. Just looking at the first failure:
> 2014-10-24 11:16:05.054 DB Error (Inserting into dd_program):
> Query was:
> INSERT INTO dd_program ( programid, title, subtitle, description, showtype, category_type, mpaara
> ting, starrating, stars, runtime, year, seriesid, colorcode, syndicatedepisodenumber, ori
> ginalairdate) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
> , ?, ?)
> Bindings were:
> :CATTYPE=series, :COLORCODE=, :DESCRIPTION=, :MPAARATING=,
> :ORIGAIRDATE=2009-01-01, :PROGRAMID=EP000009935605, :RUNTIME=,
> :SERIESID=EP00000993, :SHOWTYPE=Series, :STARRATING=, :STARS=0,
> :SUBTITLE=Teams TBA, :SYNDNUM=TBA, :TITLE=College Football, :YEAR=
> Driver error was [2/1062]:
> QMYSQL3: Unable to execute statement
> Database error was:
> Duplicate entry 'EP000009935605' for key 'PRIMARY'
>
> When I grep for 'EP000009935605' in the download data, there are 104 'schedule' tags (many, like yours). There also 2 'program' tags and 2
> 'programGenre' tags, where yours only has one of each. When I download from the old server, it also has only a single 'program' and 'programGenre'
> tags with that ID/program.

I think I see how this can happen, although it's pretty rare. I just submitted a fix, please try again.

If it doesn't work, please email me or admin@sd your username/password or tell me it's ok to reset it so I can test it.

Robert
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On Fri, Oct 24, 2014 at 3:17 PM, Robert Eden <rmeden@gmail.com> wrote:

> On 10/24/2014 2:40 PM, Anthony Brummett wrote:
>
>
> When I grep for 'EP000009935605' in the download data, there are 104
> 'schedule' tags (many, like yours). There also 2 'program' tags and 2
> 'programGenre' tags, where yours only has one of each. When I download
> from the old server, it also has only a single 'program' and 'programGenre'
> tags with that ID/program.
>
>
> I think I see how this can happen, although it's pretty rare. I just
> submitted a fix, please try again.
>
> If it doesn't work, please email me or admin@sd your username/password or
> tell me it's ok to reset it so I can test it.
>

Looks like it works now. No more insert errors in the mfd log. The
Listing Status screen on the frontend says mfd ran but did not insert any
new data. That may have been because I ran it against the old server
earlier today when I collected the logs.

Thanks for the fix!
-- Tony
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
<leo.butler@member.ams.org> writes:

> Robert Eden <rmeden@gmail.com> writes:
>
>> On 10/15/2014 9:46 AM, Jay Foster wrote:
>>>
>>> One thing I noticed in the listings from the replacement service is
>>> that it is sending data for all channels in the lineup(s), instead
>>> of just those that were enabled on the SD website. This works fine,
>>> in that one gets the listings, but does take more bandwidth and
>>> time. This is not something that I think needs to be "fixed" soon,
>>> but just an optimization for later, if ever.
>> Another bug fixed. (OTA lineups are completely different internally than cable channels)
>>>
>>> I have been using the replacement service for the last couple of
>>> days, and I am noticing something that concerns me. The listings
>>> data I am getting appears to not be equivalent enough to the
>>> previous service to match my previously recorded rules. I'm seeing
>>> shows that I know I previously recorded being rescheduled for future
>>> recordings.
>> Can you be more specific? What's wrong/missing from the raw data?
>>
>> Leo had a similar issue. It was determined he had MythTV configured
>> to use a SD station_Id that was excluded but TMS-DD was sending it
>> anyway. SD-DD was only sending data for the HD station_id and his
>> rules were not configured to use that.
>
> Sorry, Robert, I think you are wrong. The diff output here
>
> http://pastebin.com/1tZgC4yB
>
> shows 6 diff chunks, with 8 stations in total, that are in the TMS-DD
> data but not in the SD-DD data. When I checked on my SD account, these
> stations were included in the Cable source, but not the CableDigital
> source. So, the error was not with TMS-DD but SD-DD--or so it seems to
> me.
>
> However, you mentioned you may see different settings than we see on our
> SD accounts (?), and I wonder if this is what is happening to me.
>
> As I said, CableOne is killing analog cable, so I won't see this
> incongruity in a couple weeks.

First, let me thank RobertE for his signal work on the transition at
SD. I have been running from the new source almost every day since he
announced it here and it has worked nicely.


However, there is a twist.

CableOne did its switch-over, and the problem noted above persisted,
even after whitelisting all channels in the digital source. I had tons
of failed recordings. It turns out that the mythconverg channel table
had multiple channels listed for each of the channels that were causing
the problems noted above. 4 months ago, when setting up, I had
blacklisted those channels from the digital source on SD because
recordings were failing. I had resorted to blacklisting because I
couldn't figure out why recordings failed on only a small number of
channels.

In the end, this thread here

http://www.mythtv.org/pipermail/mythtv-users/2012-July/336364.html

helped me track down the problem and the wiki page

http://www.mythtv.org/wiki/UK_Channel_Assignments

guided me in doing a bulk enabling of only the channels my cable
subscription includes (after grabbing them from the hdhr3-cc).

Since I had never touched the channel table, etc., I do believe that the
corruption was caused by either bad data from SD or mythtv mishandled
good data. I can't determine which it is because my backups don't go
that far back.

Leo
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Schedules Direct DataDirect replacement service testing [ In reply to ]
On 10/25/14 23:50, Peter Bennett wrote:
>
> On 10/25/2014 12:17 PM, Keith Pyle wrote:
>
>> >Partially answering my own questions:
>> >
>> >I was able to grab a temporary file during the mfdb run. For this
>> >run, mfdb logged:
>> >
>> > Actual data from Fri Oct 24 15:56:10 2014 to Sat Nov 8 15:56:10
>> >2014 (UTC)
>> >
>> >However, the greatest value for the 'time=' field on any <schedule>
>> >line in the SD XML file is:
>> >
>> > 2014-11-06T23:58:00Z
>> >
>> >So, it appears that I am not getting data for Nov 7 or 8 from SD,
>> >contrary to what the 'Actual data' log line and the <xtvd> line in the
>> >XML show:
>> >
>> ><xtvd from='2014-10-24T15:56:10Z' to='2014-11-08T15:56:10Z'
>> >schemaVersion='1.3' xmlns='urn:TMSWebServices'
>> >xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
>> >xsi:schemaLocation='urn:TMSWebServices
>> >http://docs.tms.tribune.com/tech/xml/schemas/tmsxtvd.xsd'>
>> >
>> >Keith
>> >
> I have noticed the "Actual Data" message always shows 14 days, but there
> is really only 12 or 13 days downloaded. The result is mythtv deletes
> previous data for 14 days but only loads 12 or 13 days worth. This can
> cause some confusing results with gaps in the data on your database.
> However it does not really cause any other problems.
>
> Peter
Peter is correct. Today's mfdb update reports it was successful, shows
an "Actual data" range of 14 days in the log, really has 12 days of
data, and extended the guide data by one day.

Thanks for the comments.

Keith
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

1 2 3 4 5 6 7 8  View All