Dear all
Since 03 Mar 2017 my installation no longer works with tvguide.co.uk, I get (500) Internal Server Error.
I'm using WebGrab+Plus/w MDB & REX Postprocess -- version 1.1.1/55.27, mono on Raspberry Pi 2.
Is this a known issue? Does it help to upgrade to 2.0?
Cheers
same issue here
Update to the latest versiion v2 and it should work,
Edited
worked on one grab didnt work on the next.
It appears that it's the server that doesn't work.
Example: BBC1 for today:
http://my.tvguide.co.uk/channellisting.asp?ch=93&cTime=3/5/2017%2010:00:...
returns HTTP status 500 and the text:
Houston, we have a problem!
Oops, we appear to have encountered a problem. Our techy people have been notified.
We apologise for any inconvenience. Please try one of the other links above, or try again later.
Thank you.
Thanks for the suggestion. Sometimes I can get BBC1 HD, sometimes BBC2 HD, but the other channels (BBC 4 HD, ITV HD, Channel 4 HD, More4, E4 and Channel 5 HD) all fail consistently with:
Unable to update channel <channel_name>
Generic syntax exception:
message:
no index page data received from <channel_name>
It seems that it works for the first channel in the configuration file and then it fails for the rest. So I can run with one channel at a time, commenting out the rest...
inspectet the page and it is the same so something has been added might be a block from the same ip or something my.tvguide.co.uk/channellisting.asp?ch=74&cTime=3/6/2017 11:00:00 AM&thisTime=&thisDay=
Yes thats what I ment its the same in the ini that calls the data so nothing has changed there but as it does grab the first and second channel and then stops, thats where i thought about the blocking of the ip as it was previously done with tv.com and we had to place a 30sec delay between each channel grab for it to work. but this just doesnt respond to that as well.
Seems to work if you wait 5 min between each channel grab. What's the easiest way of doing that with .ini/config, <retry channel-delay="300"...>?
I am a relatively new user ... only a week or so whick has been looking good.
As of today however I now get this problem?? Can anyone help
[ ] --------------------------------------------------------------------------------
[ ]
[ ] Job started at 06/03/2017 12:40:22
[ Debug ]
[ Debug ] Running on: Microsoft Windows NT 6.1.7601 Service Pack 1
[ Debug ] Environment: 4.0.30319.34209
[ Debug ]
[ Debug ] Loading timezone data
[ Debug ] Embedded timezones source: WGconsole.WG.Common.timezonesdata.txt
[ Debug ] Reading config file: C:\Users\paul\AppData\Local\WebGrab+Plus\WebGrab++.config.xml
[ Info ] found: C:\Users\paul\AppData\Local\WebGrab+Plus\tvguide.co.uk.ini -- Revision 15
[ Debug ] xmltv input file - C:\Users\paul\AppData\Local\WebGrab+Plus\data.xml - found
[ Debug ] 0 superfluous shows removed
[Error ] Could find existing channel (xmltv_id=BBC1 HD) in the config file
[Error ] Could find existing channel (xmltv_id=BBC2 HD) in the config file
[Error ] Could find existing channel (xmltv_id=Channel 4 HD) in the config file
[Error ] Could find existing channel (xmltv_id=ITV HD) in the config file
[ Info ]
[ Info ]
[ Info ] i=index .=same c=change g=gab r=replace n=new
[ Info ]
[ Info ]
[ Info ] Group (0) :
[ Info ] update requested for - 4 - out of - 4 - channels for 7 day(s)
[ Debug ]
[ Info ] ( 1/4 ) TVGUIDE.CO.UK -- chan. (xmltv_id=BBC1 HD) -- mode Smart
[Error ] Unable to update channel BBC1 HD
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from BBC1 HD
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!
[ Debug ]
[ Info ] ( 1/4 ) TVGUIDE.CO.UK -- chan. (xmltv_id=BBC2 HD) -- mode Smart
[Error ] Unable to update channel BBC2 HD
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from BBC2 HD
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!
[ Debug ]
[ Info ] ( 1/4 ) TVGUIDE.CO.UK -- chan. (xmltv_id=Channel 4 HD) -- mode Smart
[Error ] Unable to update channel Channel 4 HD
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from Channel 4 HD
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!
[ Debug ]
[ Info ] ( 1/4 ) TVGUIDE.CO.UK -- chan. (xmltv_id=ITV HD) -- mode Smart
[Error ] Unable to update channel ITV HD
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from ITV HD
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!
[ Debug ]
[ Debug ] 0 shows in 4 channels
[ Debug ] 0 updated shows
[ Debug ] 0 new shows added
[ Info ]
[ Info ]
[ ] Job finished at 06/03/2017 12:40:28 done in 5s
[ Debug ] statistics upload error: The remote server returned an error: (500) Internal Server Error.
You're facing the same issues as us.
Try to add the following attribute to the WebGrab++.config.xml file's retry element:
channel-delay="300"
Example:
Change
<retry time-out="5">4</retry>
to
<retry time-out="5" channel-delay="300">4</retry>
This will add a delay of 5 min. between each channel grab.
This modification does not work for me either
<settings>
<!-- for detailed info about the settings see http://webgrabplus.com/documentation/configuration/webgrabconfigxml -->
<filename>data.xml</filename>
<mode></mode>
<postprocess grab="y" run="n">mdb</postprocess>
<user-agent>Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0</user-agent>
<logging>on</logging>
<retry time-out="5" channel-delay="300">4</retry>
<timespan>6</timespan>
<update>s</update>
<channel update="i" site="tvguide.co.uk" site_id="683" xmltv_id="BBC1 HD">BBC1 HD</channel>
<channel update="i" site="tvguide.co.uk" site_id="387" xmltv_id="BBC2 HD">BBC2 HD</channel>
<channel update="i" site="tvguide.co.uk" site_id="476" xmltv_id="Channel 4 HD">Channel 4 HD</channel>
<channel update="i" site="tvguide.co.uk" site_id="642" xmltv_id="ITV HD">ITV HD</channel>
</settings>
Try to change to m.tvguide.co.uk.
It seems to work using m.tvguide.co.uk, however there are the following differences:
Only gives 5 day EPG instead of 7.
Doesn't use xmltv_ns episode system.
Truncates descriptions.
Lots of titles have a suffix of " (?)".
Any suggestions on how to change m.tvguide.co.uk to work like tvguide.co.uk in terms of these differences?
this got to be too much of a hassle. most of what i needed was replaced by freesat.co.uk the other few channels i pieces from other webgrab sources
Sadly, that doesn't work for me. Freesat and Freeview don't provide episode information for most TV series and episode information is the only reason I use other EPG services.
Mobile working for me as well, but would love a full solution if anyone has one :)
I agree with MofTot, I used to use XMLTVGui and Radio Times for episode numbers ... a very important requirement for me
Excellent, will it fix the other issues I reported earlier as well (5 day EPG instead of 7, doesn't use xmltv_ns episode system, truncates descriptions, lots of titles have a suffix of " (?)")?
Thanks - looking forward to the fix!
It's what tvguide.co.uk.ini does, perhaps you can get inspired.
Thanks so much Netuddki :)
Thanks so much, excellent progress!
A few issues:
1. Most titles are still suffixed with " (?)". But it's inconsistent from run to run.
2. Most descriptions are suffixed with " (n)".
3. The episode info extraction is too sensitive to the word "season" in description. I've attached an example where it goes very wrong, episode-num contains most of the description.
4. The first time I ran it, no data for Channel 4 HD and More4 was downloaded:
( 5/8 ) M.TVGUIDE.CO.UK -- chan. (xmltv_id=Channel 4 HD) -- mode Force
i
Unable to update channel Channel 4 HD
Generic syntax exception:
message:
no index page data received from Channel 4 HD
unable to update channel, try again later
( 6/8 ) M.TVGUIDE.CO.UK -- chan. (xmltv_id=More4) -- mode Force
i
Unable to update channel More4
Generic syntax exception:
message:
no index page data received from More4
unable to update channel, try again later
5. I can't seem to configure it for xmltv_ns episode numbers. I'm not sure what I need to change in the .ini file, currently it's:
site {url=tvguide.co.uk|timezone=Europe/London|maxdays=7.1|cultureinfo=en-GB|charset=UTF-8|titlematchfactor=90|allowlastpageoverflow|episodesystem=xmltv_ns}
*
*site {ratingsystem=GB|episodesystem=onscreen|} *Enable for Onscreen Episode System
site {ratingsystem=GB|episodesystem=xmltv_ns|} *Enable for xmltv_ns Episode System
*
and it produces episode numbers like:
<episode-num>Season 4 Episode 8/20</episode-num>
Good, I'm indeed running:
WebGrab+Plus/w MDB & REX Postprocess -- version V2.0
I'm seeing the same issues with all channels, BBC1 HD, BBC2 HD, BBC4 HD, ITV HD, Channel 4 HD, More4 and E4.
For the channels I use the format on the index page http://my.tvguide.co.uk/mobile/channellisting.asp?ch=<channel> is always "Season x Episode y" or "Season x Episode y/z" if z is known.
OK, perhaps this helps: My anylisis for the channels I use is that
contains one of the following:
1. description
2. episode information<br><br>description
3. subtitle<br>episode information<br><br>description
4. subtitle<br>description
But if you can't get to the raw HTML of the div content I can see how that gets very difficult to parse, you need the <br> tags as delimiters.
Hi Netuddki,
Would you be able to advise how I can update the channel list for this site? I tried commenting out the channel creation section in the file you uploaded and adding the below dummy channel, but the resulting channel list is empty.
I know the channel xml is out of date so this would really help!
Does this help?
http://regexr.com/3fg73
Just omit the / in the value for xmlns_ns when you don't know the total number of episodes. These are valid values:
0.9/10.
8.2.
How would i change the episode system to xmltv_ns on this ini? Or could you make a one similar to tvguide.co.uk.ini with an option to do either/or
Sorry i meant the m.tvguide.co.uk-full.ini does not have an episodesystem in it
If i add site {ratingsystem=GB|episodesystem=xmltv_ns|} like this
It still shows as onscreen in the output <episode-num system="onscreen">Season 3 Episode 3/20</episode-num>
I am deleting the guide.xml before i test everytime and also use "f"
I think maybe the ini is missing something
Excellent, works great now thank you so much for your help