You are here

Schedules direct channels.xml generation

4 posts / 0 new
Last post
videogamingtown
Offline
Joined: 3 years
Last seen: 10 months
Schedules direct channels.xml generation

I'm trying to generate my lineup into a channels xml file to use with schedules direct, i've followed all instructions on the site and the info document but i still cannot generate a channels.xml file

i get the following error when i run webgrab with the dummy line added to the configuration file


[ Info ] Group (0) :
[ Info ] update requested for - 1 - out of - 1 - channels for 1 day(s)
[ Debug ]
[ Info ] ( 1/1 ) SCHEDULESDIRECT.ORG -- chan. (xmltv_id=dummy) -- mode Force
[Error ] Unable to update channel dummy
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from dummy
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!
[ Debug ]
[ Debug ] 0 shows in 1 channels
[ Debug ] 0 updated shows
[ Debug ] 0 new shows added

below is my code for the ini file (i've replaced my user and pass)


**------------------------------------------------------------------------------------------------
* @header_start
* WebGrab+Plus ini for grabbing EPG data from TvGuide websites
* @Site: schedulesdirect.org
* @MinSWversion: V1.1.1/56.6
* @Revision 3 - [15/11/2015] Jan van Straaten
* - remove of faulty credits role solution
* @Revision 2 - [24/10/2015] Francis De Paemeleere
* - speedup parsing (tested with V1.56.6)
* @Revision 1 - [23/10/2014] Jan van Straaten
* - adapted site changes
* @Revision 0 - [31/08/2013] Jan van Straaten / Francis De Paemeleere
* - creation
* @Remarks: You need a login and password for this site
* @header_end
**------------------------------------------------------------------------------------------------

site {url=schedulesdirect.org|timezone=UTC|maxdays=16.1|cultureinfo=en-US|charset=UTF-8|titlematchfactor=90|keepindexpage|firstshow=1}
site {ratingsystem=MPAA|subtitlestype=teletext|episodesystem=onscreen}
url_index {url|http://dd.schedulesdirect.org/schedulesdirect/tvlistings/xtvdService}
url_index.headers {method=SOAP}
url_index.headers {customheader=SOAPAction=urn:tvDataDelivery#download}
url_index.headers {customheader=Accept-Encoding=gzip,deflate}
*
url_index.headers {credentials=myusername,mypassword}
url_index.headers {credentials=pedrobmat,bmatbmat}
*
url_index.headers {accept=text/xml|contenttype=text/xml;charset="utf-8"}
url_index.headers {postdata='index_variable_element'}

scope.range {(urlindex)|end}
** timespan calculation to enable to add the requested timespan from the config
index_variable_element.modify {calculate(format=F1)|'config_timespan_days' 1 +} * add 1 day because config_timespan_days is 0 based
index_variable_element.modify {calculate(format=timespan,hours)} * convert to the proper timespan string required for index_temp_3
index_temp_1.modify {calculate(format=date,yyyy-MM-dd)|'urldate'}
index_temp_3.modify {calculate(format=date,yyyy-MM-dd)|'urldate' 'index_variable_element' +}
index_variable_element.modify {clear} * clear the timespan value
index_variable_element.modify {addstart|'index_temp_1'T00:00:00Z'index_temp_3'T00:00:00Z}
end_scope

index_showsplit.scrub {regex||||} * copies the whole index page
scope.range {(splitindex)|end}
* now break up the large chuck into 3 smaller chucks, to speedup parsing
index_variable_element.modify {clear}

global_temp_1.modify {substring(type=regex)|'index_showsplit' "^.*?(.*?)"} * contains all programs
global_temp_1.modify {replace|title>|t>}
global_temp_1.modify {replace|subt>|st>}
global_temp_1.modify {replace||}
global_temp_1.modify {replace|syndicatedEpisodeNumber>|sEN>}
global_temp_1.modify {replace|description>|d>}
global_temp_1.modify {replace|originalAirDate>|oAD>}
global_temp_1.modify {replace|showType>|sT>}
global_temp_1.modify {replace|series>|s>}

** SPEEDUP option: if you don't need production/crew/cast info, just comment out next line
global_temp_2.modify {substring(type=regex)|'index_showsplit' "^.*?(.*?)"} * contains all productionCrew
global_temp_2.modify {replace|name>|>}
global_temp_2.modify {replace|given>|g>}
global_temp_2.modify {replace|sur>|s>}
global_temp_2.modify {replace|member>|m>}
global_temp_2.modify {replace|role>|r>}
global_temp_2.modify {replace|r>Actor|r>A}
global_temp_2.modify {replace|program=|p=}

** SPEEDUP option: if you don't need extra genre info, just comment out next line
global_temp_3.modify {substring(type=regex)|'index_showsplit' "^.*?(.*?)"} * contains all programs
global_temp_3.modify {replace|genre>|g>}
global_temp_3.modify {replace|class>|c>}
global_temp_3.modify {replace|relevance>|r>}
global_temp_3.modify {replace|programGenre>|pG>}
global_temp_3.modify {replace|(.*?)"} * contains all shows we need
index_temp_2.modify {substring(type=regex)|(]* station=\''config_site_id'\' [^>]*>)}
index_temp_2.modify {cleanup(removeduplicates=equal,100)}
index_showsplit.modify {set|'index_temp_2'}

end_scope
**
index_start.scrub {regex||time='[^\']*?T(\d{2}:\d{2}):\d{2}Z'||}
index_duration.scrub {regex||duration='PT(\d*H\d*)M||}
index_temp_5.scrub {regex||program='([^\']*)||}
index_videoquality.scrub {regex||hdtv='([^\']*)||}
index_videoquality.modify {replace(not="")|'index_videoquality'|HDTV}

index_subtitles.scrub {regex||closeCaptioned='([^\']*)||}
index_subtitles.modify {replace(not="")|'index_subtitles'|true}

scope.range {(indexshowdetails)|end}
index_start.modify {calculate(format=utctime)}
index_duration.modify {replace|H|:}

** get the programs part
index_temp_4.modify {substring(type=regex)|'global_temp_1' "^.*?

(.*?)

"}
index_title.modify {substring(type=regex)|'index_temp_4' "([^([^([^([^([^([^([^(\**)[\+]*"} * full stars
index_temp_1.modify {substring(type=regex)|'index_temp_4' ".*(\+)"} * half star
index_starrating.modify {calculate(not="" type=char format=F0)|#}
index_starrating.modify {addend('index_temp_1' not="")|.5}
index_starrating.modify {addend(not="")| / 4}
index_category.modify {substring(type=regex)|'index_temp_4' "([^(.*?)"}
index_actor.modify {substring(type=regex)|'index_temp_4' "[^A(.*?)"}
index_actor.modify {cleanup(tags="")}
*index_temp_1.modify {substring(type=regex)|'index_temp_4' "[^Guest Star(.*?)"}
*index_temp_1.modify {cleanup(tags="")}
*index_temp_1.modify {addend(not "")| (Guest Star)}
*index_actor.modify {addend('index_temp_1' not "")|'index_temp_1'\|}
index_actor.modify {cleanup(removeduplicates)}
index_presenter.modify {substring(type=regex)|'index_temp_4' "[^Host(.*?)"}
index_presenter.modify {cleanup(tags="")}
index_director.modify {substring(type=regex)|'index_temp_4' "[^Director(.*?)"}
index_director.modify {cleanup(tags="")}
index_producer.modify {substring(type=regex)|'index_temp_4' "[^Producer(.*?)"}
index_producer.modify {cleanup(tags="")}
index_producer.modify {substring(type=regex)|'index_temp_4' "[^Executive Producer(.*?)"}
index_producer.modify {cleanup(tags="")}
*index_temp_1.modify {substring(type=regex)|'index_temp_4' "[^Executive Producer(.*?)"}
*index_temp_1.modify {cleanup(tags="")}
*index_temp_1.modify {addend(not "")| (Executive Producer)}
*index_producer.modify {addstart('index_temp_1' not="")|'index_temp_1'\|}
index_producer.modify {cleanup(removeduplicates)}
index_writer.modify {substring(type=regex)|'index_temp_4' "[^Writer(.*?)"}
index_writer.modify {cleanup(tags="")}
* get the genres part
index_temp_4.modify {substring(type=regex)|'global_temp_3' "^.*?(.*?)"}
index_temp_1.modify {substring(type=regex)|'index_temp_4' ".*?(.*?)"}
index_category.modify {addstart('index_temp_1' not= "")|'index_temp_1'\|}
end_scope

** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
** ##### CHANNEL FILE CREATION (only to create the xxx-channel.xml file)
**
* @auto_xml_channel_start
index_site_id.scrub {regex||()||}
scope.range {(channellist)|end}
index_site_channel.modify {substring(type=regex)|'index_site_id' "(.*?)"}
index_site_id.modify {substring(type=regex)|'index_site_id' ""}
index_site_id.modify {cleanup(removeduplicates=equal,100 link="index_site_channel")}
end_scope
* @auto_xml_channel_end

raedwa01
Offline
Joined: 2 months
Last seen: 2 months

I am getting the exact same error. I also downgraded to the base 2.0 and still am getting the error. I do see a 404 not found error at the bottom of the log...

MrGrymReaper
Offline
Donator
Joined: 4 years
Last seen: 1 month

When you come up with a solution please contact me! As I too would like to be using 2.1 or above!

MrGrymReaper
Offline
Donator
Joined: 4 years
Last seen: 1 month

This is through the JSON Schedules Direct I mean. If Webgrab++ doesn't support this I feel it would be good to add this. Since through schedules direct there wouldn't be any need for website scraping for the service. Along with its international service and the large amount of data available.

If it also is because WebGrab++ doesn't support JSON then it is also an important addition as from a post processing point of view "TheTVDB" only supports JSON REST based APIs and the same for "The Movie DB (TMDB)".

Log in or register to post comments

Brought to you by Jan van Straaten

Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl