**------------------------------------------------------------------------------------------------ * @header_start * WebGrab+Plus ini for grabbing EPG data from TvGuide websites * @Site: tvguide.com * @MinSWversion: V2.1 * @Revision 16 - [06/07/2020] Blackbear199 * - subdetail scrub fix * @Revision 15 - [22/02/2017] Blackbear199 * - re-add subdetails * @Revision 14 - [28/11/2016] Blackbear199 * - remove subdetails(doesnt work) * - update season/episode to 56.26+ * - added country and country provider channels.xml section * @Revision 13 - [24/07/2016] Blackbear199 * - remove index_start/stop modify * @Revision 12 - [22/01/2015] francis De Paemeleere * - adjust to changes of the site * @Revision 11 - [16/01/2015] francis De Paemeleere * - quick fix (index only) * @Revision 10 - [29/06/2014] Jan van Straaten * - added previousshown and repeat * @Revision 9 - [08/06/2014] Ron * - changed () to [] in channel creation (MePo problem) * @Revision 8 - [02/05/2014] Francis De Paemeleere * - adjust to the new timezone implementation of WG++ * @Revision 7 - [05/03/2014] Francis De Paemeleere * - total rewrite of the code * @Remarks: * this is the base version * @header_end **------------------------------------------------------------------------------------------------ site {url=tvguide.com|timezone=UTC|maxdays=16.1|cultureinfo=en-US|charset=UTF-8|titlematchfactor=90} site {ratingsystem=MPAA|episodesystem=onscreen} * url_index {url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/airings/##svrID##/start/|urldate|/duration/##Minutes##?channelsourceids=##SourceId##%7C##Number##&formattype=json} url_index.headers {customheader=Accept-Encoding=gzip,deflate} * urldate.format{datenumber|UNIX|0} * scope.range {(urlindex)|end} index_temp_1.modify {substring(type=regex)|'config_site_id' "\(srvID([^)]*)\)"} url_index.modify {replace|##svrID##|'index_temp_1'} index_temp_1.modify {substring(type=regex)|'config_site_id' "SourceId:([^,]*)"} url_index.modify {replace|##SourceId##|'index_temp_1'} index_temp_1.modify {substring(type=regex)|'config_site_id' "Number:([^,]*)"} url_index.modify {replace|##Number##|'index_temp_1'} index_temp_1.modify {calculate(format=D0)|'config_timespan_days' 1 + 1440 *} * calculate how may minutes we want to get (so we only download the days, we want to see) url_index.modify {replace|##Minutes##|'index_temp_1'} end_scope * index_showsplit.scrub {regex||"ProgramSchedule":.*?}}||} * index_start.scrub {single|"StartTime":||,|,} index_stop.scrub {single|"EndTime":||,|,} index_title.scrub {single|"Title":"||",|",} index_subtitle.scrub {single(exclude="Episode")|"EpisodeTitle":"||",|",} index_temp_4.scrub {regex||"AiringAttrib"\s*:\s*(\d*)||} * binairy: 1= Live, 2=??,4=New, 8=cc * grab season and episode number index_temp_5.scrub {regex||"SeasonNumber":(\d+)||} index_temp_6.scrub {regex||"EpisodeNumber":"(\d+)||} * scope.range {(indexshowdetails)|end} index_title.modify {cleanup(style=jsondecode)} index_subtitle.modify {cleanup(style=jsondecode)} * remove subtitle from title index_title.modify {remove(notnull type=string)|:'index_subtitle'} * * generate season.episode index_episode.modify {clear} index_temp_5.modify {calculate(not="" format=F0)|1 -} index_episode.modify {addend('index_temp_5' not="")|'index_temp_5'} index_episode.modify {addend|.} index_temp_6.modify {calculate(not="" format=F0)|1 -} index_episode.modify {addend('index_temp_6' not="")|'index_temp_6'} index_episode.modify {addend|.} index_episode.modify {clear(="..")} index_episode.modify {clear(~"-1")} index_episode.modify {substring(type=regex pattern="'S0'.'E0'.")|^.*$} index_temp_6.modify {clear} * * add flags according to airing attributes index_temp_5.modify {calculate(format=D0)|'index_temp_4' 1 and} index_temp_6.modify {addend('index_temp_5'=="1")| (live)} index_temp_5.modify {calculate(format=D0)|'index_temp_4' 2 and} index_temp_6.modify {addend('index_temp_5'=="2")| (repeat)} index_temp_5.modify {calculate(format=D0)|'index_temp_4' 4 and} index_temp_6.modify {addend('index_temp_5'=="4")| (new)} * next line adds * to new shows, Ttile * = new shows, Title = "not new", 2 places index_title.modify {addend('index_temp_5'=="4")| *} index_temp_5.modify {calculate(format=D0)|'index_temp_4' 8 and} index_temp_6.modify {addend('index_temp_5'=="8")| (cc)} index_urlshow {url|http://mapi.tvguide.com/listings/details?program=|"ProgramId":||,|,} index_urlshow.headers {customheader=Accept-Encoding=gzip,deflate} end_scope * title.scrub {regex||^.*?"title"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} description.scrub {regex||^.*?"description"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} rating.scrub {regex||^.*?"rating"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} productiondate.scrub {regex||^.*?"release_year"\s*:\s*([+-]?\d*)||} showicon.scrub {regex||"photos":\[[^]]*?\]||} temp_2.scrub {regex||^.*?"type"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} temp_4.scrub {regex||^.*?"category_id"\s*:\s*([+-]?\d*)||} * genre ID * scope.range {(showdetails)|end} title.modify {cleanup(style=jsondecode)} description.modify {cleanup(style=jsondecode)} rating.modify {cleanup(style=jsondecode)} productiondate.modify {cleanup(style=jsondecode)} temp_2.modify {cleanup(style=jsondecode)} temp_2.modify {remove|tv} temp_2.modify {cleanup(style=name)} * * remove subtitle from title title.modify {remove(notnull type=string)|:'index_subtitle'} * temp_5.modify {calculate(format=D0)|'index_temp_4' 4 and} * next line adds * to new shows, Title * = new shows, Title = "not new", 2 places title.modify {addend('temp_5'=="4")| *} rating.modify {clear("None")} description.modify {addend|'index_temp_6'} * category.modify {addend('temp_4' == "1")|##_##Movie} category.modify {addend('temp_4' == "2")|##_##Sports} category.modify {addend('temp_4' == "3")|##_##Family} category.modify {addend('temp_4' == "4")|##_##News} category.modify {addend|##_##'temp_2'} category.modify {replace|##_##|\|} category.modify {remove|other} * previousshown.modify {substring(type=regex)|'description' "(\(repeat\))"} previousshown.modify {replace(not "")|(repeat)|true} * description.modify {remove(type=regex)|"(\(repeat\))"} *optional removes (repeat) from the description premiere.modify {substring(type=regex)|'description' "(\(new\))"} *premiere.modify {replace(not "")|(new)|true} *optional removes (new) from the description description.modify {remove(type=regex)|"(\(new\))"} description.modify {cleanup} * showicon.modify {substring(type=regex)|"url":"([^"]*)"} * get all the photos for this show showicon.modify {substring(type=element)|0 1} * only get the first image (multi images are not yet supported by WG++) showicon.modify {cleanup(style=jsondecode)} end_scope * urlsubdetail {url||"seo_url":"||"|"} urlsubdetail.modify {cleanup(style=jsondecode)} urlsubdetail.modify {replace(type=regex)|(/)\d+/$|/cast/} urlsubdetail.headers {customheader=Accept-Encoding=gzip,deflate} * subdetail_director.scrub {multi|>Director (||\n |\n } subdetail_actor.scrub {multi(max=8)|class="tvobject-cast-featured-role-content">|itemprop="name">|

|

} subdetail_writer.scrub {multi(max=2)|>Writer (||\n |\n } subdetail_producer.scrub {multi(max=2)|>Producer (||\n |\n } * subdetail_director.modify {substring(type=regex)|"itemprop=\"name\"[^>]*>(.*?)<\/span>"} subdetail_writer.modify {substring(type=regex)|"itemprop=\"name\"[^>]*>(.*?)<\/span>"} subdetail_producer.modify {substring(type=regex)|"itemprop=\"name\"[^>]*>(.*?)<\/span>"} *subdetail_actor.modify {cleanup(tags="")} subdetail_actor.modify {replace(type=regex)|"^.*?(<\/span><\/a>.*?name header-5\">)"|(role=} subdetail_actor.modify {replace(type=regex)|"^.*?\(role=[^<]*(<.*$)"|)} subdetail_actor.modify {replace(type=regex)|"^[^<]*(<.*$)"} *subdetail_actor.modify {remove(type=regex)|"\(\d{4}\)"} *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### COUNTRY FILE CREATION (only to create the xxx-channel.xml file) ** ** @auto_xml_country_start *url_index {url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/providercountries?formattype=json} *index_site_id.scrub {multi(separator=",")|[||]|]} *index_site_id.modify {remove|"} *index_site_channel.scrub {multi(separator=",")|[||]|]} *index_site_channel.modify {remove|"} ** @auto_xml_country_end *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### PROVIDER FILE CREATION BY COUNTRY(only to create the xxx-channel.xml file) ** ** @auto_xml_country-provider_start *url_index {url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/serviceproviders/zipcode/0?country=|channel|&formattype=json} *index_site_id.scrub {regex||{"City":"[^\"]*","Id":(\d+),"Name":"[^\"]*"||} *index_site_channel.scrub {regex||{"City":"[^\"]*","Id":\d+,"Name":"([^\"]*)"||} ** @auto_xml_country-provider_end *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### PROVIDER FILE CREATION BY ZIPCODE (only to create the xxx-channel.xml file) ** * replace site_id="xxxx" with your zipcode * dummy * ** @auto_xml_provider_start *url_index {url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/serviceproviders/zipcode/|channel|?formattype=json} *index_site_id.scrub {regex||"Id"\s*:\s*([+-]?\d*)||} *index_site_channel.scrub {regex||"Name"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} ** @auto_xml_provider_end *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### CHANNEL FILE CREATION BY PROVIDER(only to create the xxx-channel.xml file) ** ** @auto_xml_channel_start *url_index {url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/schedules/|channel|/start/|urldate|/duration/1?ChannelFields=Name%2CFullName%2CNumber%2CSourceId&ScheduleFields=&formattype=json&disableChannels=} *index_site_id.scrub {regex||"FullName":"[^"]*"[^}]*("Number":"[^"]*"[^}]*"SourceId":\d*)||} *index_site_channel.scrub {multi|{"Channel":{"FullName":"||","Number"|","Number"} *scope.range {(channellist)|end} *index_site_id.modify {remove(type=regex)|("Sort"\s*:\s*\d*\s*,*)} *index_site_id.modify {remove|"} *index_site_id.modify {addstart|(srvID'config_site_id')} *add the config site_id, so, we can keep this siteini generic (no need for the user to edit it) *index_site_id.modify {cleanup(removeduplicates=equal,100 link="index_site_channel")} *index_site_channel.modify {replace|","Name":"| - } *index_site_channel.modify {remove|\} *index_site_channel.modify {replace|)|]} *index_site_channel.modify {replace|(|[} *end_scope ** @auto_xml_channel_end *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### TIMEZONE FILE CREATION (only to create the xxx-channel.xml file) ** this will create a .channels.xml file (channels by timezone) that will contain all the channels for all the supported timezones ** this can be used to generate a new tvguide.com_channels_byTimezone.xml ** @auto_xml_timezone_start *url_index {url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/schedules/|subpage|/start/|urldate|/duration/1?ChannelFields=Name%2CFullName%2CNumber%2CSourceId&ScheduleFields=&formattype=json&disableChannels=} *subpage.format {list|80001|80002|80003|80004|80005|80006} *index_site_id.scrub {single||||} *scope.range {(channellist)|end} ** ! remark: we loop from the back to the front! *index_temp_4.modify {clear} *index_temp_4.modify {addstart|80006} * *index_site_id.modify {replace|[\{"Channel":|\n(srvIDNNNN)} * "split" the full lising into timezone separate parts * *index_site_channel.modify {substring(type=regex)|'index_site_id' "(srvIDNNNN)\|"FullName":"([^"]*)"[^}]*"SourceId":\d*"} *index_site_id.modify {substring(type=regex)|'index_site_id' "(srvIDNNNN)\|"FullName":"[^"]*"[^}]*"SourceId":(\d*)"} * *index_temp_1.modify {calculate(type=element format=F0)|'index_site_channel' #} * how many elements? Used as loop counter. *index_temp_6.modify {clear} *index_temp_5.modify {clear} *index_temp_6.modify {addstart|'index_site_channel'} * copy original elemets in temp *index_temp_5.modify {addstart|'index_site_id'} * copy original elemets in temp *index_site_channel.modify {clear} * to be refilled later *index_site_id.modify {clear} * to be refilled later *loop {('index_temp_1' > "0" max=5000)|end} *index_temp_1.modify {calculate(format=F0)|1 -} * decrease the loop counter *index_temp_2.modify {substring(type=element)|'index_temp_6' 'index_temp_1' 1} * element to work on *index_temp_3.modify {substring(type=element)|'index_temp_5' 'index_temp_1' 1} * element to work on * *index_site_channel.modify {addstart('index_temp_2' not== "srvIDNNNN")|##_####TIMEZONE##'index_temp_2'} * fill elements again , add a placeholder ##_## as separator *index_site_id.modify {addstart('index_temp_2' not== "srvIDNNNN")|##_####SERVERID##Number:*,SourceId:'index_temp_3'} * fill elements again , add a placeholder ##_## as separator * *index_site_id.modify {replace('index_temp_2' == "srvIDNNNN")|##SERVERID##|(srvID'index_temp_4')} * after all channels are added, change the server id to the correct value *index_site_channel.modify {replace('index_temp_2' == "srvIDNNNN")|##TIMEZONE##|##TIMEZONE'index_temp_4'##} * after all channels are added, change the server id to the correct value *index_temp_4.modify {calculate('index_temp_2' == "srvIDNNNN" format=F0)|'index_temp_4' 1 -} * decrease the server id for the next time (80006, 80005, 80004, ...) * *end_loop *index_site_channel.modify {replace|##TIMEZONE80001##|(Eastern)} *index_site_channel.modify {replace|##TIMEZONE80002##|(Central)} *index_site_channel.modify {replace|##TIMEZONE80003##|(Mountain)} *index_site_channel.modify {replace|##TIMEZONE80004##|(Pacific)} *index_site_channel.modify {replace|##TIMEZONE80005##|(Alaskan)} *index_site_channel.modify {replace|##TIMEZONE80006##|(Hawaiian)} *index_site_channel.modify {replace|##_##|\|} *index_site_id.modify {replace|##_##|\|} *index_site_id.modify {cleanup(removeduplicates=equal,100 link="index_site_channel")} ** replace in channels with [ ] *index_site_channel.modify {replace|)|]} *index_site_channel.modify {replace|(|[} *end_scope ** @auto_xml_timezone_end