USA EPG

Uthall wrote:

Hello,
Im trying to find an upto date file for USA epg data.
Can someone point me to the best file to be using please.
Thnx

 
I've been using a combo of TV Guide (mainly) and TV Passport.
You need to run your own channels list for your preferred city/provider. I was helped big time to fix a few things in the TVGuide ini by Blackbear and mat8861 on the TVGuide ini.
  
Not sure if you know how to run a provider list and channels list.  So I made the different files and saved them for easier providers and channels lists.
 
I'm uploading the normal ini files (for TVGuide & TVPassport) and also the ones you use to create a Provider list (for TVGuide) and then you create a Channels list (for TV Guide). When you run each one, you will need to rename them to the normal "tvguide.com.ini" name as they wont work with "_PROVIDER" and "_CHANNELS" in the names.  I would put all 3 of the TVGuide ini files in a separte folder so you can just paste to replace the files.
 
First put the "_PROVIDER" one (renamed of course) in the main Webgrab++ folder.  Edit the channels list in the Webgrab config file (put your own zip in site_id)
<channel update="i" site="tvguide.com" site_id="74989" xmltv_id="Spectrum (Compton)">Spectrum (Compton)</channel>
Run WG++
You should get a "tvguide.com.channels.xml" with different Providers in that zip code.  I would rename that xml to reflect its providers.
 
 
 
 
Second, replace the tvguide.com.ini (that was originally the "_PROVIDER" one) with the "_CHANNELS" (and make sure to remove "_channels").  Copy and paste a provider from your "channels.xml" into the the Webgrab config file
<channel update="i" site="tvguide.com" site_id="85471" xmltv_id="AT&amp;T - U-verse (Houston)">AT&amp;T - U-verse (Houston)</channel>
 
Run WG++
You should get another "tvguide.com.channels.xml" with different channels from that provider.
 
 
Now replace the tvguide.com.ini with the normal one i uploaded.  Paste any channels from the "tvguide.com.channels.xml" that was created for instance:
<channel update="i" site="tvguide.com" site_id="(srvID74989)Number:518,SourceId:11096" xmltv_id="HBO HDTV [East]">HBO HDTV [East]</channel>
 
 
That's it.  Sounds long, but it really isn't bad. I apologize if you knew how to do this, but i just got into Webgrab++ and i know i was so confused when i first started.  I welcomed any help.
 
 
I didn't do that to TVPassport, because someone was kind enough to help me by giving me a channels list for TVPassport.  I ended up using TVGuide mainly anyway, because they provide Season and Episode info for most shows, and Passport does not.

also, tvguide.com_.ini and tvpassport_.com.ini are not supposed to have the "_" in them.  They are added when i upload them. Make sure to remove those underscores.

Here's my channels list from my city (Houston Texas) and DirecTV is the provider. If you're looking for the most common channels, they should be in there. Then you don't need to create your own list, just use mine.

The one i uploaded works (tvguide.com_.ini) just remember to remove the "_" so that it should be tvguide.com.ini

Uthall wrote:

Heya.....i copied the tvguide.com.ini to webgrab, and added a few channels from your xml to my webgrab xml.
I got an error tho
error downloading page: Error: SecureChannelFailure (The authentication or decryption has failed.)
pausing 1 of 4 times for 5 seconds before re-try.
Whant am i missing?

 
Are you using windows or Linux? if Win, make sure you have the latest .Net, if Linux, make sure you have the latest Mono. 

Uthall wrote:

Thanks Bud, updated mono and now OK.
Couple questions tho:
1. I get the following warning for each channel...still seems to work tho:
!! -- WARNING : tvguide.com doesn't allow epg grabbing !! it is advised to disable this channel / site from your channel list
2. The offset is 0000...which doesnt seem right for Houston.
Cheers

 
1. That warning is normal for some websites, it's fine.  Read the FAQs
 
http://webgrabplus.com/faq#faq_568
 
2. I set the timezone in the ini to UTC.  That way the time should adjust to anyone that uses it around the world

Uthall wrote:

No drama,
Ok, so its converting from Houston gmt-6 to utc0...ie adding 6 hours to all the times in the xml.

 
"No drama,"  Huh? What do you mean by that? Did you maybe misread my response? 
 
Not sure if the experts can explain it better, but +0000 will let the app/player use the local time and adjust accordingly.  Hope that makes sense. 
 
Once again, an expert probably can explain better. I'm still new to all of this myself.

u guys pretty much explained it.
fyi the timezone= setting in the ini has nothing to do with where the channels from or where the users grabbing the data lives.
this is set by the ini creator and is set to whatever the timezone the actual epg data is in.
just because when u view the epg page on their site and the data appears in your local time it doesn't mean that's the timezone the actual data is in,alot goes on behinds the scene in your browser.
and yes your epg viewing software should work similar by looking at the data start/stop time offset and your local time offset and doing the magic so that when you view the epg everything appears in your local timezone.

duplicate what?
log file or show me what u mean plse.

Uthall wrote:

lol.....no drama = no problem!
yea 0000 is fine in the xml, as long as the time in the xml has has 6 hours added to it to accommodate the local -6 Houston time.
BTW, I loaded all the channels in your xml file and got loads of duplicate warnings.....am i meant to cleans the file or something?

haha, ok.  Was so confused.
 
so you are copying all the channels you want from my DTV channels list onto the WG config file, yes? More than likely you have 2 channels in your WG config file that have the same
 
What also could happen is if you have MTV from like a UK source and  DTV happens to call that channel the by the same name, WG will spit that error.  You'll need to call one of them differently.
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:335,SourceId:1275" xmltv_id="MTV">MTV</channel>
 
<channel update="i" site="uk-sky.com" site_id="2501" xmltv_id="MTV">MTV</channel>
 
You can add  UK to the one from UK-Sky
<channel update="i" site="uk-sky.com" site_id="2501" xmltv_id="MTV UK">MTV UK</channel>
 
These are just edited for example sakes, they may not be the actual channel line.

jhr1948 wrote:

Uthall wrote:
lol.....no drama = no problem!
yea 0000 is fine in the xml, as long as the time in the xml has has 6 hours added to it to accommodate the local -6 Houston time.
BTW, I loaded all the channels in your xml file and got loads of duplicate warnings.....am i meant to cleans the file or something?

haha, ok.  Was so confused.
 
so you are copying all the channels you want from my DTV channels list onto the WG config file, yes? More than likely you have 2 channels in your WG config file that have the same
 
What also could happen is if you have MTV from like a UK source and  DTV happens to call that channel the by the same name, WG will spit that error.  You'll need to call one of them differently.
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:335,SourceId:1275" xmltv_id="MTV">MTV</channel>
 
<channel update="i" site="uk-sky.com" site_id="2501" xmltv_id="MTV">MTV</channel>
 
You can add  UK to the one from UK-Sky
<channel update="i" site="uk-sky.com" site_id="2501" xmltv_id="MTV UK">MTV UK</channel>
 
These are just edited for example sakes, they may not be the actual channel line.

Ok, yeah, if you past the entire channels list, you're bound to get errors. Plus that is like a 24 hour job. haha. Just choose the channels you want.

all these error as because of your wgconfig.xml
every channel line has to have a unique xmltv_id="xxx"
obv u have many that are the same.
these need to be unique to each channel as this is whats used by your epg software to map the epg to the correct channel.
having duplicates means this cant happen as it wont know what epg is the correct one.
so u need to edit your config.xml and fix this.
xmltv_id="xx" values can be anything you want,just make sure no 2 are alike.
usually this is set to your tvg-id="xxx" from your m3u file,this is how your epg software knows what channel the epg is for.

I couldn't quite follow how you guys resolved the message about the web grabbing warning for tvguide.com. I get the error, but it is no producing any channels after the process stops.

you cannot have 2 channel lines with same xmltv id. Try with one only and you will see that works.

Uthall wrote:

@jhr1948 ive found some channels i need arent in the xml you generated.
I assume that because they are part of maybe another region/zip?
Anyway, i tried generating a new channel list from LA but it just wont work......complains about the cookie file and dummy entry etc.
Can you just gimme the exact steps you followed the generate the xml....then i can just do my own when im missing channels?
EDIT: Just occurred to me, is the fact im trying this from outside the US part of the problem, and if it is, im running webgrab from ubuntu, so how would i solve it??

Hey, my first reply to you gave you the actual steps. But I did paste the wrong line as an example for step one. So fix my mistake and follow the rest of the steps.
 
Remember, I uploaded 3 ini files.
tvguide.com.ini
tvguide_PROVIDER.com.ini
tvguide_CHANNELS.com.ini
 
Put those three in separtate folder where ever is convinient, because they all need to be named:  tvguide.com.ini
 
Start with the Provider one, put that in the root folder of Webgrab, then change the preferred zip code where it says site_ini.  I messed up what you were suppposed to put in, it should look like this:
 
<channel update="i" site="tvguide.com" site_id="90001" xmltv_id="srvID generation">srvID generation</channel>
 
That will generate an xml file called:  tvguide.com.channels.xml
I would rename that, because the second step will generate an xml file with the same name and overwrite that one.
 
Follow the rest of the steps.
 
 
**I will point out, i'm having issues and it might be due to some of the modifications to the ini that I got help for.  Webgrab is taking long to grab becauuse i'm getting this frequently.
"[ Debug ] Can't match any (sub)detail title(s) with any index title(s):
[ Debug ] index title(s):"
 
I might stick try the original if i notice the same issue tomorrow morning.

Uthall wrote:

Ok, i think i answered my own question, and yes, i need to scrape the relevant provider.
So, can i merge the channels from houston with the channels from LA?
What happens to timezones if i do that?

You absolutely can combine all the channels on the same WG config.  My config is grabbing from US, Can, UK.
 
The timezone in the TVGuide ini (UTC) will be fine.  I think the UK (Sky) is also set to UTC.  Your player will use the local time.

Hey, i understand you are not from the US, so let me help out here. Some of the channels are in that Channels.xml i gave you (you might not be familiar with the name), while some might not be available in a cable/satellite lineup.  Personally, I just google a lot of stuff especially non US channels.  I know more about UK channels than i'll ever need to. haha. 
 
I assume you are using IPTV, sometimes they have the wrong name/country.  FYI, in US "East" channels are default.  If you find a channel is called "East" from your provider, the guide doesn't have to say "East".  On the other hand, if a US channel says "West" or "Pacific", you do need the "West" version.
 
Either way, here's some info on those channels.
 
Nbc golf channel (us) (that is in the xml i gave you, its just called the Golf Channel)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:218,SourceId:1185" xmltv_id="The Golf Channel">The Golf Channel</channel>
 
Big ten network (us) (also in the xml)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:610.3,SourceId:19694" xmltv_id="Big Ten Network [National]">Big Ten Network [National]</channel>
 
Cnn (us) (it spells out CNN)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:202,SourceId:429" xmltv_id="Cable News Network">Cable News Network</channel>
 
Cnbc world (us) 
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:357,SourceId:94447" xmltv_id="Consumer News &amp; Business Channel World">Consumer News &amp; Business Channel World</channel>
 
 
Nat geo (us) (spelled out)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:276,SourceId:9426" xmltv_id="National Geographic Channel">National Geographic Channel</channel>
 
Discovery science (us) (just have to know that's the nickname)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:284,SourceId:1164" xmltv_id="Science">Science</channel>
 
Hgtv east (us) (spelled out)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:229,SourceId:424" xmltv_id="Home &amp; Garden Television [East]">Home &amp; Garden Television [East]</channel>
 
Msg plus (us) (spells it out)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:635,SourceId:12498" xmltv_id="Madison Square Garden Plus HDTV">Madison Square Garden Plus HDTV</channel>
 
msg (us) (spells it out)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:634,SourceId:12497" xmltv_id="Madison Square Garden HDTV">Madison Square Garden HDTV</channel>
 
tlc east (us)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:280,SourceId:452" xmltv_id="The Learning Channel [East]">The Learning Channel [East]</channel>
 
 
buzzr (us) (local programming broadcasted in most major local cities. Just grabbed from from NYC, since they all show the same)
<channel update="i" site="tvguide.com" site_id="(srvID901078)Number:9.3,SourceId:62811" xmltv_id="WWOR Buzzr">WWOR Buzzr</channel>
 
a&e (us)
<channel update="i" site="tvguide.com" site_id="(srvID889041)Number:265,SourceId:426" xmltv_id="A&amp;E Network [East]">A&amp;E Network [East]</channel>
 
 
 
Not in the xml
Newsy (us) (pulled from Sling TV in Houston)
<channel update="i" site="tvguide.com" site_id="(srvID906640)Number:290,SourceId:83838" xmltv_id="NEWSY">NEWSY</channel>
 
 
Willow cricket(us) (pulled from Sling TV in Houston)
<channel update="i" site="tvguide.com" site_id="(srvID906640)Number:606,SourceId:44981" xmltv_id="Willow Cricket HDTV">Willow Cricket HDTV</channel>
 
 
Wwe network (us) (I use a line from TV Passport for Canada)
<channel update="i" site="tvpassport.com" site_id="wwe-network-hd-canada/14184" xmltv_id="WWE">WWE</channel>
 
 
Cbs news (us) - i looked that one up and its a app for TV Players, not going to be available in cable/satellite
 
 
setanta sports (never heard of it, quick google looks like its based in Ireland broadcasts in Russia & Ukraine.)
 
 
Sky news (us) (don't think a US Sky channel exists, I believe it is just Sky News from Europe/UK).
 
 
Phew! That was a lot

Uthall wrote:

Buddy that is brilliant, i owe you a beer. Virtual 6 pack coming your way......!!

Haha, cool man. YW

Uthall wrote:

Heya, i have noticed i get loads of warning like the below, and the grab seems quite slow. Anything to worry about?
Oh, and would you be so kind as to share me the tvpassport channel file?
"!! -- WARNING : tvguide.com doesn't allow epg grabbing !!
it is advised to disable this channel / site from your channel list"

 
I mentioned above that that warning can be normal, however, I'm also having some issues with that warning really and serious slowdownds.  Maybe we should bring it to a new topic about TVGuide.com getting many errors and serious slowdowns.  I've been told the ini is a little outdated, hopefully someone with the knowledge can look into the ini and see if TVGuide.com changed something that needs to be reflected in the ini.  Quite above my head.
 
What i've noticed is i have more errors (with TVGuide) and slowdown after about 9 or 10 pm (US Central time) until about 4 am.  I just ran my WG++ config right now (has a lot of TVGuide channels in it) and it breezed right through TVGuide. Not sure if that's a website issue.
 
TVPassport definitely works well, although i prefer TVGuide since it provides SxxExx info and TVPassport does not.I'm having trouble getting a TVPassport channels linup using a zipcode, so i'll share what someone shared with me quite a few months ago, you have to give me some time while i locate it.

Uthall wrote:

No drama, good to know its not just me then.
Hopefully some can update the ini as well, that would be great.
If you could find that tvpassport channel list that would be great. I cant seem to generate it.

 
Here is the channels list he gave me. Its not in any particular order, so you have to do some serious detective work. Also, there are a crap tons of locals in there.

I used UTC when i grabbed

Hi - Can you please post your .ini and xml config file here? I have been trying for a long time but couldn't get it to work. Even one channel from tvpassport will be fine. I just wanted to get it to work.

upload your webgrab log file.
sayin u cannot get it to work tells us nothing as to what the issue may be.

I seem to not be getting any data for Sling TV providers on tvguide.com. I produced a channel list with the correct channel line-up, but when I run the script for the channels, it doesn't show any data. I verify it against the website and there is clearly data for each channel.

I don't get any error messages and the program appears to execute the grab as expected. The major problem is, the XML channel file has no data at all. I am striking out with over 90% of the channels. There are some channels that produce one hour's worth of data.

But nonetheless, the website shows plenty of data. Any idea why it's not scraping the data during the grab?

First, thanks for posting this as the responses have been very helpful. I successfully got tvguide.com to work, but I haven't figured out the tvpassport yet. I tried following the same steps, but the guide.xml / channel list is all jacked and doesn't make sense. Is there a good tutorial on how to use the tvpassport.com.ini?

Thanks,
Will

First, thanks for posting this as the responses have been very helpful. I successfully got tvguide.com to work, but I haven't figured out the tvpassport yet. I tried following the same steps, but the guide.xml / channel list is all jacked and doesn't make sense. Is there a good tutorial on how to use the tvpassport.com.ini?

Thanks,
Will

xer01ne wrote:

First, thanks for posting this as the responses have been very helpful. I successfully got tvguide.com to work, but I haven't figured out the tvpassport yet. I tried following the same steps, but the guide.xml / channel list is all jacked and doesn't make sense. Is there a good tutorial on how to use the tvpassport.com.ini?
Thanks,
Will

I think i remember having trouble creating a channel list with TVPassport. The cool think is you can get the channel line easy with TVPassport. Look at this example:
 
<channel update="i" site="tvpassport.com" site_id="nbc-sports-northwest/4747" xmltv_id="NBCSN NORTH WEST">NBCSN NORTH WEST</channel>
 
All you have to do is go to the TVPassport website, enter the zip code, choose provider, and click on a channel. The URL will have the site_id.  For instance, that channel line above's URL is:
https://www.tvpassport.com/tv-listings/stations/nbc-sports-northwest-hd/4747
 
Notice "nbc-sports-northwest-hd/4747"is the site_id. You can put what ever you want as th xmltv_id

Following the tip above; If you want to find channels without having to search by zip/state/city on tvpassport you can use google to find it for you.

Examples (enter this on google search box):

pbs site:tvpassport.com
pbs ATLPBA site:tvpassport.com
pbs knoxville site:tvpassport.com

I'm in the same area as you and tried using this epg with plex and a hdhomerun. But when I try to select the channel name the only thing in the drop down is "my guide"