You are here

Multithreaded grabbing

10 posts / 0 new
Last post
petike
Offline
petike's picture
Joined: 4 years
Last seen: 3 years
Multithreaded grabbing

Hi Guys,
what about adding multithreaded grabbing for different providers? It can speedup the grabbing process a lot. It could be configurable eg. like: threads for the same provider (default 1), threads total: (eg. 4 for default...)

WGMaker
Offline
WGMaker's picture
WG++ Team memberDonator
Joined: 4 years
Last seen: 1 day
Is the support helpful?
support us

Hi Petike,
 
I wish I understood your idea. Can you explain with a bit more details? Explain what you mean with multithreaded grabbing ?
Also, do you feel it is important to speed up the grabbing? For most user the grabbing runs unattended, either sheduled or manually started and doesn't need any user interaction after that. (Also, WG++ is already the fastest grabber there is! I believe .. !?)
 
Jan

petike
Offline
petike's picture
Joined: 4 years
Last seen: 3 years

HI Jan, 
of course webgrab+plus is the best and fastest ;-)
i'm a bit "hardcore" user, i'm grabbing almost 200 channels from different providers. My idea was to make grabbing parallelly running from different providers, eg. one from tvtv.de and in the same time 1 from port.hu...

doubleg
Offline
Joined: 10 months
Last seen: 4 months

I use this tool called wgmulti. it can read the webgrab config xml file and split the grabbing process by starting multiple instances of webgrab+plus.exe

 

Blackbear199
Offline
Joined: 2 years
Last seen: 7 hours

this looks very nice,i also see he has some other nice xmltv tools also.

this will make grabbing alot easier for some i'm sure.

 

Jay888
Offline
Donator
Joined: 1 year
Last seen: 1 month
doubleg wrote:

I use this tool called wgmulti. it can read the webgrab config xml file and split the grabbing process by starting multiple instances of webgrab+plus.exe

 

Great work..

 

Thanks dude!

 

 

WGMaker
Offline
WGMaker's picture
WG++ Team memberDonator
Joined: 4 years
Last seen: 1 day
Is the support helpful?
support us

Bright idea, thanks!

1NSdbZVbpZDX
Offline
Joined: 3 years
Last seen: 1 month

linux bash solution:

#!/bin/bash

# guide.xml
cd $HOME/Applications/wg++ && mono WebGrab+Plus.exe "$(pwd)" &
# guideWin.xml
cd $HOME/Applications/wg++/WG+2 &&  mono WebGrab+Plus.exe "$(pwd)" &
# directv.xml
cd $HOME/Applications/wg++/WG+3 && mono WebGrab+Plus.exe "$(pwd)" &
# tvgrabar.xml
cd $HOME/.wine/drive_c/ProgramData/ServerCare/WebGrab && mono WebGrab+Plus.exe "$(pwd)"

then merging all 4 *.xml resulting files onto final xml guide for further processing

tricxx
Offline
Joined: 2 weeks
Last seen: 7 hours

Hi, can someone want to explain me how to use wgmulti in webgrab 2.1 ?

Blackbear199
Offline
Joined: 2 years
Last seen: 7 hours

hmm its not that complicated.

download and put the wgmulti.exe and wgmulti.exe.config in the same folder as your webgrab++config.xml and webgrab.exe

edit the wgmulti.exe.config to suite your needs.

make sure you change this..

<add key="BuildConfigFromJson" value="true" />

to

<add key="BuildConfigFromJson" value="false" />

also read here..

https://github.com/harrygg/xmltv-tools/issues/4

 

Log in or register to post comments

Brought to you by Jan van Straaten

Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl