WebGrab+Plus/w MDB & REX Postprocess -- version V5.2.2.0
I am running my grabber on a RHEL9 VM machine (Redhat) and recently detected that two siteini's run into an issue.
Site 1)
FLIXED.IO
Issue:
[Error ] error downloading page: The archive entry was compressed using an unsupported compression method.
Site 2)
TVARENASPORT.COM
Issue:
[Error ] error downloading page: Decoder ran into invalid data.
Own observations:
I have tried to switch between compression methods gzip and br. Doesn't help.
I also tried to install brotli package from RHEL9 official RPM upstream source (dnf install brotli). Doesn't help.
Anything that I need to configure for this version or is it a bug?
Aree you sure the version is correct? can you post log and config please
Sure thing, I have blurred out my account details though. Send me a private message if you need more information.
Installation method is to unzip 5.2 version and then v5.2.2 (beta) in the same folder.
EDIT:
V5.2 works, but not 5.2.2 (beta)
I think you have 2 problems:
1. update your user-agent to:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36 Edge/18.18362 Firefox/75.0
2. Your flixed.io channel line in config belong to old version of flixed.io.
Please update wg++ and siteini.pack, then create new channel list and it will work see attached. (second is your channel line)
I have done changes as You asked, I have also made two different tests one with version 5.2.1 and one with 5.2.2.
Included are the results of the tests with version numbers. I think there might be some bug or issue with the latest beta release.
Please run in linux: dotnet --list-sdks
if you have:
root@ubuntu:/home/ubuntu/Desktop/.wg++# dotnet --list-sdks
5.0.408 [/usr/share/dotnet/sdk]
6.0.417 [/usr/share/dotnet/sdk]
7.0.404 [/usr/share/dotnet/sdk]
8.0.401 [/usr/share/dotnet/sdk]
If not update to dotnet 8
I only use dotnet-8.0 runtime not any SDK at all.
[root@epg-grabber-2 ~]# dnf list installed | grep dotnet
dotnet-host.x86_64 8.0.8-1.el9_4 @appstream
dotnet-hostfxr-8.0.x86_64 8.0.8-1.el9_4 @appstream
dotnet-runtime-8.0.x86_64 8.0.8-1.el9_4 @appstream
[root@epg-grabber-2 ~]# dotnet --info
Host:
Version: 8.0.8
Architecture: x64
Commit: 08338fcaa5
RID: rocky.9-x64
.NET SDKs installed:
No SDKs were found.
.NET runtimes installed:
Microsoft.NETCore.App 8.0.8 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
DOTNET_ROOT [/usr/lib64/dotnet]
global.json file:
Not found
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
have you tried other channels? Or is just with those 2 siteini you have the problem?
please also try the one i used https://github.com/SilentButeo2/webgrabplus-siteinipack/blob/master/eval...
Btw I would also install sdk
I tested with the version you suggested V5.2.1.5 and it works. It doesn't matter if I put one or more channels from these two siteini's. I have tried with some others but only these seem to have these issues with the latest beta 5.2.2 version.
I haven't tried to install the SDK, but it should work with only the dotnet8.0-runtime (it has worked for me this far with all other versions).
Good to hear that works.
Same here with the german tvdigital.ini (MinSWversion: V3.2.2.0
* @Revision 7 - [02/03/2023])
With Version 5.2.2.0 i got: error downloading page: Decoder ran into invalid data.
With Version 5.2.1.0 it works fine.
No dot.net SDKs installed
dot.net runtimes: Microsoft.NETCore.App 8.0.8 [/app/dotnet/shared/Microsoft.NETCore.App]
See post #8 Use 5.2.1.5
https://github.com/SilentButeo2/webgrabplus-siteinipack/blob/master/eval...
Works with 5.2.1.5
This issue persists with newest version 5.3 release as well on Linux. Could it be a bug?
I get the following errors still from version 5.2.2+ onward. I have also tried to change compression method from gzip to br with no luck:
( 65/85 ) FLIXED.IO -- chan. (xmltv_id=TNT HD(245)) -- mode Incremental
i
error downloading page: The archive entry was compressed using an unsupported compression method.
error downloading page: The archive entry was compressed using an unsupported compression method.
error downloading page: The archive entry was compressed using an unsupported compression method.
error downloading page: The archive entry was compressed using an unsupported compression method.
index subpage loop reached maximum number (8)
see log file for details
i
error downloading page: The archive entry was compressed using an unsupported compression method.
error downloading page: The archive entry was compressed using an unsupported compression method.
error downloading page: The archive entry was compressed using an unsupported compression method.
error downloading page: The archive entry was compressed using an unsupported compression method.
Unable to update channel TNT HD(245)
Generic syntax exception:
message:
no index page data received from TNT HD(245)
unable to update channel, try again later
Existing guide data restored!
Many channels are blocked at 27 sep since then, no update
Yes this is correct, I found a working site on flixed.io but it only works with versions prior to 5.2.2.
Also this one is having issues with the latest version 5.3 (Started from version 5.2.2)
EDIT: Whatever was done with version 5.2.1.5 is working great on Linux. Maybe have a look at the code and compare what changed?
( 1/2 ) TVARENASPORT.COM -- chan. (xmltv_id=Sport 1) -- mode Incremental
error downloading page: Decoder ran into invalid data.
error downloading page: Decoder ran into invalid data.
error downloading page: Decoder ran into invalid data.
error downloading page: Decoder ran into invalid data.
i
error downloading page: Decoder ran into invalid data.
error downloading page: Decoder ran into invalid data.
error downloading page: Decoder ran into invalid data.
error downloading page: Decoder ran into invalid data.
Unable to update channel Sport 1
Generic syntax exception:
message:
no index page data received from Sport 1
unable to update channel, try again later
No guide data to restore
for flixed.io and tvarenasports edit the ini and on the site {xxx} line change compression=gzip to compression=br
works for me on linux.
or..
remove the compression=xxx setting.
compression speeds up downloading.
The latest version 5.3 works when I remove the compression flag completely. My question is if this is expected function or if a bug/issue is introduced starting from version 5.2.2 onward?
Version 5.3.0.0 works for me if I delete the compression=br line in the german tvdigital.de.ini.
This is getting very frustrating with the updates.
I confirm, when you delete line conversion=br in version 5.3 there is no error: Decoder ran into invalid data.
Talking about tvdigital.ini
the copmression make dowloads faster(or should).
site data is in json format which is already fast to download.
this site download all channel data at once,why compression was added to settings.
i dont know what happened to latest relases about this.
for sure its a wg issue.
i will ask jan abtout it as i asked already.