I have tried to optimize the cosmote.gr ini file, since I found out there is a JSON file we could download which includes all the data (and only the data) for a specific day! But the data there is sometimes superfluous (some shows have the same time - I think it must be duplicates) and this causes a strange (but understandable) effect in grabbing... the second show which has the same time is moved to the next day, replacing the normal show of that day and getting the schedule really mixed up! Is there anything that could be done, or do we have no luck and must revert to the html index file which is really enormous and per channel?
Brought to you by Jan van Straaten
Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl
OK, I've tried to find a workaround by using these commands:
scope.range{(splitindex)|end} ** scrub all data first. Do not put them in showsplit element, at first - scrubbing works only in index_variable_element, not global_temp_x, I don't know why ** index_variable_element.scrub {multi(debug)|{"ID":"'index_variable_element'",|{"articleId":|}|],"sort"} ** get the data in global_temp ** global_temp_1.modify {set(debug)|'index_variable_element'} ** find show start time in each show ** global_temp_2.modify {substring(debug type=regex)|'global_temp_1' "\"startTime\":\"(.*?)\","} ** remove duplicates in time and link them to the global_temp_1 element which contains all of the show info ** global_temp_2.modify {cleanup(debug removeduplicates span=1 keeplast link="global_temp_1")} ** copy the cleanedup shows to showsplit ** index_showsplit.modify {set(debug)|'global_temp_1'} end_scopeHowever, it gives me the following error at the removeduplicates command:
[ Debug ] Debugging information SiteIni [ Debug ] Element: GLOBAL_TEMP_2 [ Debug ] Modify [ Debug ] command & arguments : cleanup(debug removeduplicates span=1 keeplast link="global_temp_1") [ Debug ] Element value before operation: [ Debug ] 00:15|01:15|02:15|03:00|03:45|04:30|05:15|06:00|10:00|12:00|14:00|15:00|16:45|17:50|18:00|20:00| 21:00|22:15|23:15|00:15|01:15|02:15|03:00|03:45|04:30|05:15|06:00|07:00|10:00|11:00|12:15|13:15|13:15|14:15| 14:30|15:15|16:00|18:00|18:15|19:15|20:00|21:00|22:00|22:15|01:00|02:00|02:15|03:00|03:45|04:15|05:15|06:00| 07:00|10:00|11:00|12:00|13:00|15:15|16:00|17:45|18:00|20:00|21:00|22:00|22:15|00:15|02:00|03:00|03:45|04:15| 05:15|06:00|06:00|07:00|08:00|10:00|12:00|12:00|13:00|14:00|14:30|15:15|15:45|16:45|17:50|18:00|18:00|20:00| 21:00|22:00|23:45|00:45|01:15|02:00|03:00|03:45|05:15|06:00|10:00|12:00|14:00|15:00|16:45|17:50|18:00|20:00| 21:00|22:15|23:15|00:15|01:15|02:00|03:00|03:45|05:15|06:00|10:00|12:00|14:00|15:00|16:45|17:50|18:00|20:00| 21:00|22:10|22:15|23:15|00:15|01:15|02:00|03:00|03:45|05:15|06:00|10:00|12:00|14:00|15:00|16:45|17:50|18:00| 20:00|21:00|22:10|22:15|23:15 [ Debug ] Duplicates removed on index 85,76,71,31 [ Debug ] Linked element: global_temp_1 [Error ] Unable to update channel skai [Critical] See log file for details [Critical] Exception.Message: Το αντικείμενο δεν συμφωνεί με τον τύπο προορισμού. [Critical] Exception.StackTrace: σε System.Reflection.RuntimeMethodInfo.CheckConsistency(Object target) σε System.Reflection.RuntimeMethodInfo.InvokeArgumentsCheck(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) σε System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder,/ Object[] parameters, CultureInfo culture) σε System.Reflection.RuntimePropertyInfo.GetValue(Object obj, Object[] index) σε WGconsole.Scrub.RemoveDuplicateFromLinks(String[] linkedelements, String[] str_indices) σε WGconsole.Scrub.Edit(String element, String[] operation, String condition) σε WGconsole.Scrub.ScrubOperations(Dictionary`2 operations, String type, Boolean channelfile, Boolean next) σε WGconsole.Scrub.SplitIndex(String index, SiteIni ScrubStrings) σε WGconsole.Program.UpdateChannel(String strIndex, ChannelToUpdate Chan, XmlTarget xTarget) σε WGconsole.Program.ConsoleApplication(String[] args) [ Info ] Existing guide data restored!The Greek message reads as "The object does not match to the destination type".
Hi, first of all thanks for the help, and you were right that I am overthinking it some times... However after a lot of debugging I am starting to give up on using JSON (and it's a pity, since it is really faster)... Because the situation is more complex than we thought at first. There are superfluous shows all over the place, which aren't even always at the same time and I think the final schedule is determined using the following senario:
For example with the following data:
only shows 1, 3 and 5 make it through the final schedule... and if 4 had larger articleId than 3 (at least I think that's why) it would be 4 instead!
Pretty complex algorithm to handle...
I've tried to check on the javascript that creates the page (without really comprehending how it actually works) and it sure does a sort on start times and I 've seen some instances of variables called "old start", "old end" and "new start" and "new end" which is maybe a hint that I am right.
If I am indeed right, is there anything that could be done to use the JSON data? Or is it not worth the effort?
As for the episode info don't worry, I've got it covered and works all the times. I think it is typed by hand, so sometimes they type E and K in English and sometimes in Greek, that's why it doesn't work all the time.
I am attaching the two versions (my original edit and new with JSON-not working) for reference...
It seems very strange, but the grabbing of the JSON worked for me for all channels without using POST! And with the URL I sent you! I don't know why you had so much problems in getting it... (although, your file is technically more correct and as the web page expects it) ;-)
But my problem is with the data itself, and the unwanted shows. In the example I gave you, shows 2 and 4 shouldn't be in the guide and indeed they are not in the final web page (however they are inside the JSON data)!
For show 2 it is logical that it is left out because if it existed it would be on top of show 1 and 3...
But for shows 3 and 4 there is no apparent logic in what show is the correct one, because they are in exactly the same times... Sometimes it is the first one, sometimes it is the last one... I have researched it a little and came to the conclusion that it is the show with the higher articleId that it is the correct one.
Note that the unwanted shows may have entirely different titles and data (as well as times of course) than the correct ones. This isn't a duplicates scenario, at all!
Check data for channels skai (15/8 after 12:00am) and mega (I think 14/8 noon) for actual overlapping data...
Thanks a lot for the code you gave me... I have used it for this case and changed it a little to compensate for all the quirks of the site... For example, most of the times, the last show of every day finishes in another time than the first show of the next day... Usually this makes some gaps in the guide, but sometimes they overlap!!! So, everything gets messed up... (I have found it the hard way, by using a lot of debugging... and it takes a lot of time, since the index page is really huge and the preparation code takes its time as well...) I think it would be better, if the program could force the day and time for each show (in the details there is the full day and time reported...)
I think the final ini works correctly, so I attach it below. Tell me if you see any problems!