Page 4 of 23

Re: Microsoft ending EPG for WMC in January

Posted: Mon Nov 04, 2019 5:57 am
by Space
The episode description for "Supergirl" in your snapshot is definitely from the Rovi/TiVo/Microsoft guide source, not from a Gracenote/zap2it source.

Re: Microsoft ending EPG for WMC in January

Posted: Mon Nov 04, 2019 6:49 am
by ferrellsl
garyan2 wrote: Mon Nov 04, 2019 5:02 am When you performed TV Setup, did you deny the program guide terms of service? If not, you got MS/Rovi. Comparing your Supergirl episode...

Here is what Zap2It says:
zap2it.PNG

Here is what Rovi says:
Rovi.PNG

Here is where it is important to deny the ToS:
It does look like my latest install of WMC pulled an initial guide update from Rovi, but I've disabled any further auto downloads, so I'll be curious to see what my guide looks after 2 weeks when there's no more data left from Rovi. I'll keep you posted on what does and doesn't get updated starting on the 17th. Hoping you're wrong but suspect that my guide will be much sparser on the 17th.

But Zap2It does provide the channel icons as well as the program thumbnails. Here's a screen shot of the channel icons in my cache folder:

https://www.dropbox.com/s/eblu1lhwzzm7n ... s.png?dl=0

Re: Microsoft ending EPG for WMC in January

Posted: Mon Nov 04, 2019 9:10 am
by Waynezo
ferrellsl wrote: Mon Nov 04, 2019 12:11 am
Waynezo wrote: Sun Nov 03, 2019 10:50 am Yes ferrellsl, Please post instructions. I have medium learning skills :)
OK, the instructions are pretty lengthy so I've put everything into a PDF file that you can download from here: https://www.dropbox.com/s/lczi5jllklmaf ... l.pdf?dl=0
Thanks for the detailed tutorial!

Re: Microsoft ending EPG for WMC in January

Posted: Mon Nov 04, 2019 6:26 pm
by garyan2
Last night I had enabled the option to download information from TMDB and TVDB and it did fill in quite a few gaps for categories and some artwork. At that point I would say it was completely functional... it did take a long time for that initial data gathering for just 26 stations and 14 days though.

It did, however cause WMC to crash when I tried to look at program details, but that may have been unique to my system/setup. I'll set up a new VM and try again.

Right now, I think it will be important for anyone else trying this to run TV Setup using zip code 00000. That way MS/Rovi won't be confusing things, and the registry will be set to allow the guide to be visible.

Re: Microsoft ending EPG for WMC in January

Posted: Thu Nov 07, 2019 8:03 pm
by Green Lantern
Firstly, thanks to everyone (particularly ferrellsl) for this worthwhile and helpful endeavor. Secondly, I presume all this old vs new guide data (and related workarounds such a re-running TV Setup using zip code 00000) will become a moot issue for the majority of people this guide is intended for... WMC users who come here after their guide stops populating. For anyone who doesn't implement this new/free method for at least 10-14 days after the Rovi data stops (and thus there is no more guide data in their system) it seems like there should be no old vs new issues, no need to re-run TV Setup (which many people won't have done in years and will likely find confusing), etc. In fact, would you agree that it's probably even be best to simply recommend that most people just wait 2 weeks past the cutoff (manually schedule any interim recordings if necessary) and then following the instructions provided? I'm thinking there will be far fewer problems that way.

Re: Microsoft ending EPG for WMC in January

Posted: Sat Nov 09, 2019 11:11 pm
by garyan2
Okay, I did some experimenting, as well as created my own program to convert a ZAP2XML XMLTV file into a proper MXF file. I squeezed every last drop out of it and you can see what I got in the below table. I would consider EPG123 the GOLD STANDARD for the complete experience, and the program I created (ZAP2MXF), I can honestly say, is the best ever created in the history of man kind for the ZAP2XML XMLTV file. :P
zap2xml_compare.jpg
As far as usability and functionality, something like ZAP2MXF with the output from ZAP2XML is perfectly fine... the "missing" items I would not consider critical though some may really need the HD flag, MPAA ratings, and rating advisories for better control of recording schedules and parental controls. Or some may just really demand more series images than is available.

I have reservations with EPGC concerning series recordings and New/Repeat recordings. I'm just not sure how robust it is and it really depends on the settings selected. Also, it appears to be a high risk to need to re-map all channels if you happen to add or remove an existing channel. I think that may be overcome if you setup your tuners in EPGC as well so it can give consistent unique channel IDs, but that makes the initial setup even more difficult.

The big question is, how much is FREE worth to you?

Re: Microsoft ending EPG for WMC in January

Posted: Sun Nov 10, 2019 1:07 pm
by Space
Wow, nice job!

Does the ZAP2XML file contain unique series and episode identifiers so that WMC can keep proper track of the series and episodes that have already been recorded? Are these identifiers gotten from zap2it or are the created dynamically and need to be kept track of locally?

Have you overcome the issues you eluded to with EPGC (series and new/repeat recordings) with ZAP2MXF?

Re: Microsoft ending EPG for WMC in January

Posted: Sun Nov 10, 2019 2:22 pm
by adam1991

Re: Microsoft ending EPG for WMC in January

Posted: Sun Nov 10, 2019 3:40 pm
by garyan2
Space wrote: Sun Nov 10, 2019 1:07 pm Wow, nice job!

Does the ZAP2XML file contain unique series and episode identifiers so that WMC can keep proper track of the series and episodes that have already been recorded? Are these identifiers gotten from zap2it or are the created dynamically and need to be kept track of locally?

Have you overcome the issues you eluded to with EPGC (series and new/repeat recordings) with ZAP2MXF?
The ZAP2XML XMLTV file contains the dd_progid episode_num system, which if you are not familiar is the TMS ID or Zap2It ID... in other words, exactly the same as what Gracenote/SD/EPG123 uses. In fact, the series IDs and episode IDs from EPG123 and ZAP2MXF (using ZAP2XML XMLTV file) are exactly the same. You could transition from EPG123 to ZAP2MXF and back without "breaking" that history or changing any recording requests.

EPGC, to me, is a bit of a mess when it comes to series IDs, program IDs, and movies. Definitely stay away from using the Windows Media Centre Output Series and repeats setting on 'Not used'. Original air dates are not used and program IDs are based on the channel and date/time of the transmission. This means every program is unique... even the same exact program being broadcast on 2 different channels at the same time would be considered different and unique. Series IDs are based on the program title and movies are included. So if you want to record "Charlie's Angels", you will record the 1976 series, the 2000 movie, the 2011 series, and the 2019 movie.

Using a different "Series and repeats" setting in EPGC appears that it may work for tracking repeat episodes, but I don't know how the unique ID is generated. The other options are "Use programme title to generate links" which I assume to be the episode title, and "Use broadcaster references to generate links". Both create a different 9-10 digit numeric value, but without understanding how they are created I don't know what the risks are.

BTW: You do know you still have a coupon for SD waiting for you, right?

Re: Microsoft ending EPG for WMC in January

Posted: Sun Nov 10, 2019 6:28 pm
by garyan2
garyan2 wrote: Sat Nov 09, 2019 11:11 pmAlso, it appears to be a high risk to need to re-map all channels if you happen to add or remove an existing channel.
I just verified that if the lineup changes in the XMLTV file in any way, added/removed channels or different order, it will be necessary to remap anywhere from 1 to all of the channels in WMC. There is no way to establish a consistent and unique ID for a channel in EPGC from an XMLTV file.

Way too much risk and I strongly discourage using EPGC unless/until it gets addressed.

Re: Microsoft ending EPG for WMC in January

Posted: Sun Nov 10, 2019 11:42 pm
by Space
Gary,

Thanks for the explanation, and once again, the great work you have done and continue to do to keep WMC alive!

...and, yes, I have not forgotten about that coupon and plan to take advantage of it soon, although I have a few hurdles to jump before I can get my system back in tip-top shape as explained here...

Re: Microsoft ending EPG for WMC in January

Posted: Mon Nov 11, 2019 5:09 pm
by tonywagner
garyan2 wrote: Mon Nov 04, 2019 6:26 pmRight now, I think it will be important for anyone else trying this to run TV Setup using zip code 00000. That way MS/Rovi won't be confusing things, and the registry will be set to allow the guide to be visible.
Neat trick!

I've just started testing, and I think the same effect is achieved by disabling/disconnecting internet during the setup (while accepting the guide ToS, regardless of zip code, and disabling WMC automatic downloads). But zip 00000 might be more graceful!

Re: Microsoft ending EPG for WMC in January

Posted: Mon Nov 11, 2019 5:19 pm
by tonywagner
garyan2 wrote: Sat Nov 09, 2019 11:11 pm Okay, I did some experimenting, as well as created my own program to convert a ZAP2XML XMLTV file into a proper MXF file.
Thanks for the report!

Any plans to share your ZAP2MXF program? I also found this one on GitHub:

https://github.com/raydouglass/xmltv-to-mxf

Once I changed the subchannel detection from dashes to periods, it generated a perfectly usable WMC guide too, although as the author notes, lacking categories and a few other items. Just wondering if I should work on more mods for that program, or wait for yours...

Re: Microsoft ending EPG for WMC in January

Posted: Tue Nov 12, 2019 2:04 am
by garyan2
tonywagner wrote: Mon Nov 11, 2019 5:19 pmAny plans to share your ZAP2MXF program?
I knew this question was coming and was buying time trying to think of how to respond. Right now, I would say if I do release it, it will be limited to using XMLTV files generated from ZAP2XML, and I will probably not include the ability to use any images from Zap2it.

Limiting to ZAP2XML is so I can ensure a quality and consistent output for WMC. It means I fully understand and support the source data... coming from Gracenote, I know how the series and episodes are identified. If I try to support another source, things can get out of hand and it will be another full time job just to make sure the XMLTV file that Uncle Bob in Nowheresville creates is supported.

In order to use images from Zap2it, they all need to be downloaded and cached on the local drive. I think this will put an undue burden on their servers especially since each request will be of a very large jpeg to be resized to a smaller one. When you add up a few thousand images for a few thousand WMC machines, that adds up. There is also the Zap2it site's Terms of Service that has some language saying we might be bad people if we do that. The images downloads would be iterative, but the initial investment will be large.

In regards to the xmltv-to-mxf program, I'm embarrassed to say I don't know how to run that to create an MXF file. I've never dealt with Java. I found out about it shortly after I brought EPG123 to the community by the author. I think he tabled that project once we saw how far EPG123 could go to address the need. If you could send me a sample, or tell me how to do it, I can tell you what is missing and where the weaknesses are.

Re: Microsoft ending EPG for WMC in January

Posted: Tue Nov 12, 2019 2:55 am
by tonywagner
garyan2 wrote: Tue Nov 12, 2019 2:04 am
tonywagner wrote: Mon Nov 11, 2019 5:19 pmAny plans to share your ZAP2MXF program?
I knew this question was coming and was buying time trying to think of how to respond. Right now, I would say if I do release it, it will be limited to using XMLTV files generated from ZAP2XML, and I will probably not include the ability to use any images from Zap2it.
That would be fine by me! I'm dealing with some metered-internet environments, so I don't even want images.
garyan2 wrote: Tue Nov 12, 2019 2:04 am In regards to the xmltv-to-mxf program, I'm embarrassed to say I don't know how to run that to create an MXF file. I've never dealt with Java. I found out about it shortly after I brought EPG123 to the community by the author. I think he tabled that project once we saw how far EPG123 could go to address the need. If you could send me a sample, or tell me how to do it, I can tell you what is missing and where the weaknesses are.
I ran it / compiled it on a Mac -- I'll see if I can make some Windows instructions and share something here. It works pretty well in WMC, but I can see its weaknesses already: it doesn't put the categories in the MXF, chiefly. Also the "xmltv_ns" season/episode numbers need to be converted properly (so 0.4 should become season 1, episode 5). And original airdates are mostly "Jan. 1, 1970" (ah, zero Unix time!). I was going to make those fixes but I'd gladly welcome a working non-Java solution instead!

Re: Microsoft ending EPG for WMC in January

Posted: Tue Nov 12, 2019 6:30 pm
by tonywagner
Here's that "xmltv-to-mxf" Java program (https://github.com/raydouglass/xmltv-to-mxf) with my fix for period subchannels:

https://file.io/ndLsGO

To run it, you need a Java JRE installed and then you can simply follow the instructions on the GitHub page above. But when I ran it on Windows with zap2xml input, I got a new error: "Invalid byte 2 of 3-byte UTF-8 sequence". Then the resulting MXF file can't successfully import into WMC either. But running it on my Mac works just fine (and presumably Linux would too), so presumably this is Windows character encoding weirdness.

If you want to edit/build it yourself, you'll need to install a Java JDK and Apache Maven: https://maven.apache.org/install.html

Once the Java JDK and Maven are installed, and you've set the environment variables as noted in that Maven link, you can just download and extract the "xmltv-to-mxf" code from GitHub and run these commands in the extracted folder:

Code: Select all

mvn clean
mvn package
That will put a snapshot jar file in the "target" directory, which you can then run with the same GItHub instructions as noted above. (Be sure to include my fix for period subchannels from the GitHub issues page!)

Re: Microsoft ending EPG for WMC in January

Posted: Tue Nov 12, 2019 8:38 pm
by Green Lantern
tonywagner wrote: Tue Nov 12, 2019 2:55 am
garyan2 wrote: Tue Nov 12, 2019 2:04 am
tonywagner wrote: Mon Nov 11, 2019 5:19 pmAny plans to share your ZAP2MXF program?
I knew this question was coming and was buying time trying to think of how to respond. Right now, I would say if I do release it, it will be limited to using XMLTV files generated from ZAP2XML, and I will probably not include the ability to use any images from Zap2it.
That would be fine by me! I'm dealing with some metered-internet environments, so I don't even want images.
Ditto. I also have no interest in the images. Nor does any other WMC user I know. So, very much hope you release your ZAP2MXF program. It would surely be just fine, as proposed, for many people. Thanks!

Re: Microsoft ending EPG for WMC in January

Posted: Wed Nov 13, 2019 3:36 pm
by DavidinCT
ferrellsl wrote: Sun Nov 03, 2019 4:07 am
EPG123 is definitely the easiest to set up and configure but like you, I like free stuff and I'm not afraid to take a little time to research alternatives, especially if they're cost free. I installed EPG Collector and it had a medium learning curve, but once configured properly it's rock solid. I had been planning to pay for a subscription to SchedulesDirect but now I will stay with Zap2It and zap2xml and EPG Collector. Once you have your zap2xml batch file sorted out, you just schedule it as a task in Windows to get an XMLTV file that can then be imported into EPG Collector. I've also created a scheduled task for EPG Collector to import the XMLTV file created by zap2xml and it takes care of the rest. If others are interested, I can write up some instructions for North American OTA users.
Cool, let me know the process, I'll look over it and try it, so we can work together just so there is a post here with it.

I know for sure, people who don't come here often will be freaking out because there is no guide data. I bet we see a lot more users here when it officially goes down !

Is Microsoft killing the TV feature on the Xbox one ? I thought that is why they changed WMC to ROVI because they were using it there.

So much for Windows 8.1 WMC support to 2023.

Re: Microsoft ending EPG for WMC in January

Posted: Wed Nov 13, 2019 6:43 pm
by tonywagner
DavidinCT wrote: Wed Nov 13, 2019 3:36 pmCool, let me know the process, I'll look over it and try it, so we can work together just so there is a post here with it.
If you read the rest of this thread, you'll see that the reports of EPG Collector being useful were likely premature. (Sounds like that user was still actually seeing Rovi data.)

I tried it out myself, but my local channels only provide a few hours worth of PSIP program data. (And EPG Collector needs access to a tuner to work, so it's not necessarily even feasible to run an update every few hours without running into watch/record conflicts.)

If your locals provide longer range PSIP program data, it might be feasible. But that seems like a big "if" in my experience.

Re: Microsoft ending EPG for WMC in January

Posted: Thu Nov 14, 2019 2:25 pm
by DavidinCT
tonywagner wrote: Wed Nov 13, 2019 6:43 pm
DavidinCT wrote: Wed Nov 13, 2019 3:36 pmCool, let me know the process, I'll look over it and try it, so we can work together just so there is a post here with it.
If you read the rest of this thread, you'll see that the reports of EPG Collector being useful were likely premature. (Sounds like that user was still actually seeing Rovi data.)

I tried it out myself, but my local channels only provide a few hours worth of PSIP program data. (And EPG Collector needs access to a tuner to work, so it's not necessarily even feasible to run an update every few hours without running into watch/record conflicts.)

If your locals provide longer range PSIP program data, it might be feasible. But that seems like a big "if" in my experience.
Sorry, I did step away from this thread for a bit and must of missed it. A TON of info here. I have a dual boot (with no startup menu), nicely scripted to select an option in a menu to boot to Windows 8.1, and back to 7 (my current WMC install). 8.1 can be my test.

I can try this on 8.1 and try to figure it all out. Anything can be ran via the task scheduler.

I normally ran ROVI updates every day (4 am), so day to day the guide was updated to the newest. I did notice on Simpson episodes that they would be a generic description on repeats and it would record as new. After updating the guide every day, it update it and showed it as repeats, so they would not record. It took me deleting almost 150 episodes to figure that one out.

So task scheduling every day to update the guide can be done but, a few hours, that would be headache...

I should have some time in the next 2 weeks to play with this. So WMC task list... Get Youtube working again, Get HDR movies playing in the movie library and trying to change to a free guide solution for testing.

Maybe it's time to move to 8.1...