Fixing the sports plugin

Post Reply
User avatar
DavidinCT

Posts: 1556
Joined: Mon Feb 13, 2012 3:45 pm
Location:

HTPC Specs: Show details

#81

Post by DavidinCT » Fri Mar 17, 2017 3:20 pm

jachin99 wrote:After poking around some more, and asking a few questions, It looks like all guide data ends up in the mcepg database, weather that guide data is for the TV guide, Sports On now, or Sports On Later. Essentially, what happens is guide data is downloaded from an external source in a specific format (XML) on both ROVI, and EPG123, and using the LoadMXF.exe program, the data is populated into a database. I wanted to see if two databases could coexist in WMC but apparently they cannot, and if you try, WMC marks the older database as corrupt, and uses the newer, rebuilt database so I am thinking that all Sports related guide data comes from the .MXF files contained within sports. One of these Sports.MXF files works, and shows on now, while the other doesn't I GUESS. Looking through random posts around this forum, I came across this log file, and I can swear I've seen something similar elsewhere. This is just a small portion of it but it looks to me like WMC is still pointed to its old plugins, but cannot find any data in them. My next thought is to try populating some of the Sports.MXF files, and see if the On Later tab works. I also got a set back while I was changing things in my TV room WMC. I changed the permissions of the guide database, sports folders, sports strip, and Microsoft Netflix plugin to add full control, and they all dissapeared. I'm not sure if WMC does an integrity check before it loads files, or if changing permissions will alter the hash of a file but using all of the same methods I have previously used to install these did not work.

This is real PITA for me but it might also help me figure out exactly how to get them back. I'm wondering If I have to add the sports folders, and renable them in the registry. Either way, if someone sees that I am either getting closer or further away from a solution please feel free to speak up.

- Retrieving SportsSchedule.
2:14:08 AM - Processing SportsSchedule.
2:14:08 AM - Retrieving SportsSchedule-2.enc.
2:14:11 AM - Extracting SportsSchedule.mxf.
2:14:11 AM - Processing SportsSchedule.mxf.
2:14:11 AM - Processing of SportsSchedule is complete.
2:14:11 AM - Retrieving SportsV2.
2:14:12 AM - Processing of SportsV2 is complete.
2:14:12 AM - The SportsV2 package is already up to date.
This is a good point to find out... I forgot completely about this update that is done every few hours (shocked they didn't remove that when they killed off the Sports Strip)

Finding out where Microsoft is downloading the sports files from is the next step, then you can do some funky DNS to have it point to your local machine where the updated file is.

This is as long as it does not come from the same place as normal guide data, as that would break guide data...
-Dave
Twitter @TheCoolDave

Windows Media Center certified and WMC MVP 2010 - 2012

Space

Posts: 2838
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#82

Post by Space » Fri Mar 17, 2017 3:51 pm

Seems like there are few threads on here regarding similar things...

I just posted on the other thread regarding loading multiple MXF files in to WMC.

I don't think you would need to do any funky DNS redirection to get this working.

You just have to know the format of the various MXF files and once formatted and populated with proper data, load them in to the WMC database with LoadMXF.exe.

The key is knowing the syntax of the MXF file (it is XML, but you need to know the proper keywords and formatting to load different types of data) for loading the data in to the proper tables in the database.

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#83

Post by jachin99 » Fri Mar 17, 2017 4:45 pm

In terms of loading the MXF files, as long as I get a sports specific lineup from Schedules direct, shouldn't that file already be in the correct format? Then if I'm not mistaken, it would be a matter of running software that would pull that data from the schedule direct website, and creating guide data out of it that the sports plugin reads. EPG123 already does this with the regular guide, and all that is needed is to have EPG123 do the same thing for the sports strip.

rkulagow

Posts: 246
Joined: Sun Jul 19, 2015 1:04 am
Location: Schedules Direct

HTPC Specs: Show details

#84

Post by rkulagow » Fri Mar 17, 2017 4:58 pm

Since I don't run MCE, can someone give me more information on what the sports plugin does? We do have the capability of letting an application know when a sporting event is over, including the final score. Is that what the "sports strip" does?

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#85

Post by jachin99 » Fri Mar 17, 2017 5:08 pm

The sports strip would display sporting events that are on now, along with a sports news RSS feed in one tab, show you sporting events that are on later in another tab, and let you check scores, and player stats for in a third tab. This is just what i know about it but there might be more.

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#86

Post by jachin99 » Fri Mar 17, 2017 5:12 pm

It also had a notification service built into it. This, I believe, was the only part of the program that could be reached by external programs.

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#87

Post by jachin99 » Fri Mar 17, 2017 5:19 pm

Can it display a kind of end of day, week or conference summary where If I wanted to do something like see the results of every NCAA bracket game that was on last night, I could?

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#88

Post by jachin99 » Fri Mar 17, 2017 6:05 pm

If EPG123 can manipulate mcepg, than we probably can also. I know Loadmxf.exe loads the database and takes these parameters:

-s storeFilename (The filename of the data store. If this is not specified, the default data store is used.)
-i inputFilename (The name of the MXF file to load.)

Something I don't know a lot about is data stores. Are these database files, or tables within a database, or can it be both?
Is the data store this explanation refering to mcepg? How would I load the mxf file in a way where WMC recognizes it, and displays it?

Space

Posts: 2838
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#89

Post by Space » Fri Mar 17, 2017 6:35 pm

Have you looked at the contents of the MXF file produced by EPG123? You can see the data structure that is used for loading the guide data there. That file represents the entire guide.

There are other files that you can load in to the database with LoadMXF.exe such as files in:

\Program Data\Microsoft\eHome\mcepgX-X\backup\recordings

There will be files looking like: 20140917_055222

If you view these files in notepad or another text editor, you can see their format. These files can also be loaded in to the database with LoadMXF.exe
They are the MXF/XML representation of the scheduled TV series/shows that you have scheduled.

When you look inside these files, you will see how they are formatted. WMC understands the format and converts that data into a format that it can load into it's database.

Now, there may be other aspects of the MXF format that allow you to load data into different locations (tables?) in the database, and those tables may drive other aspects of WMC such as the Sports stuff, Internet TV data, etc.

The problem is that I don't think specifics of doing this was ever made public, due to Microsoft never intending third parties to make any changes to this information (unlike guide data, which they specifically published documentation for).

It is not clear that it is even possible to do this, however. The MXF files to change these other things may need to be signed/encrypted in order for them to be accepted by WMC, in which case it may not be possible to load anything other than stuff that came from Microsoft (except for stuff that they specifically allowed such a guide data updates and loading in the backups from the above folders).

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#90

Post by jachin99 » Fri Mar 17, 2017 6:47 pm

I kind of get the Guide XML file format but I'm wondering if the Guide data for sports was stored in mcepg or in another location. I am looking more towards opening up mcepg to see if there is more than one table in there. Where in your post prior to this you were unsure about your definition of tables, you are correct to best of my knowledge. How do you check to see if an MXF file has been signed. If they sign one then I bet they sign them all but programs like EPG123 can still change the guide database anyway. If they have to be signed with MICROSOFT'S private key, then that will make it more difficult but if they have to be signed with SOME key, then not so much, and a developer can make their own key pair and sign it. I'm guessing its signed with a private key from my limited knowledge of cryptography, and seeing files that contain a public key token in WMC. Maybe its possible to make a new mcepg file.

Space

Posts: 2838
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#91

Post by Space » Fri Mar 17, 2017 8:04 pm

I am just guessing that they may be signed, I have no idea if they are or aren't. However, I do know that the guide data downloaded from Microsoft (the MXF file) IS encrypted.

The reason that EPG123 can create MXF files that can be loaded in to the database is because Microsoft specifically designed it so that third parties could do that. You may also be able to load other data as well, I don't know. But that is the problem, there is very little information out there about how to do anything like this with WMC, and finding how to do it will probably prove difficult (if it is even possible).

User avatar
DavidinCT

Posts: 1556
Joined: Mon Feb 13, 2012 3:45 pm
Location:

HTPC Specs: Show details

#92

Post by DavidinCT » Fri Mar 17, 2017 9:39 pm

rkulagow wrote:Since I don't run MCE, can someone give me more information on what the sports plugin does? We do have the capability of letting an application know when a sporting event is over, including the final score. Is that what the "sports strip" does?
It's been a long time from when I used it but, as a sports fan, I used it a LOT when it was part of WMC...

I hunted on the web to see if I could find some images to give you a better idea on what it was about.

The SCORES, I think this is the MAIN thing we are looking for here. There are some other aspects of the "sports strip" but, some of it is about fantasy sports and the other is a On now (that I believe sort of still works). That showed a smaller TV box, with a strip of current games across the top, when you selected one, it would change the channel to that game.

would give sport type across the top and when you selected it would give you the scores for the day/current, like in this picture, there is no scores in the listed games, it would give the current record of the team Wins-Losses.

348888-sports.jpg
This will give you an idea on how it was formatted and how it worked...

The data came from Fox Sports at the time. The mxf file he refers to has some references to fox but, of course the data is not working at all. It was kind of tricky finding a working image of it :-\

Even youtube didn't have any videos on it that I could find...

and thanks for coming by to see if you can help !!!!
-Dave
Twitter @TheCoolDave

Windows Media Center certified and WMC MVP 2010 - 2012

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#93

Post by jachin99 » Sat Mar 18, 2017 1:16 am

I just tried to open mcepg with sql lite and it asked for an encryption key

User avatar
Crash2009

Posts: 4357
Joined: Thu May 17, 2012 12:38 am
Location: Ann Arbor, Michigan

HTPC Specs: Show details

#94

Post by Crash2009 » Sat Mar 18, 2017 4:10 am

I invited Big Screen to join the conversation.


http://bigscreenglobal.com/bgproduct.aspx?PID=BSWV2


support AT bigscreenglobal.com

Niall

Posts: 7
Joined: Tue Jul 05, 2011 2:57 am
Location:

HTPC Specs: Show details

#95

Post by Niall » Sat Mar 18, 2017 9:03 am

Crash2009 wrote:I invited Big Screen to join the conversation.
Thanks for the invite Crash ;)

You guys are definitely on the right track - the sports data is loaded in via MXF files into the EPG Database (at least this is how the official one was done). This is pretty much the mechanism for all the dynamic content - not just your EPG but stuff like the Internet TV channels etc. The mxf > db injection is basically just taking the supplied serialized objects (mxf xml) and mapping them into relevant tables. You can even create your own ones too and pick it up from a custom MCE plugin that could utilize that data ..

If you have the right bits + pieces there's a viewer app for the db (hint : I think it was available with one of the beta's and then removed from dist) - or if you have the correct encryption key you can get directly into the db programmatically using the various dll's as well. I was also in the past able to decrypt the incoming US mxf files (that come through as .cab downloads) - so you can see what the expected formats + object id's are for the Sports channels, Internet TV etc.

Personally I wouldn't be able to make anything - apart from the obvious (no market or customers for this stuff anymore) - the good free sources of sports data (like ESPN) can only be used strictly non-commercially and other paid live sports data is prohibitively expensive. That being said - if someone were to write an importer from the free ESPN api's to either directly be loaded into the EPG (via the dll's) or injected via mxf imports + do it completely for free (no ads etc) - then you would be able to utilize this legally.

Note also that for it to work - (from memory) the source Provider ID you would import the data under would need to match the ones used for the previous one (I think same as the primary MS EPG provider) - although it may be possible to change this in one of the config tables. (sorry - been a very long time since I've looked at this stuff).

NB: please don't ask me for any of these 'secret' bits mentioned above as I'd get in trouble for providing them - however if you look in the right places - well 'seek and ye shall (probably) find' .. :D

User avatar
Crash2009

Posts: 4357
Joined: Thu May 17, 2012 12:38 am
Location: Ann Arbor, Michigan

HTPC Specs: Show details

#96

Post by Crash2009 » Sun Mar 19, 2017 3:04 am

Niall wrote:
Thanks for the invite Crash ;)

You guys are definitely on the right track.

If you have the right bits + pieces
Thanks for the confirmation and the direction to find the right bits.

User avatar
DavidinCT

Posts: 1556
Joined: Mon Feb 13, 2012 3:45 pm
Location:

HTPC Specs: Show details

#97

Post by DavidinCT » Sun Mar 19, 2017 2:29 pm

Niall wrote:
Crash2009 wrote:I invited Big Screen to join the conversation.
Thanks for the invite Crash ;)

You guys are definitely on the right track - the sports data is loaded in via MXF files into the EPG Database (at least this is how the official one was done). This is pretty much the mechanism for all the dynamic content - not just your EPG but stuff like the Internet TV channels etc. The mxf > db injection is basically just taking the supplied serialized objects (mxf xml) and mapping them into relevant tables. You can even create your own ones too and pick it up from a custom MCE plugin that could utilize that data ..

If you have the right bits + pieces there's a viewer app for the db (hint : I think it was available with one of the beta's and then removed from dist) - or if you have the correct encryption key you can get directly into the db programmatically using the various dll's as well. I was also in the past able to decrypt the incoming US mxf files (that come through as .cab downloads) - so you can see what the expected formats + object id's are for the Sports channels, Internet TV etc.

Personally I wouldn't be able to make anything - apart from the obvious (no market or customers for this stuff anymore) - the good free sources of sports data (like ESPN) can only be used strictly non-commercially and other paid live sports data is prohibitively expensive. That being said - if someone were to write an importer from the free ESPN api's to either directly be loaded into the EPG (via the dll's) or injected via mxf imports + do it completely for free (no ads etc) - then you would be able to utilize this legally.

Note also that for it to work - (from memory) the source Provider ID you would import the data under would need to match the ones used for the previous one (I think same as the primary MS EPG provider) - although it may be possible to change this in one of the config tables. (sorry - been a very long time since I've looked at this stuff).

NB: please don't ask me for any of these 'secret' bits mentioned above as I'd get in trouble for providing them - however if you look in the right places - well 'seek and ye shall (probably) find' .. :D
Wow, thanks. No question down the right path but, a big step forward thanks to you.

Now the next challenge... correct encryption key or viewer app for the db. I wonder if this can be grabbed from documentation from the WMC dev kit. I am sure Microsoft has documented how this system works and how to import data.

Interesting that it was available on a beta, I wish I knew what build, I would install another VM and give it a shot, I might have a CD with 7600 someplace but, not sure if I have to go earlier.
-Dave
Twitter @TheCoolDave

Windows Media Center certified and WMC MVP 2010 - 2012

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#98

Post by jachin99 » Mon Mar 20, 2017 3:40 pm

Niall, thanks for keeping us headed in the right direction. If we could find that beta build, then this might get a whole lot easier. In the mean time, I'm thinking that because that database has to be decrypted, and sent through memory, the easiest way to find the key might be a memory capture. We can do this one of two ways that I know of. We can either do a live capture, or we can analyze the windows hibernation file.

About the tool I was attempting to use to view the database, SQL Lite, whatever version I downloaded asked for a password when i tried to open anything so I'll need to find a database viewer that has documentation or one that I am more familiar with. An observation about getting the sports strip itself back, If mcepg itself gets marked as corrupt, then I lose the sports strip, and a few other start menu entries, and I have only been able to get them back under a few specific circumstances so there they are:

1.Using the clean start method with EPG123, which deletes ehome. This will give you the sports strip back if windows sees your old database file has been corrupted
2.If windows sees the database as corrupt, running media center studio with media center studio fix is not enough. MCSFix has to be ran at startup, and then the sports strip may be restored through media center studio. I have not performed enough testing to confirm this but this method has worked for me once so far, and it would explain why some machines cannot restore it even with MCSFix, and MCS on an already running machine.

I didn't get any time to work on this over the weekend because I was busy tearing down my oven to clean it. Fun stuff!! :thumbup:

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#99

Post by jachin99 » Mon Mar 20, 2017 6:00 pm

To clear up this last post, for method 1, you still have to run MCSFix, and restore the sports strip with Media Center Studio.

jachin99

Posts: 1293
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#100

Post by jachin99 » Thu Mar 23, 2017 5:42 pm

Just an update on this, I have contact glugglug, who authored the guide editor, and it looks like his programs injects statements directly into the database, and he even made a standalone program just for loading mxf data into the guide but he hasn't been on the forum in a while, and I don't expect a response right away. He has posted the source code for both, and I have peeked around at it but i'm not to the point where I can put these kinds of statements together, and create something that works. I'm working a little bit everyday to understand C# but thats pretty slow going, and I have had to make a few changes on some of my WMC machines this has been taking up most of my free time.

After thinking about this a little bit, I don't see how this one database file that houses our TV guide would be any different than every other database in windows. In other words, the same injection statements that have worked on other MS Windows SQL databases might work on the guide. I have been poking around on other forums, and one person pointed me to the Windows Embedded Database engine, that is essentially a trimmed down version of any other SQL server. I cannot find a working instance of this on my machine at work but then i would think that stock windows 7 has to have a db engine somewhere because it needs to store information, such as file extensions, in a way that make a database the perfect choice. In terms of the Windows Embedded, I have found plenty of references to it on server machines, but I cannot find any examples of it running on something like Win 7 Home. What little I know about this DB engine is that cannot be accessed remotely, and it uses your regular Windows credentials to encrypt the database so, working backwords, I am guessing these same credentials could unlock it as well? In terms of a database viewer app, others have suggested I use SQL Server Management Studio Express, so maybe that same program will unlock the guide database?

After poking around some last night, I did find a key file in C:\Programdata\Microsoft\ehome\Cache labled MCDIndiv01_64key. I wonder if I can pipe that into mcepg.db through the powershell, and unlock it. I'm that great with powershell commands but it seems like something that could be easily learned.

Just posting a few thoughts, and hoping to keep this conversation alive. When I get my behaving correctly, and I have my sports strip back on my HTPC machine, I'll try to inject Sports.MXF into the guide through LoadMXF.exe and see what happens.

Post Reply