Feature Request - Lineup merging

An evolving, supported alternative to Rovi
Forum rules
★ Download the latest EPG123 here: https://garyan2.github.io/ <> Setup guide here: https://garyan2.github.io/install.html
User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#21

Post by garyan2 » Sun Mar 20, 2016 6:45 am

stuartm wrote:So I have both Cablecard and OTA tuners on the same HTPC, will this all work for both. How will lineup merging work when multiple tuner types with different lineups are involved? (still waiting for a non-busy weekend to dive into trying this stuff out)
It will work for both. You will have 2 lineups from SD to create a single lineup from epg123. If you will be using Lineup Selector to match channels and stationIds, pay special attention to my directions on the website.

Do not add channels that are missing.
You will press the Sync Now button twice ... once for "Scanned (Digital Antenna (ATSC))" and again for "Scanned (Digital Cable (CableCARD))"
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#22

Post by garyan2 » Sun Mar 20, 2016 7:44 pm

Has nobody been able to execute the program from this post? I'm desperate to know whether it is possible to use the native functions of WMC to automatch correctly.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

3rob3

Posts: 548
Joined: Sun Aug 28, 2011 8:48 pm
Location:

HTPC Specs: Show details

#23

Post by 3rob3 » Sun Mar 20, 2016 8:32 pm

Here you go (this is on a cablecard system):
Attachments
matchNames.zip
(81.82 KiB) Downloaded 70 times

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#24

Post by garyan2 » Sun Mar 20, 2016 10:10 pm

Well, that was disappointing ... I expect the only native automatch I am going to be able to do is ATSC. The channels must have been scanned to generate the matchName. Back to the original plan.

Do have a question. The program I asked to run uses the same mechanism that will be used to manually match the stations and channels. Could you guys (3rob3 & artm) tell me which OS you are running? I need to know why one of you could generate a file while the other couldn't.

Thank you.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

glorp

Posts: 369
Joined: Sun Sep 23, 2012 2:54 pm
Location:

HTPC Specs: Show details

#25

Post by glorp » Sun Mar 20, 2016 11:07 pm

Here's another run for you.

CableCARD only system.

Win7 x64
Attachments
matches.zip
(62.82 KiB) Downloaded 63 times

KenL

Posts: 60
Joined: Thu Mar 03, 2016 3:35 pm
Location:

HTPC Specs: Show details

#26

Post by KenL » Mon Mar 21, 2016 1:02 am

For all it isn't worth... for me it reported 698 entries like this:

Code: Select all

Lineup: Scanned (Digital Antenna (ATSC))
     DisplayChannelNumber: 11.1		OriginalNumber: 11	OriginalSubNumber: 1	Number: 11	MatchName: OC:11:1
     DisplayChannelNumber: 11.2		OriginalNumber: 11	OriginalSubNumber: 2	Number: 11	MatchName: OC:11:2
     DisplayChannelNumber: 11.3		OriginalNumber: 11	OriginalSubNumber: 3	Number: 11	MatchName: OC:11:3
     DisplayChannelNumber: 12.1		OriginalNumber: 12	OriginalSubNumber: 1	Number: 12	MatchName: OC:12:1
     DisplayChannelNumber: 12.2		OriginalNumber: 12	OriginalSubNumber: 2	Number: 12	MatchName: OC:12:2
...
Lineup: Scanned (Satellite)
     DisplayChannelNumber: 70		OriginalNumber: 70	OriginalSubNumber: 0	Number: 70	MatchName: DNTWRK
     DisplayChannelNumber: 71		OriginalNumber: 71	OriginalSubNumber: 0	Number: 71	MatchName: INFOM
     DisplayChannelNumber: 104		OriginalNumber: 104	OriginalSubNumber: 0	Number: 104	MatchName: INFOM
     DisplayChannelNumber: 217		OriginalNumber: 217	OriginalSubNumber: 0	Number: 217	MatchName: INFOM
     DisplayChannelNumber: 107		OriginalNumber: 107	OriginalSubNumber: 0	Number: 107	MatchName: COMEDY
     DisplayChannelNumber: 107		OriginalNumber: 107	OriginalSubNumber: 0	Number: 107	MatchName: COMEDY
     DisplayChannelNumber: 669		OriginalNumber: 669	OriginalSubNumber: 0	Number: 669	MatchName: DRAMA
     DisplayChannelNumber: 72		OriginalNumber: 72	OriginalSubNumber: 0	Number: 72	MatchName: PAID
     DisplayChannelNumber: 73		OriginalNumber: 73	OriginalSubNumber: 0	Number: 73	MatchName: BLUEHYW
     DisplayChannelNumber: 81		OriginalNumber: 81	OriginalSubNumber: 0	Number: 81	MatchName: ICTV
....
And in an unwanted surprise it apparently kicked off a Rovi update (first one for this db) which I attempted to cancel. But it loaded "28509683_ListingData.mxf" and bloated mcepg2-0.db an extra 100 megs or so. I mapped a Rovi channel with Guide Tool to check and sure enough 2 weeks of Rovi listings right next to my 21 days of SD gracenote.

In a strange twist this time, I don't notice anything amiss from the contamination. Searches initiated from "AMG" screens still find items both in and not in guide. This as opposed to the oddities I experienced when I first ran for a few days with a fully Rovi infested install of EPG123.

But why this data point may not matter much is because I have to "agree to terms" in order to get a functioning Colossus input from the MC Hauppauge software tuner. I spent hours trying and the only way to get valid sat channels in MC is to "agree" and let it successfully down load the zip code setup data. Then I can pull the ethernet and it will finish a full scan for sat channels without the Rovi listing download completing (in the background) and hitting the db.

I went 16 days without any Rovi listing downloads reaching C:\ProgramData\Microsoft\eHome\Packages\EpgListings. Everything working perfect. It seemed to only be doing client maintenance every 5 days during that time.

So do unmapped Rovi listing downloads matter beyond wasted bandwidth and db bloat? I think I will let this go for a while and see if I notice anything. The most obvious problem is I haven't yet found a way to complete a Colossus setup (offline) without the rovi zip code setup data successfully downloading as part of the process.

This may age it's way out of the db but it says the next update is scheduled in 2 days. Wish I could NOT agree and get 10,000 blank working sat channels but I suspect those must originate from something with the vendor setup software/drivers for the cable card tuner.
Last edited by KenL on Mon Mar 21, 2016 1:34 am, edited 1 time in total.

3rob3

Posts: 548
Joined: Sun Aug 28, 2011 8:48 pm
Location:

HTPC Specs: Show details

#27

Post by 3rob3 » Mon Mar 21, 2016 1:02 am

Hi. I am running Win7 64. I noticed in your file it didn't look like it matched anything.

rkulagow

Posts: 246
Joined: Sun Jul 19, 2015 1:04 am
Location: Schedules Direct

HTPC Specs: Show details

#28

Post by rkulagow » Mon Mar 21, 2016 1:05 am

If there's something that we can add to the lineup file that are basically "hints" or "aka"'s that will make lineup matching easier, please let me know. I understand that our upstream may call a station "WABC-DT" and on the raw cable scan the provider is calling it "WABC HD", but if we can determine if it's feasible to create that cross-reference it may may thinks simpler.

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#29

Post by garyan2 » Mon Mar 21, 2016 2:29 am

What I am finding out, is the best way to do the automatching is the same way Lineup Selector does it; the channel number. The ATSC scanned channels are easy and I will go ahead and use the native method, the scanned satellite you see is strictly the callsign, cableCARD is empty, DVB-x are totally different ... so channel number it is.

A benefit of working on this so far is that I found out what Guide Tool needed to make epg123 viewable in the right panel.:clap: I'll be working on the automatch further. Hopefully it will be ready soon.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#30

Post by garyan2 » Mon Mar 21, 2016 2:40 am

KenL wrote:And in an unwanted surprise it apparently kicked off a Rovi update (first one for this db) which I attempted to cancel. But it loaded "28509683_ListingData.mxf" and bloated mcepg2-0.db an extra 100 megs or so.
Sorry, I have no idea why that happened. All the program did was peek into the database and record the information you see in the file. I didn't modify anything or even "save" anything. I really hope it doesn't mess things up.

If you want to see if Rovi has their claws in there, check out the search categories. Also, if all your channels are still assigned to epg123, then I think everything else will look/work fine. Interested to see how this turns out.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

glorp

Posts: 369
Joined: Sun Sep 23, 2012 2:54 pm
Location:

HTPC Specs: Show details

#31

Post by glorp » Mon Mar 21, 2016 1:44 pm

garyan2 wrote:Sorry, I have no idea why that happened. All the program did was peek into the database and record the information you see in the file. I didn't modify anything or even "save" anything. I really hope it doesn't mess things up.
I noticed this AM I got 2 messages in my WMC Event Log at exactly the time this ran. The first message a Warning level from "mcstore"

Code: Select all

Recovery of guide listings and recording schedule is complete.
Followed immediately by an MCUpdate Information event doing package updates despite me having them shut off in download options:

Code: Select all

4:04:15 PM - Testing internet connection.
4:04:15 PM - Connection established.
4:04:15 PM - Retrieving Directory.
4:04:15 PM - Processing Directory.
4:04:15 PM - Processing of Directory is complete.
4:04:15 PM - The NetworkMetadata package is not available.
4:04:15 PM - Testing internet connection.
4:04:15 PM - Connection established.
4:04:15 PM - Retrieving ClientUpdate.
4:04:16 PM - Processing ClientUpdate.
4:04:26 PM - Retrieving ClientUpdate-2.enc.
4:04:26 PM - Extracting DefaultGeoSettings.mxf.
4:04:26 PM - Processing DefaultGeoSettings.mxf.
4:04:26 PM - Retrieving NetopWhitelist-2.cab.
4:04:26 PM - Extracting Whitelist.xml.
4:04:26 PM - Processing Whitelist.xml.
4:04:26 PM - Processing of ClientUpdate is complete.
4:04:26 PM - Retrieving MCEClientUX.
4:04:26 PM - Processing MCEClientUX.
4:04:26 PM - Retrieving dSM-2.cab.
4:04:27 PM - Extracting StartResources.dll.
4:04:27 PM - Retrieving Logos-2.cab.
4:04:27 PM - Extracting cbs.png.
4:04:27 PM - Extracting comedy.png.
4:04:27 PM - Extracting drama.png.
4:04:27 PM - Extracting movies.png.
4:04:27 PM - Extracting msn.png.
4:04:27 PM - Extracting msnbc.png.
4:04:27 PM - Extracting news.png.
4:04:27 PM - Extracting sports.png.
4:04:27 PM - Retrieving SMTiles-2.cab.
4:04:27 PM - Extracting SM_Netflix.png.
4:04:27 PM - Extracting StartMenu_ITV_Browse.png.
4:04:27 PM - Extracting StartMenu_ITV_Search.png.
4:04:27 PM - Retrieving UpdateableMarkup-2.cab.
4:04:28 PM - Extracting markup.dll.
4:04:28 PM - Processing of MCEClientUX is complete.
4:04:28 PM - Retrieving Broadband.
4:04:28 PM - Processing Broadband.
4:04:28 PM - Retrieving Broadband-2.enc.
4:04:28 PM - Extracting null-us_null_13_BBPkg.mxf.
4:04:28 PM - Processing null-us_null_13_BBPkg.mxf.
4:04:28 PM - Processing of Broadband is complete.
4:04:34 PM - Update successful.
4:04:34 PM - Next update at 3/25/2016 3:58:59 AM.

Sammy2

Posts: 1708
Joined: Fri Aug 24, 2012 7:35 pm
Location:

HTPC Specs: Show details

#32

Post by Sammy2 » Mon Mar 21, 2016 3:33 pm

garyan2 wrote:Looking for folks to help out with this. This is to automatically match tuned channels to the guide information using the built-in functionality of the media center store and the loadmxf import. I need samples from all possible transports as above. If this pans out, will be able to skip the Lineup Selector tool.

The program attached will create a text file which I would like you to zip up and PM it to me. Contents are simply some channel information and the matchname required for the automatch to work.

BOTH FILES IN THE 7ZIP FILE MUST BE EXTRACTED <= I'll have to learn how to reduce this into 1 file for epg123.

Thank you!
I will run this when I have a chance. Just saw it now.

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#33

Post by garyan2 » Mon Mar 21, 2016 5:01 pm

I've removed the file as there is no longer a need to gather information on matchnames. Also appears there was some uninteded consequences based on KenL and glorp's experiences above.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

KenL

Posts: 60
Joined: Thu Mar 03, 2016 3:35 pm
Location:

HTPC Specs: Show details

#34

Post by KenL » Thu Mar 24, 2016 2:59 am

KenL wrote:...it apparently kicked off a Rovi update... In a strange twist this time, I don't notice anything amiss from the contamination. Searches initiated from "AMG" screens still find items both in and not in guide. This as opposed to the oddities I experienced when I first ran for a few days with a fully Rovi infested install of EPG123.
Spoke too soon. Either matchNameExtractor facilitated a Rovi takeover of the channels it found, or more likely it revealed channels at risk from Rovi epg downloads in my unusual setup. Channels the tool didn't list weren't affected by Rovi.

Another pass of Lineup selector brought back EPG123 listings to afffected channels however there are signs of oddities from the Rovi contamination.

Back to the drawing board for either a way to produce a working Colossus input without "agreeing to terms" or failing that some other strategy to prevent Rovi epg downloads.

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#35

Post by garyan2 » Fri Mar 25, 2016 3:32 am

Have you tried to archive your latest lineup file in programdata\microsoft\ehome\mcepg2-x\backup\lineup? I have had some success in the past in starting fresh and using loadmxf to import the backup lineup file to restore all tuner functions and frequencies.

You could theoretically accept the TOS, setup all the tuners, archive the backup, rerun tv setup and denying the guide TOS, and importing the lineup backup file... then run epg123.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

KenL

Posts: 60
Joined: Thu Mar 03, 2016 3:35 pm
Location:

HTPC Specs: Show details

#36

Post by KenL » Fri Mar 25, 2016 7:04 pm

garyan2 wrote:...You could theoretically accept the TOS, setup all the tuners, archive the backup, rerun tv setup and denying the guide TOS, and importing the lineup backup file... then run epg123.
That worked!

So now I'm on the reservation, should be protected from Rovi and have an easy backup of the tuner/input/channels which apparently can be restored (offline) without Microsoft.

I suspect the Colossus input/lineup needs an initial lineup backup per installation but then should be forever for that hardware/tuner/lineup combination.

Post Reply