Strange "BrainDead" issue

An evolving, supported alternative to Rovi
Forum rules
★ Download the latest EPG123 here: https://garyan2.github.io/ <> Setup guide here: https://garyan2.github.io/install.html
User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#41

Post by garyan2 » Fri Aug 05, 2016 7:18 pm

schottjy wrote:
garyan2 wrote:I would, absolutely. It looks like in some cases, rather than changing what program the schedule entry references, it overwrites the program. In my case, I saw a generic program get overwritten with a specific program. Results was all generic schedule entries for that series was listed as that specific episode.
OK, and you agree with glugglug that re-running Setup TV is the best way to reset the guide? I had previously used ObjectStoreRecoveryTask in the olden days...
A TV Setup should work no problem.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

schottjy

Posts: 45
Joined: Wed May 16, 2012 3:02 am
Location:

HTPC Specs: Show details

#42

Post by schottjy » Fri Aug 05, 2016 7:23 pm

Thanks Gary (and glugglug)!

schottjy

Posts: 45
Joined: Wed May 16, 2012 3:02 am
Location:

HTPC Specs: Show details

#43

Post by schottjy » Sat Aug 06, 2016 3:31 am

Well that was relatively painless and seems to have cleaned up the immediate issue. Series recordings and one-time recordings seem to be preserved and the guide logo images survived as well. I do have to disable some duplicate channels from the lineup since EPG123 doesn't seem to let me deselect them without also deselecting it's pair.

glugglug, how did you identify the number of errors you mentioned in your past message? For the time being, an app that would scan for these errors would provide some advance warning and rerun Setup TV.

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#44

Post by glugglug » Sat Aug 06, 2016 4:29 am

schottjy wrote: glugglug, how did you identify the number of errors you mentioned in your past message? For the time being, an app that would scan for these errors would provide some advance warning and rerun Setup TV.
In order to identify what needs to be fixed in the MergeScheduleEntries wrapper I'm working on after the built in MergeScheduleEntries logic, it first needs to figure out what's wrong, and I have it outputting debug info about that to the console.
Also, there are still a lot more than I thought.... not sure what is going on. Cleaned things up a bit and have it logging more detail to the tracelog instead of the console. It was scaring me showing 795 incorrect entries in the log, but I see what's happening now is the entries for shows that are in the past got garbage collected but are still in my MXF, and the adds for shows in the past end up not doing anything :-) This makes the current output pretty noisy.

Gary: I think I figured out why my changes are sometimes ending up in infinite recursion, and it's a bit disturbing, but should be easy to fix within EPG123 when you do the channel matching:
  • At the end of the MS version of MergeScheduleEntries which I'm wrapping, if anything was updated, it calls channel.NotifyScheduleEntriesChanged on every channel associated with the current service schedule being imported.
  • Channel.NotifyScheduleEntriesChanged then does the same for it's associated MergedChannel, passing a reference to itself so the MergedChannel knows which scanned channel triggered it.
  • MergedChannel.OnScheduleEntriesChanged has some complicated logic, towards the end of which is a call to CloneScheduleEntries from the service of the channel that notified it (identified by first parameter) to it's own associated service
  • CloneScheduleEntries calls ScheduleEntries.Add(List<ScheduleEntry>) in turn going back to MergeScheduleEntries, etc.
MergedChannel.OnScheduleEntriesChanged has a short-circuit exit to avoid this under either of 2 conditions:
1. The notifying channel is a user added channel (not the case because the channels were created by channel scan or CableCARD pseudo "scan")
2. The notifying channel has the same Service as the MergedChannel (or more specifically Service.Id matches). -- This is the case in the corresponding scanned channels created by WMC when you select a lineup, but not the case when using EPG123 currently.

As a side effect of the MergedChannel.OnScheduleEntriesChanged calling CloneScheduleEntries every time something gets updated, I am seeing a lot of duplicates of the problematic ScheduleEntrys in the db. I am not sure if this would be the case for normal EPG123 users or only because the alternate MXF Loader I am working on is processing updates which the built in logic would have skipped.

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#45

Post by glugglug » Sat Aug 06, 2016 6:09 am

My attempts to delete the dupes screwed something up and now even the regular loadmxf is failing with an access violation. Guess I'll have to redo TV setup on the test box and wait for some new changes to come in.

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#46

Post by glugglug » Sat Aug 06, 2016 3:15 pm

I think I figured out how the dupes came to be -- normal epg123 users should be OK.

The FindStartScheduleEntry method sometimes returning null or seeking one element off from where it intuitively seems like it should was making my logic think a ScheduleEntry was missing when it wasn't, creating a dupe.
The recursion through MergedChannel.CloneScheduleEntries caused that duplication to multiply.

Without that logic bug the recursion would not be infinite (until a stack overflow), because the inner call to MergeScheduleEntries would not call NotifyScheduleEntriesChanged as it had nothing left to fix.

User avatar
STC

Posts: 6808
Joined: Mon Jun 06, 2011 4:58 pm
Location:

HTPC Specs: Show details

#47

Post by STC » Sat Aug 06, 2016 3:21 pm

Yup, my thoughts exactly.... :wave:

...some things best left to those who can.
By the Community, for the Community. 100% Commercial Free.

Want decent guide data back? Check out EPG123

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#48

Post by garyan2 » Sun Aug 07, 2016 5:16 pm

garyan2 wrote:I would, absolutely. It looks like in some cases, rather than changing what program the schedule entry references, it overwrites the program. In my case, I saw a generic program get overwritten with a specific program. Results was all generic schedule entries for that series was listed as that specific episode.
Looks like I was being a bit of an alarmist on this. I was seeing some funny entries in the WMC trace log and thought it was related to the flags I was setting. That in combination to me looking up the wrong series on zap2it to "confirm" sent me down the wrong path. The same episode being shown over and over again on a channel was actually correct, and not something epg123 influenced/affected.

Either way, I've removed the beta version and continuing on right now without the flags. Might revisit later.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#49

Post by glugglug » Mon Aug 08, 2016 3:20 am

Some of the updates from my alternate mxf importer aren't sticking, and debugging it I'm seeing an interesting scenario repeatedly:

MXF ScheduleEntry shows a timeslot like 2:00pm-5:00pm
ScheduleEntries.FindStartScheduleEntry finds a ScheduleEntry from 2:00pm-7:00pm in the ObjectStore DB.
My code to find the set of entries overlapping the timeslot of the MXF ScheduleEntry that didn't match after the normal import is seeing there are actually BOTH the 2-5pm and 2-7pm versions in the DB. This is particularly happening a lot for Olympics coverage.

But when I attempt to find and delete all overlapping entries that may be causing issues like this, it is consistently corrupting the DB.

Another fun one:

Sometimes comparing the start time and end time of the MXF vs db ScheduleEntrys says there is a mismatch, when in fact there is not, at least not in the date time strings printed with 1s precision...

At first I thought: "If there is rounding error parsing the DateTime from a string, this rounding error should be consistent, so WTF is going on here..."
Then I realized, the MXF file only has a date/time for the 1st ScheduleEntry on each channel. The others just have a duration in the file, and it is populating the other start/end times by calculating from the durations during loading. This calculation can end up with different rounding error when the schedule changes or when the imported ScheduleEntries block starts at a different timestamp.

I am wondering if this rounding error could sometimes be preventing the updates from working properly, because the time slot overlaps that of an existing show by a fraction of a second? Unfortunately my code will need some major refactoring to account for this -- I think I need to iterate through all the shows in the db, while iterating through the ones in the array built from the MXF like the existing MergeScheduleEntries is doing, rather than trying to use FindStartScheduleEntry to find what should match each one.

I'm wondering if maybe the ScheduleEntries in the MXF should be in 1-day chunks like the data from SchedulesDirect in order to have rounding error in the time calculations be consistent between imports? And the last show of the day would end at Z23:59:59 instead of midnight to avoid overlap? Actually I forget, how are shows crossing the border between days handled in the SchedulesDirect feed?

webminster

Posts: 657
Joined: Tue Dec 20, 2011 11:05 pm
Location:

HTPC Specs: Show details

#50

Post by webminster » Fri Aug 12, 2016 1:01 am

Had a few problems recently with blocks of time having "To Be Announced" on my main HTPC. Test laptop with same config doesn't seem to exhibit problem, and has programming listed on those blocks. Looks like I'll have to break down and run a LiveTV setup this weekend to see if it resets whatever's gone sick...
-Alan

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#51

Post by glugglug » Fri Aug 12, 2016 1:05 am


webminster

Posts: 657
Joined: Tue Dec 20, 2011 11:05 pm
Location:

HTPC Specs: Show details

#52

Post by webminster » Fri Aug 12, 2016 2:20 am

Thanks, I'll consider giving it a test on the test machine first, but don't tend to put software I'm not really familiar with on the production box. I've followed along on the thread, but why do you think it'll be the cure?
-Alan

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#53

Post by glugglug » Fri Aug 12, 2016 2:32 am

That's the main point of this alternate MXF Importer -- to make the updates that the one supplied by MS skips so that your schedule completely matches what is in the MXF file.

Post Reply