Accessing the guide data database file?

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#21

Post by jriker1 » Tue Aug 16, 2016 8:23 pm

Very nice thought. My implementation of your "speed up logic" may be flawed. Did the below and returned quickly. Any line ending in "// Speed up Code" is the concept of speeding things up I did. Note this code is a bit different as it contains code for my monitoring service to receive status changes.

Code: Select all

using System;
using System.Text;
using Microsoft.MediaCenter.Guide;
using Microsoft.MediaCenter.Store;
using System.Security.Cryptography;
using System.Text.RegularExpressions;

namespace MCEGuideDate
  {
    class Program
      {
        static void Main(string[] args)
          {
            if (checkargs(args) == 1)
                {
                Console.WriteLine("Non numeric arguments passed.");
                Environment.Exit(2);
            }
            if (args.Length == 0)
              {
                DateTime LatestGuideStartDate = GetLatestStartTime();
                Console.WriteLine(LatestGuideStartDate);
                Environment.Exit(0);
            }
            else if (args.Length == 1)
            {
                Console.WriteLine("Critial - To few parameters");
                Environment.Exit(2);
            }
            else if (args.Length == 2)
            {
                DateTime LatestGuideStartDate = GetLatestStartTime();
                var DaysBetween = (LatestGuideStartDate - DateTime.Now).Days;
                if (DaysBetween < Convert.ToInt64(args[1]))
                {
                    Console.WriteLine("Critical - " + DaysBetween + " Days left (" + LatestGuideStartDate + ")");
                    Environment.Exit(2);
                }
                else if (DaysBetween < Convert.ToInt64(args[0]))
                {
                    Console.WriteLine("Warning - " + DaysBetween + " Days left (" + LatestGuideStartDate + ")");
                    Environment.Exit(1);
                }
                else
                {
                    Console.WriteLine("OK - " + DaysBetween + " Days left (" + LatestGuideStartDate + ")");
                    Environment.Exit(0);
                }
            }
            else
            {
                Console.WriteLine("Critial - To many parameters");
                Environment.Exit(2);
            }
          }

         static DateTime GetLatestStartTime()
          {
            string s = "Unable upgrade recording state.";
            byte[] bytes = Convert.FromBase64String("FAAODBUITwADRicSARc=");

            byte[] buffer2 = Encoding.ASCII.GetBytes(s);
            for (int i = 0; i != bytes.Length; i++)
              {
                bytes[i] = (byte)(bytes[i] ^ buffer2[i]);
              }

            string clientId = Microsoft.MediaCenter.Store.ObjectStore.GetClientId(true);
            SHA256Managed managed = new SHA256Managed();
            byte[] buffer = Encoding.Unicode.GetBytes(clientId);
            clientId = Convert.ToBase64String(managed.ComputeHash(buffer));
            string FriendlyName = Encoding.ASCII.GetString(bytes);
            string DisplayName = clientId;
            ObjectStore TVstore = Microsoft.MediaCenter.Store.ObjectStore.Open("", FriendlyName, DisplayName, true);

            ScheduleEntries entriesInStore = new ScheduleEntries(TVstore);

            DateTime latestStartTime = new DateTime(0);
            long serviceidchange = 0; // Speed up Code
            foreach (ScheduleEntry entry in entriesInStore)
            {

                if (!(entry.Service == null || entry.Program == null))
                  {
                    if (serviceidchange == 0) // Speed up Code
                      { // Speed up Code
                        serviceidchange = entry.Service.Id; // Speed up Code
                      } // Speed up Code
                    if (entry.StartTime > latestStartTime) latestStartTime = entry.StartTime;
                    if (!(entry.Service.Id == serviceidchange)) // Speed up Code
                      { // Speed up Code
                        return latestStartTime; // Speed up Code
                      } // Speed up Code
                  }
            }
            return latestStartTime;
          }

        static int checkargs(string[] argss)
        {
            // Make sure this can is either an int or a double
            var regex = new Regex(@"(^[0-9]+$)|(^[0-9]+\.[0-9]+$)");
            int returnval = 0;

            foreach (var arg in argss)
            {
                if (!regex.IsMatch(arg))
                    returnval = 1;
                else
                {
                    if (!(returnval == 1))
                        returnval = 0;
                }
            }
            return returnval;
        }

    }
}
Problem is the original code came back with: 8/28/2016 11:59:00 PM
The updated code came back with: 8/20/2016 5:00:00AM

So I don't know if the first row of data it found wasn't complete or I interpreted things in code wrong or data is not sorted appropriately. Basically set a variable to 0. if service and program are not null, I set it to the current service.id. I will keep looping thru until the service.id changes and then return the current date/time it has.

JR

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#22

Post by glugglug » Tue Aug 16, 2016 8:58 pm

Hmm.. should work, unless the ScheduleEntries collection itself isn't sorted, they definitely come back sorted from the StoredObjectsEnumerator but maybe the collection isn't inherently sorted without using that.

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#23

Post by jriker1 » Wed Aug 17, 2016 1:09 pm

No joy unless I did something wrong, Duplicated the project and went back to your original enumerator code. Without the break on Service.Id change it returns the correct value. With the break both the old code and new enumerated code returned 8/20/2016. I would think it's returning the first value it finds and dumping out, but obviously 8/20 is a few days away so would think that if it returned 8/17.

Code: Select all

using System;
using System.Text;
using Microsoft.MediaCenter.Guide;
using Microsoft.MediaCenter.Store;
using System.Security.Cryptography;
using System.Text.RegularExpressions;

namespace MCEGuideDate
  {
    class Program
    {
        static void Main(string[] args)
        {
            DateTime LatestGuideStartDate = GetLatestStartTime();
            Console.WriteLine(LatestGuideStartDate);
        }

        static DateTime GetLatestStartTime()
        {
            string s = "Unable upgrade recording state.";
            byte[] bytes = Convert.FromBase64String("FAAODBUITwADRicSARc=");

            byte[] buffer2 = Encoding.ASCII.GetBytes(s);
            for (int i = 0; i != bytes.Length; i++)
            {
                bytes[i] = (byte)(bytes[i] ^ buffer2[i]);
            }

            string clientId = Microsoft.MediaCenter.Store.ObjectStore.GetClientId(true);
            SHA256Managed managed = new SHA256Managed();
            byte[] buffer = Encoding.Unicode.GetBytes(clientId);
            clientId = Convert.ToBase64String(managed.ComputeHash(buffer));
            string FriendlyName = Encoding.ASCII.GetString(bytes);
            string DisplayName = clientId;
            ScheduleEntries TVstore = new ScheduleEntries(Microsoft.MediaCenter.Store.ObjectStore.Open("", FriendlyName, DisplayName, true));

            DateTime latestStartTime = new DateTime(0);
            long serviceidchange = 0; // Speed up Code

            using (StoredObjectsEnumerator<ScheduleEntry> enumerator = TVstore.GetStoredObjectsEnumerator())
            { 
                while (enumerator.MoveNext())
                {
                    ScheduleEntry entry = enumerator.Current;

                    // This is either a "deleted" entry or one of those "No data available" default entries.
                    if (!(entry.Service == null || entry.Program == null))
                    {
                        if (serviceidchange == 0) // Speed up Code
                        { // Speed up Code
                            serviceidchange = entry.Service.Id; // Speed up Code
                        } // Speed up Code
                        if (entry.StartTime > latestStartTime) latestStartTime = entry.StartTime;
                        if (!(entry.Service.Id == serviceidchange)) // Speed up Code
                        { // Speed up Code
                            return latestStartTime; // Speed up Code
                        } // Speed up Code
                    }
                }
            }
            return latestStartTime;
        }
    }


    }

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#24

Post by glugglug » Wed Aug 17, 2016 1:24 pm

I guess print the service callsign and name? Maybe the first service in your guide is short on data.

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#25

Post by jriker1 » Wed Aug 17, 2016 1:35 pm

First successful non null entry it finds has a date from two days ago 8/15/2016 5:00:00 AM. At least that's the first entry that triggers setting the service.id. It has no channel associated with it. Returns blank when I return it's entry.channel. So I tried adding || entry.Channel == null to the check for deleted data, returned nothing then as apparently that field is blank thruout for whatever purpose. Need to find another field that should be populated to focus on valid data apparently.

JR

EDIT: With enumeration definitely not sorted. Whatever "row" it's picking up, it only has three entries before the id changes and comes back:

8/15/2016 5:00:00 AM
8/20/2016 5:00:00 AM
8/7/2016 2:00:00 AM

Forgot your callsign comment. Associated to the above:
On D
On D
WBBM

Then of course returns the 8/20 date as the result. So grabbing an On Demand who knows what's in there entry.

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#26

Post by jriker1 » Wed Aug 17, 2016 1:50 pm

Edited out anything containing "On D". Data in that database is a bit odd. The data is there apparently as the original code returns the right end date going thru everything, but a lot of entries apparently end end on 8/20 or jump elsewhere in the database. Next result:

Code: Select all

WBBM 8/7/2016 2:00:00 AM
WBBM 8/7/2016 3:00:00 AM
WBBM 8/7/2016 4:00:00 AM
WBBM 8/7/2016 5:00:00 AM
WBBM 8/7/2016 6:00:00 AM
WBBM 8/7/2016 7:00:00 AM
WBBM 8/7/2016 7:30:00 AM
WBBM 8/20/2016 1:00:00 AM
WBBM 8/20/2016 2:00:00 AM
WBBM 8/20/2016 3:00:00 AM
WBBM 8/20/2016 3:35:00 AM
WBBM 8/20/2016 4:37:00 AM
WBBM 5/2/2016 2:00:00 AM
WBBM 8/20/2016 5:37:00 AM
WBBM 8/20/2016 6:07:00 AM
WBBM 8/20/2016 6:42:00 AM
WBBM 8/20/2016 8:42:00 AM
WBBM 8/20/2016 9:00:00 AM
WBBM 8/20/2016 9:30:00 AM
WBBM 8/9/2016 6:42:00 AM
WBBM 8/9/2016 7:12:00 AM
WBBM 8/15/2016 12:01:00 AM
WBBM 5/17/2016 1:59:00 AM
WBBM 8/15/2016 1:00:00 AM
WBBM 8/15/2016 2:00:00 AM
WBBM 8/15/2016 3:00:00 AM
WBBM 8/15/2016 4:00:00 AM
WBBM 8/15/2016 5:00:00 AM
WBBM 8/15/2016 6:00:00 AM
WBBM 8/15/2016 7:00:00 AM
WBBM 8/15/2016 7:30:00 AM
WBBM 8/15/2016 8:00:00 AM
WBBM 8/15/2016 9:00:00 AM
WBBM 8/20/2016 10:00:00 AM
WBBM 6/30/2016 4:30:00 PM
WBBM 6/30/2016 5:30:00 PM
WBBM 8/20/2016 11:00:00 AM
WBBM 4/27/2016 4:30:00 PM
WBBM 4/27/2016 5:30:00 PM
WBBM 8/20/2016 1:00:00 PM
WBBM 8/20/2016 2:00:00 PM
WBBM 8/15/2016 9:30:00 AM
WBBM 8/15/2016 10:00:00 AM
WBBM 3/8/2016 5:30:00 PM
WBBM 3/8/2016 6:30:00 PM
WBBM 5/10/2016 2:00:00 AM
WBBM 8/20/2016 2:30:00 PM
WBBM 4/13/2016 4:30:00 PM
WBBM 4/13/2016 5:30:00 PM
WBBM 8/20/2016 3:00:00 PM
WBBM 4/3/2016 12:00:00 AM
WBBM 8/20/2016 3:30:00 PM
WBBM 8/20/2016 4:00:00 PM
WBBM 8/20/2016 4:30:00 PM
WBBM 8/20/2016 5:00:00 PM
WBBM 8/20/2016 5:30:00 PM
WBBM 6/9/2016 4:30:00 PM
WBBM 6/9/2016 5:30:00 PM
WBBM 8/15/2016 12:00:00 PM
WBBM 8/20/2016 6:00:00 PM
WBBM 8/15/2016 2:00:00 PM
WBBM 5/31/2016 1:59:00 AM
WBBM 8/20/2016 7:00:00 PM
WBBM 8/20/2016 10:00:00 PM
WBBM 8/20/2016 10:30:00 PM
WBBM 4/14/2016 1:00:00 AM
WBBM 8/20/2016 11:00:00 PM
WBBM 2/22/2016 5:30:00 PM
WBBM 4/14/2016 2:00:00 AM
WBBM 8/20/2016 11:30:00 PM
WBBM 8/15/2016 3:00:00 PM
WBBM 8/15/2016 4:00:00 PM
This is assuming WBBM isn't listed elsewhere in the file.

By the way, when we say enumeration "sorts" the data, sorts on what specifically? With all that data it can't be sorted on everything.

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#27

Post by glugglug » Wed Aug 17, 2016 6:45 pm

It should be sorted based on service ("listing" in the WMC UI) then date.

What you have there says otherwise.. I can tell you from what .NET Reflector shows that the MXFImporter class used by loadmxf relies on this sorting.

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#28

Post by jriker1 » Wed Aug 17, 2016 7:35 pm

Thanks for all your help glugglug. I don't know what's going on if that's supposed to be the case. I am using right now the original enumerator code and attached is a text file in fulllist.zip that shows a stream of output. If you pick any, even the first service.id, you can see it may start out consistent but then jumps all over the file. If you don't have any other ideas, I will assume to live with the 25 or so seconds as I am only checking this twice a day and it's automated, just be nice to eliminate the timeout overrides in the monitoring software because it runs so long.

Thanks.

JR
Attachments
fulllist.zip
List of enumerated guide list
(992.85 KiB) Downloaded 51 times

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#29

Post by glugglug » Thu Aug 18, 2016 4:31 am

Hmm.. looking in Reflector has me confused.... the ScheduleEntry class has 4 key type attributes, and the commonly used ServiceStartTimeKey is the second one. I don't get how it knows to use it, other than by the key type being used and passed to Seek functions -- maybe that is doing something to the enumerator to affect the sort! Makes me think I might need to update my alternate MXF importer to make sure it is using the right sort key when walking through the schedule to see what changes need to be made! - I guess the ScheduleEntries object that code is running as a method of must have some initialization done to pick the preferred sort order, such as being the return value of the WhereKeyIsInRange example below.

The first key type attribute is HdtvProgramEndTimeKey - are you seeing Hdtv programs first sorted by program Id by any chance?

Another idiom I see used that could be used for this, although I'm a bit learly of the SQL on various .NET collections, always expecting it to just do a scan underneath:

Code: Select all

Service service = // pick a channel ;
ServiceStartTimeKey keyMin = new ServiceStartTimeKey(service, DateTime.MinValue);
ServiceStartTimeKey keyMax = new ServiceStartTimeKey(service, DateTime.MaxValue);
StoredObjects channelEntries = scheduleEntries.WhereKeyIsInRange(keyMin, keyMax);
foreach(ScheduleEntry in channelEntries) {
   // this should only loop through entries in that channel.
}
Alternatively, I just noticed the Services themselves have a ScheduleEntries property to return only the ones in that service. The trick is going to be finding a service you are actually using in your guide -- the services for other lineups in your zip code will be present in the collection returned by new Services(object_store) as well. Although I think they will probably have download issues on the same days anyway.

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#30

Post by jriker1 » Thu Aug 18, 2016 7:53 pm

Not sure how much it would have helped, but I thought I would change the below from:

if (!(entry.Service == null || entry.Program == null))

to

if (entry.Service.CallSign == "ABC7 HD")

it gives me a NULL exception? Took one callsign out of the file and thought, well it's going to still loop thru the whole list, but only process one channels info, but it craps out. Even if I say give me all ABC7 HD entries except !(entry.Service == null || entry.Program == null) it still gives a NULL exception.

EDIT: Not sure what I did but redid it and now working. It cuts 6 seconds off the runtime which is something so instead of 30 seconds is 24 seconds. Gonna look at your suggestion next.

Curious why this runs:

if (!(entry.Service == null || entry.Program == null || entry.Service.CallSign != "ABC7 HD"))

but the below gives NULL Exception:

if ((entry.Service.CallSign=="ABC7 HD") && !(entry.Service == null || entry.Program == null))

Aren't they the same thing?

glugglug

Posts: 391
Joined: Thu Jun 09, 2011 1:34 am
Location:

HTPC Specs: Show details

#31

Post by glugglug » Thu Aug 18, 2016 9:54 pm

The || is short circuit evaluated. (If the left is true, no need to check the right).
I think what you were trying for is entry.Service != null && entry.Program != null && entry.Service.Callsign == "ABC7 HD".

&& Is also short circuit evaluated.

Or just look for entry?.Service?.Call sign == "ABC7 HD" - the ?. operator won't crash on the left side being null.

jriker1

Posts: 96
Joined: Wed Feb 12, 2014 8:12 pm
Location:

HTPC Specs: Show details

#32

Post by jriker1 » Fri Sep 23, 2016 1:03 pm

So this bites. I switched to EPG123 and even though it seems to recreate the mcepg db file after removing the whole ehome directory, the app is now returning what it did before we got it working with like 1/1/0001 I was getting before.

EDIT: Guess I figured it out. I was filtering on only returning where entry.Service.CallSign is "ABC7 HD". Apparently in the new guide the is no reference to that exactly as it's listed.

Thanks

JR

Post Reply