LoadXMF Cast and Crew

Global guide related discussion. All Windows versions.
Post Reply
User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

LoadXMF Cast and Crew

#1

Post by TinusTrotyl » Thu May 30, 2019 12:32 pm

I'm using LoadXMF.exe to import my EPG data into (Win7) MCE. Each time I run it, cast and crew info is getting duplicated, so after a while I end up with Actors, Directors, Hosts being listed 3, 4, 5,... times. Example:
directors
Steven C Miller
Steven C Miller
Steven C Miller
cast
Bruce Willis
Bruce Willis
Bruce Willis
Christopher Meloni
Christopher Meloni
Christopher Meloni
...
Is there any way to avoid this? Maybe command switches that can be set for LoadXMF, or a way to clear EPG data before loading the new XMF?

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#2

Post by garyan2 » Fri May 31, 2019 5:23 am

There are no switches with loadmxf for something like this. The only switches are below:

Code: Select all

 LoadMXF [-s storeFilename] [-v] [-i inputFilename]

 where:
   -s storeFilename   - the filename of the store.
                        If not specified, the default store will
                        be used.
   -v                 - verbose. Show progress during the load.
   -i inputFilename   - the filename of the MXF file to load.
and also "/?" and "-?".

What is your MXF source? If each time you import, the person entry has a different UID, then you will get this behavior.
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

#3

Post by TinusTrotyl » Fri May 31, 2019 7:25 am

I have written an app that combines several XMLTV sources into one XMLTV file, which I process with MCImportXMLTV-16d (http://mcefun.nrossen.dk) to get an MXF file. I found that MCImportXMLTV only handles Directors - not Actors or Hosts. I therefore post-process the MXF file with a separate app for additional Actors, Directors and Hosts that I obtain from mc2xml (http://mc2xml.awardspace.info), before it executes LoadMXF.exe.

MCImportXMLTV maintains a list of person IDs between calls - I guess I need to add the persons I add during post-processing to this list to maintain unique person IDs over time. I will try this, and let you know how that fans out.

Thanks for your message: I think you pointed me in the right direction.

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#4

Post by garyan2 » Fri May 31, 2019 2:44 pm

I checked out MCImportXMLTV last night and it appears to handle persons good enough. It doesn't bring over the roles, but that's just a nice to have.

XMLTV source (provided by EPG123):

Code: Select all

  <programme start="20190531020000 +0000" stop="20190531023000 +0000" channel="EPG123.42676.schedulesdirect.org">
    <title>A.P. Bio</title>
    <sub-title>Handcuffed</sub-title>
    <desc>When he's kicked off his Harvard phone plan, Jack is offered a chance to leapfrog onto Mary's on one condition -- he must help her break up with her scuzzy boyfriend; Anthony becomes Durbin's right-hand man.</desc>
    <credits>
      <director>Jennifer Arnold</director>
      <actor role="Jack Griffin">Glenn Howerton</actor>
      <actor role="Devin">Jacob McCarthy</actor>
      <actor role="Michelle">Jean Villepique</actor>
      <actor role="Stef">Lyric Lewis</actor>
      <actor role="Mary">Mary Sohn</actor>
      <actor role="Principal Durbin">Patton Oswalt</actor>
      <actor role="Miles">Tom Bennett</actor>
      <writer>Paula Pell</writer>
      <producer>Mike O'Brien</producer>
      <producer>Lorne Michaels</producer>
      <producer>Seth Meyers</producer>
      <producer>Andrew Singer</producer>
      <producer>Mike Shoemaker</producer>
      <guest>Aparna Brielle</guest>
      <guest>Nick Peine</guest>
      <guest>Allisyn Ashley Arm</guest>
      <guest>Jacob Houston</guest>
      <guest>Eddie Leavy</guest>
      <guest>Miguel Chávez</guest>
      <guest>Jacob Timothy Manown</guest>
      <guest>Marisa Baram</guest>
      <guest>Elizabeth Alderfer</guest>
      <guest>Brendan Jennings</guest>
      <guest>David Neher</guest>
      <guest>Charlie McCrackin</guest>
      <guest>Yuyao Deng</guest>
      <guest>Jaime Moyer</guest>
      <guest>David Gborie</guest>
    </credits>
    <date>20190530</date>
    <category>Series</category>
    <category>Sitcom</category>
    <language>en</language>
    <icon src="https://s3.amazonaws.com/schedulesdirect/assets/p14153924_b_h12_aa.jpg" width="960" height="540" />
    <icon src="https://s3.amazonaws.com/schedulesdirect/assets/p14153924_b_h3_aa.jpg" width="360" height="270" />
    <icon src="https://s3.amazonaws.com/schedulesdirect/assets/p14153924_b_v3_aa.jpg" width="270" height="360" />
    <icon src="https://s3.amazonaws.com/schedulesdirect/assets/p14153924_b_v5_aa.jpg" width="240" height="360" />
    <episode-num system="dd_progid">EP02695542.0024</episode-num>
    <episode-num system="xmltv_ns">1.9.0/1</episode-num>
    <video>
      <quality>HDTV</quality>
    </video>
    <audio>
      <stereo>dolby digital</stereo>
    </audio>
    <new />
    <subtitles type="teletext" />
    <rating system="USA Parental Rating">
      <value>TV14</value>
    </rating>
    <rating system="VCHIP">
      <value>TV-14</value>
    </rating>
  </programme>
MXF result from MCImportXMLTV:

Code: Select all

      <Program id="5" uid="!Program!s1_20190531020000" series="si5" title="A.P. Bio" episodeTitle="Handcuffed" description="When he's kicked off his Harvard phone plan, Jack is offered a chance to leapfrog onto Mary's on one condition -- he must help her break up with her scuzzy boyfriend; Anthony becomes Durbin's right-hand man." keywords="g1" halfStars="9" guideImage="i30">
        <ProducerRole person="p114" rank="1" />
        <ProducerRole person="p33" rank="2" />
        <ProducerRole person="p34" rank="3" />
        <ProducerRole person="p115" rank="4" />
        <ProducerRole person="p116" rank="5" />
        <DirectorRole person="p113" rank="1" />
        <ActorRole person="p106" rank="1" />
        <ActorRole person="p107" rank="2" />
        <ActorRole person="p108" rank="3" />
        <ActorRole person="p109" rank="4" />
        <ActorRole person="p110" rank="5" />
        <ActorRole person="p111" rank="6" />
        <ActorRole person="p112" rank="7" />
        <GuestActorRole person="p117" rank="1" />
        <GuestActorRole person="p118" rank="2" />
        <GuestActorRole person="p119" rank="3" />
        <GuestActorRole person="p120" rank="4" />
        <GuestActorRole person="p121" rank="5" />
        <GuestActorRole person="p122" rank="6" />
        <GuestActorRole person="p123" rank="7" />
        <GuestActorRole person="p124" rank="8" />
        <GuestActorRole person="p125" rank="9" />
        <GuestActorRole person="p126" rank="10" />
        <GuestActorRole person="p127" rank="11" />
        <GuestActorRole person="p128" rank="12" />
        <GuestActorRole person="p129" rank="13" />
        <GuestActorRole person="p130" rank="14" />
        <GuestActorRole person="p131" rank="15" />
        
        ...
        
      <Person id="p33" name="Lorne Michaels" uid="!Person!Lorne Michaels" />
      <Person id="p34" name="Seth Meyers" uid="!Person!Seth Meyers" />

      <Person id="p106" name="Glenn Howerton" uid="!Person!Glenn Howerton" />
      <Person id="p107" name="Jacob McCarthy" uid="!Person!Jacob McCarthy" />
      <Person id="p108" name="Jean Villepique" uid="!Person!Jean Villepique" />
      <Person id="p109" name="Lyric Lewis" uid="!Person!Lyric Lewis" />
      <Person id="p110" name="Mary Sohn" uid="!Person!Mary Sohn" />
      <Person id="p111" name="Patton Oswalt" uid="!Person!Patton Oswalt" />
      <Person id="p112" name="Tom Bennett" uid="!Person!Tom Bennett" />
      <Person id="p113" name="Jennifer Arnold" uid="!Person!Jennifer Arnold" />
      <Person id="p114" name="Mike O'Brien" uid="!Person!Mike O'Brien" />
      <Person id="p115" name="Andrew Singer" uid="!Person!Andrew Singer" />
      <Person id="p116" name="Mike Shoemaker" uid="!Person!Mike Shoemaker" />
      <Person id="p117" name="Aparna Brielle" uid="!Person!Aparna Brielle" />
      <Person id="p118" name="Nick Peine" uid="!Person!Nick Peine" />
      <Person id="p119" name="Allisyn Ashley Arm" uid="!Person!Allisyn Ashley Arm" />
      <Person id="p120" name="Jacob Houston" uid="!Person!Jacob Houston" />
      <Person id="p121" name="Eddie Leavy" uid="!Person!Eddie Leavy" />
      <Person id="p122" name="Miguel Chávez" uid="!Person!Miguel Chávez" />
      <Person id="p123" name="Jacob Timothy Manown" uid="!Person!Jacob Timothy Manown" />
      <Person id="p124" name="Marisa Baram" uid="!Person!Marisa Baram" />
      <Person id="p125" name="Elizabeth Alderfer" uid="!Person!Elizabeth Alderfer" />
      <Person id="p126" name="Brendan Jennings" uid="!Person!Brendan Jennings" />
      <Person id="p127" name="David Neher" uid="!Person!David Neher" />
      <Person id="p128" name="Charlie McCrackin" uid="!Person!Charlie McCrackin" />
      <Person id="p129" name="Yuyao Deng" uid="!Person!Yuyao Deng" />
      <Person id="p130" name="Jaime Moyer" uid="!Person!Jaime Moyer" />
      <Person id="p131" name="David Gborie" uid="!Person!David Gborie" />
      </Program>
For comparison, here is the MXF entry EPG123 creates:

Code: Select all

      <Program id="5" uid="!Program!EP02695542_0024" title="A.P. Bio" episodeTitle="Handcuffed" description="When he's kicked off his Harvard phone plan, Jack is offered a chance to leapfrog onto Mary's on one condition -- he must help her break up with her scuzzy boyfriend; Anthony becomes Durbin's right-hand man." shortDescription="Jack helps Mary break up with her scuzzy boyfriend; Anthony becomes Durbin's right-hand man." language="en" seasonNumber="2" episodeNumber="10" originalAirdate="2019-05-30" keywords="k6,k6001" season="sn2" series="si5" isSeries="true" hasExtendedCastAndCrew="true">
        <ActorRole person="p3" rank="1" character="Jack Griffin" />
        <ActorRole person="p4" rank="2" character="Devin" />
        <ActorRole person="p5" rank="3" character="Michelle" />
        <ActorRole person="p6" rank="4" character="Stef" />
        <ActorRole person="p7" rank="5" character="Mary" />
        <ActorRole person="p8" rank="6" character="Principal Durbin" />
        <ActorRole person="p9" rank="7" character="Miles" />
        <WriterRole person="p31" rank="8" />
        <GuestActorRole person="p11" rank="8" character="Sarika" />
        <GuestActorRole person="p12" rank="9" character="Marcus" />
        <GuestActorRole person="p13" rank="10" character="Heather" />
        <GuestActorRole person="p14" rank="11" character="Victor" />
        <GuestActorRole person="p15" rank="12" character="Anthony" />
        <GuestActorRole person="p16" rank="13" character="Eduardo" />
        <GuestActorRole person="p17" rank="14" character="Caleb" />
        <GuestActorRole person="p18" rank="15" character="Marissa" />
        <GuestActorRole person="p19" rank="16" character="Lynette" />
        <GuestActorRole person="p20" rank="17" character="Dale" />
        <GuestActorRole person="p21" rank="18" character="Dave" />
        <GuestActorRole person="p22" rank="19" character="Coach" />
        <GuestActorRole person="p23" rank="20" character="Yuyao" />
        <GuestActorRole person="p24" rank="21" character="Joyce" />
        <GuestActorRole person="p25" rank="22" character="Gary" />
        <ProducerRole person="p26" rank="1" />
        <ProducerRole person="p27" rank="2" />
        <ProducerRole person="p28" rank="3" />
        <ProducerRole person="p29" rank="4" />
        <ProducerRole person="p30" rank="5" />
        <DirectorRole person="p10" rank="7" />
      </Program>
      
      ...
      
      <Person id="p3" name="Glenn Howerton" uid="!Person!Glenn Howerton" />
      <Person id="p4" name="Jacob McCarthy" uid="!Person!Jacob McCarthy" />
      <Person id="p5" name="Jean Villepique" uid="!Person!Jean Villepique" />
      <Person id="p6" name="Lyric Lewis" uid="!Person!Lyric Lewis" />
      <Person id="p7" name="Mary Sohn" uid="!Person!Mary Sohn" />
      <Person id="p8" name="Patton Oswalt" uid="!Person!Patton Oswalt" />
      <Person id="p9" name="Tom Bennett" uid="!Person!Tom Bennett" />
      <Person id="p10" name="Jennifer Arnold" uid="!Person!Jennifer Arnold" />
      <Person id="p11" name="Aparna Brielle" uid="!Person!Aparna Brielle" />
      <Person id="p12" name="Nick Peine" uid="!Person!Nick Peine" />
      <Person id="p13" name="Allisyn Ashley Arm" uid="!Person!Allisyn Ashley Arm" />
      <Person id="p14" name="Jacob Houston" uid="!Person!Jacob Houston" />
      <Person id="p15" name="Eddie Leavy" uid="!Person!Eddie Leavy" />
      <Person id="p16" name="Miguel Chávez" uid="!Person!Miguel Chávez" />
      <Person id="p17" name="Jacob Timothy Manown" uid="!Person!Jacob Timothy Manown" />
      <Person id="p18" name="Marisa Baram" uid="!Person!Marisa Baram" />
      <Person id="p19" name="Elizabeth Alderfer" uid="!Person!Elizabeth Alderfer" />
      <Person id="p20" name="Brendan Jennings" uid="!Person!Brendan Jennings" />
      <Person id="p21" name="David Neher" uid="!Person!David Neher" />
      <Person id="p22" name="Charlie McCrackin" uid="!Person!Charlie McCrackin" />
      <Person id="p23" name="Yuyao Deng" uid="!Person!Yuyao Deng" />
      <Person id="p24" name="Jaime Moyer" uid="!Person!Jaime Moyer" />
      <Person id="p25" name="David Gborie" uid="!Person!David Gborie" />
      <Person id="p26" name="Mike O'Brien" uid="!Person!Mike O'Brien" />
      <Person id="p27" name="Lorne Michaels" uid="!Person!Lorne Michaels" />
      <Person id="p28" name="Seth Meyers" uid="!Person!Seth Meyers" />
      <Person id="p29" name="Andrew Singer" uid="!Person!Andrew Singer" />
      <Person id="p30" name="Mike Shoemaker" uid="!Person!Mike Shoemaker" />
      <Person id="p31" name="Paula Pell" uid="!Person!Paula Pell" />      
I don't know why MCImportXMLTV caches the person id, but it is not necessary. The id field only has to be unique to the file, not the database. The real id that must be unique and constant for all MXF files is the UID field. So if in your processing, you generate a file for Joe Shmoe (<Person id="p32" name="Joe Shmoe" uid="!Person!0123456789" />) and then the next file for Joe Shmoe (<Person id="p32" name="Joe Shmoe" uid="!Person!9876543210" />) then you would now double up all Joe Shmoes in your guide because the UID is different.

The WMC database does not see the two Joes as the same person since the UIDs are different. It is important to note that when you update the new file, it will not take away or replace any persons, it will just keep adding new persons. I'd take a look at your resulting MXF file(s) and make sure the person UIDs are consistent.
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

#5

Post by TinusTrotyl » Sun Jun 02, 2019 9:27 am

Mmm, I made sure I keep all person UIDs consistent over time, but I still see this behavior. Is there a way to read the person UIDs from the database directly (VB .NET, or C#)?

On another note, can EPG123 be used with a custom XMLTV file (i.e., without Membership to Schedules Direct)?

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#6

Post by garyan2 » Sun Jun 02, 2019 3:48 pm

To view the internals of the database, you can run epg123Client.exe with the -X switch (epg123Client.exe -x). There will be 2 available buttons to explore the database and to export the database to MXF. The export can take a pretty long time but once done can be sometimes easier to navigate.
dbTools.PNG
Actually, you might want to try using the EPG123 client to manually import your MXF file. It does a couple database indexing tasks after the import that might help and the client doesn't require a membership to Schedules Direct. Another thing to try would be to run "start /wait c:\windows\ehome\mcupdate.exe -uf -dbgc"... a database cleanup (dbgc) may straighten things out as well.
Last edited by garyan2 on Sun Jun 02, 2019 4:56 pm, edited 2 times in total.
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#7

Post by garyan2 » Sun Jun 02, 2019 4:02 pm

I realize I didn't answer your second question. So far, I've stayed away from adding the ability to import an XMLTV file simply because I will then be asked to do what you are doing to try and fill in more information. The quality of XMLTV files provided by the user will vary greatly so EPG123 would then be asked to make the data "perfect" by using other databases such as theTVDB and themoviedb. It will get out of hand quickly and no one will be completely satisfied. Keywords would probably be the most difficult to try and manage... and then there is the international aspect with different languages.
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

#8

Post by TinusTrotyl » Mon Jun 03, 2019 8:12 am

Excellent! Thanks.

epg123Client.exe -x: I clicked the button to export to MXF, and nothing happened (I must say I ran it as a normal user). From the log I found: "[ERROR] Unhandled thread exception caught from epg123Client.exe. message: Could not find a part of the path 'C:\Users\BlahBlah\Desktop\epg123_v1.2.12.0\output\mxfExport.mxf'". After I created a folder "C:\Users\BlahBlah\Desktop\epg123_v1.2.12.0\output\" it worked.

I'm now going through the resulting MXF file... I'll keep you posted.

User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

#9

Post by TinusTrotyl » Mon Jun 03, 2019 10:52 am

One thing I found is that the same person can be listed as !Person! and !MCPerson!:

Code: Select all

    <Person id="1800000002817B" provider="5000000002402" name="Gorden Kaye" wdsTimestamp="2019-06-02T15:01:54.9558879Z">
      <UId>!MCPerson!Gorden Kaye</UId>
    </Person>
    <Person id="1800000003A730" provider="5000000003002" name="Gorden Kaye" wdsTimestamp="2019-06-03T08:16:05.9931295Z">
      <UId>!Person!Gorden Kaye</UId>
    </Person>
But even with the !MCPerson! 'removed' (by starting from a clean DB, and doing a LoadMXF with persons as !Person!), I still have duplicate entries. I'll look further...

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#10

Post by garyan2 » Mon Jun 03, 2019 2:17 pm

Do you get duplicate entries when importing the same MXF file?

The MCPerson namespace is what Microsoft uses for the guide data they provide and I don't believe it should have been a factor in what you are seeing.

If you want, I can try to recreate your issue on a VM here, but I will need a couple of your resulting MXF files. What operating system are you running?
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

#11

Post by TinusTrotyl » Mon Jun 03, 2019 3:38 pm

I'm running Win 7 64-bit with SP1.

I tried again: I rebuilt the EPG database from within MCImportXMLTV, and verified I didn't have any EPG data in MCE. I then ran LoadMXF to load a particular MXF file twice, and ended up with double actors and directors. I checked the MXF exported from EPG123Client, and there were no !MCPerson! entries.

How can I get this MXF file to you? (please don't spend too much time on it - I'm feeling guilty already :oops: )

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#12

Post by garyan2 » Mon Jun 03, 2019 3:53 pm

No bother. I never back down from a challenge and enjoy it quite a bit. End of the day, we will all learn something we don't know currently.

I'll send you a PM with an email address.
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#13

Post by garyan2 » Mon Jun 03, 2019 6:07 pm

There are a couple of things I've noticed that are different between my MXF files and your example. As a whole, I don't see anything wrong with your MXF file, but there is something you can try quickly to see if it is the issue.

As a test, modify your MXF file to move the <People></People> section to be after the <GuideImages></GuideImages> section. The LoadMXF program loads the MXF file serially, so I'm wondering if when it hits the people at the end of the file it does it twice? The Microsoft sequence defines the referenced children before the parents.

Sequence of elements described by Microsoft:

Code: Select all

Keywords
KeywordGroups
GuideImages
People
SeriesInfos
Seasons
Programs
Affiliates
Services
ScheduleEntries
Lineups
Sequence of your MXF:

Code: Select all

Keywords
KeywordGroups
SeriesInfos
GuideImages
Services
Programs
ScheduleEntries (only the 1st one)
Lineups
ScheduleEntries (the rest)
People
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

User avatar
TinusTrotyl

Posts: 8
Joined: Thu May 30, 2019 10:24 am
Location:

HTPC Specs: Show details

#14

Post by TinusTrotyl » Mon Jun 03, 2019 9:16 pm

This seems to do the trick: I changed the order, and I've not seen duplication of cast and crew (with LoadMXF.exe).

You rock! Thanks soooo much!

User avatar
garyan2

Posts: 3491
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#15

Post by garyan2 » Mon Jun 03, 2019 9:52 pm

That is great! Glad I could help.
- Gary
Stomping out the Rovi disease, one media center at a time. http://epg123.garyan2.net

Post Reply