XMLTV duplicate channel handling and workaround

An evolving, supported alternative to Rovi
Forum rules
★ Download the latest EPG123 here: https://garyan2.github.io/ <> Setup guide here: https://garyan2.github.io/install.html
Post Reply
bilditup1

Posts: 21
Joined: Fri Oct 25, 2013 9:21 pm
Location:

HTPC Specs: Show details

XMLTV duplicate channel handling and workaround

#1

Post by bilditup1 » Fri Dec 04, 2020 8:41 am

Noticed some weirdness recently with my Plex DVR where some stations would disappear or would have no data. Sure enough these were the ones that, for whatever reason, are the same channel that show up under two different channel numbers. The way EPG123 deals with this right now is to create however many discrete channel listings as there are channels with that station ID (so in my case I have a few channels listed twice). From reading e.g. this post, it seems like the behavior used to be that all channel numbers were listed under as <display-name>s under the same root channel with the same unique station id, but that this didn't work consistently with the consuming apps--the first channel or the last channel would get picked up, but not both of them. I'm not sure if I'm experiencing problems because Verizon changed the channel line up for me so that more channels were duplicated, if you changed the behavior of EPG123, if the Plex devs changed how they process XMLTV again, or all of the above, but in any case, now what happens is that Plex either omits the channels entirely so that they don't even show up in the UI, or they'd show up, but with 'Unknown Airing' as far as the eye can see. So it looks like they do not know what to do with multiple channels that have the same unique ID. This is regardless of the fact that I only have one of each of the duplicate channels enabled in the Plex DVR UI.

I'm not sure if you want to give us the ability to toggle individual channels instead of individual station ids in the UI, or some other way such that we'd end up with just one channel with a unique channel ID (like a checkbox section that allows for, say, 'use only first', 'use only second', 'use both', though this assumes that duplicates never happen more than once), but it could be helpful. My guess is that nobody ever needs any station selectable by more than a single channel number. In the event the channel number the user had selected was dropped, a warning can show up in the log or something, and they'd have to make whatever configuration changes to their DVR app to make it work again. Otherwise, I guess I'll try to file a bug report with Plex and see what they make of it (fwiw, I also have emby but have not used their DVR and don't know how it would respond to this).

In the meantime regardless of what you decide I've devised a basic workaround using Cygwin, xmlstarlet, and Task Scheduler, which deletes any instance of a channel after the first one (the behavior I want). In case this helps anyone else, I made a short write-up, which I'll put in a separate post below.

bilditup1

Posts: 21
Joined: Fri Oct 25, 2013 9:21 pm
Location:

HTPC Specs: Show details

#2

Post by bilditup1 » Fri Dec 04, 2020 8:42 am

Automatic Channel Deduplication Guide (using cygwin, xmlstarlet, and Task Scheduler)

Note that after I finished setting this up, I found out xmlstarlet has a native win32 implementation, so you can technically just use a regular command line or powershell script and eschew a heavy installation of Cygwin if you don't already have it, don't want it, or are more comfortable with the Windows command line or with Powershell. The win32 version may be faster, too, but I doubt this will matter very much for most people today in practice. That said, the creation of the task in Task Scheduler, which is what the bulk of this little guide is about, will be identical except for the minutiae of how you invoke the action, so this may yet prove a useful read. Anyway, here goes...
  • Download and install Cygwin, selecting the xmlstarlet package during your installation (just search for it on the package selection screen and toggle it).
  • After your installation completes, take the following script and modify it per your needs:

    Code: Select all

    #!/usr/bin/env bash
    xmlstarlet ed --inplace -d  "/tv/channel[@id=preceding-sibling::channel/@id]" complete/Cygwin-style/path/to/xmltv
    
    • Double and triple-check this path; if it's wrong, nothing happens.
  • You then need to have it run right after the nightly epg123_update task completes. This can be accomplished like so:
    • Start Task Scheduler (search for it in the Start Menu).
    • Select "Create a New Task" from the menu on the right, and then, in the "Create Task" dialog that just showed up,
      • give your task a sensible name (I went with epg123_dedup) and description,
      • select the 'Run whether user is logged on or not' radio button, and
      • select the the "Do not store password" and "Run with the highest privileges" checkboxes.
    • Then, go to the "Actions" tab, and hit "New" from the row of buttons on the bottom of this dialog box.
      • The type of action should already be "Start a program", so leave this be.
      • Put the full path to your Cygwin bash.exe under "Program/script", and
      • Put the following under "Arguments":
      • Code: Select all

        -l -c "complete/Cygwin-style/path/to/script"
    • Go to the "Triggers" tab, and hit "New" from the row of buttons on the bottom of this dialog box.
      • From the 'Begin the task' dropdown, select "On an event".
      • Hit the 'Custom' radio button, and then
      • Hit the 'New Event Filter...' button that shows up to the right of it.
        • In the 'New Event Filter' dialog that just popped up, go to the 'XML' tab.
        • select the 'Edit query manually' checkbox under the greyed-out text box, and then click 'OK' to dismiss the warning about the Filter tab not working when you manually edit a query.
        • Enter this in the now-functional text box:

          Code: Select all

          <QueryList>
            <Query Id="0" Path="Microsoft-Windows-TaskScheduler/Operational">
              <Select Path="Microsoft-Windows-TaskScheduler/Operational">*[EventData[@Name='TaskSuccessEvent'][Data [@Name='TaskName']='\epg123_update']]</Select>
            </Query>
          </QueryList>
          This site has a helpful guide with screenshots that directs you where to put the above XML so that the task triggers properly, in case my own description was unclear.
  • That's it. Click 'OK' until all the dialog boxes are gone. Your custom task should be visible by selecting 'Task Scheduler Library' from the folder tree on the left.
    • You can test whether the script itself works by right-clicking your custom task and selecting 'Run', and you can test whether your task fires properly by instead running the epg123_update task (if you had an instance of epg123 running this won't work, so you may want to double-check and close it first) and looking at the results.
  • Might seem obvious, but, you will have to manually run this task or script if you do a manual update of epg123 using the 'Save and Execute' button inside epg123 itself, as the instance of the app that then runs wasn't created by the Task Scheduler, and there is no event created upon whose completion your custom event would then run.
Have not run into issues yet, and it does exactly what I need it to do (remove every instance of a channel except the first one) with zero guesswork. Importantly, it does not modify the original epg123_update task in any way, but still will never run independently of it. On my 4-core Haswell machine with the xmltv file on a good SATA SSD, it only takes a few seconds to complete (though it does use a good amount of RAM in the process, comparable to epg123 itself). I hope this is helpful to others who've had their channels inexplicably disappear recently!

User avatar
garyan2

Posts: 7438
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#3

Post by garyan2 » Sat Jan 30, 2021 5:51 pm

For the latest release (v1.6.1.3), this will no longer be needed. The xmltv file generated by epg123 will now be compliant with the DTD again and no longer have multiple channel elements with the same id.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io

bilditup1

Posts: 21
Joined: Fri Oct 25, 2013 9:21 pm
Location:

HTPC Specs: Show details

#4

Post by bilditup1 » Thu Mar 31, 2022 10:50 pm

garyan2 wrote: Sat Jan 30, 2021 5:51 pm For the latest release (v1.6.1.3), this will no longer be needed. The xmltv file generated by epg123 will now be compliant with the DTD again and no longer have multiple channel elements with the same id.
(I noticed last month that this had changed, and disabled the task, but forgot to update this thread--thanks for pointing it out!)

Post Reply