Introducing EPG123

An evolving, supported alternative to Rovi
Forum rules
★ Download the latest EPG123 here: http://epg123.garyan2.net <> Setup guide here: http://epg123.garyan2.net/downloads/epg123_Guide.pdf
User avatar
garyan2

Posts: 5185
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

Re: Introducing EPG123

#1801

Post by garyan2 » Fri Mar 20, 2020 2:09 am

glorp wrote:
Thu Mar 19, 2020 11:55 pm
I'm glad to see that the cache will finally be a single file. That should substantially improve the disk thrashing. Are there any instructions or concerns about upgrading? What happens to the tens of thousands of old cache files on an upgraded system and what do you suggest as memory requirements in order to see any performance gain?
Nothing special to do or any concerns. It's just the first time you run the new version, it will take a bit longer to create the new cache file and delete all the old ones.

Memory requirements will certainly be dependent on the number of stations and days you are downloading. With 680 stations and 15 days, with no cached files and creating an XMLTV, memory usage got up to 1650MB for my tests. The same number of stations and days, with all data cached and no XMLTV, required 870MB.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

glorp

Posts: 354
Joined: Sun Sep 23, 2012 2:54 pm
Location:

HTPC Specs: Show details

#1802

Post by glorp » Fri Mar 20, 2020 4:45 pm

Yup got rid of ~60,000 cache files for one ~140K JSON with the first run this morning. That should speed things up going forward and reduce disk thrashing substantially. Nice job. Didn't really pay attention to memory use since it's being run on the headless server but that has 16GB available so shouldn't be an issue.

glorp

Posts: 354
Joined: Sun Sep 23, 2012 2:54 pm
Location:

HTPC Specs: Show details

#1803

Post by glorp » Sun Mar 22, 2020 9:28 pm

My task run times have fallen from 12-15 minutes to under 2.

User avatar
garyan2

Posts: 5185
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#1804

Post by garyan2 » Sun Mar 22, 2020 10:29 pm

Yah, I really wish I had done this a long time ago. I had always imagined I'd have to learn to use SQL/SQLite or something, but once I sat down and really looked at it, my solution was simple and the efforts paid out in spades. It's also nice to have a single file that I can search through to find info for troubleshooting if need be.

I'm now researching a little into how to compress the file, but I think it might take too much memory (serializing the data to a memory stream, to then send to a compression stream and to a file stream), but I could probably get the cache file to 15% of its' current size with normal compression.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

User avatar
garyan2

Posts: 5185
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#1805

Post by garyan2 » Mon Mar 23, 2020 2:16 am

All right, I've got compression going on. The table below says it all. This is with maximum compression. I'm going to try and benchmark on some slow systems to see if there are any side effects.
cacheImprovements.PNG
cacheImprovements.PNG (6.85 KiB) Viewed 611 times
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

Space

Posts: 2004
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#1806

Post by Space » Mon Mar 23, 2020 2:42 am

Personally I think it's fine the way it is. Not sure disk space is a huge concern for most people. The only reason I mentioned the disk space was due to the slack space and the efficiency aspect. The actual amount of space it took up was of little concern to me.

Plus I also like the idea that I can look through the cache file, although I probably will never do it again.

I'd rather have it run fast then save some disk space, but if it is a negligible increase in execution time then I guess it's fine.

Note that ages ago I did some tests with different compression algorithms for text (not binary) data and the ones that are better at text compression can be orders of magnitude better then generic compression algorithms that are designed for binary data.

Although, like I said, that was years ago, so compression may have improved since then.

EDIT:
Just looked back at my old test results and the best algorithm for compressing text at that time was PPMd with 7zip. Still looks like this is one of the best ones as far as speed and compression.

https://www.quora.com/What-is-the-best- ... -algorithm

Here's one of the notes I took:
gzip 16.697% of original size
7zip 2.727% of original size

I believe that gzip was using Lempel-Ziv and 7zip was using PPMd.

User avatar
IT Troll

Posts: 801
Joined: Sun Nov 27, 2011 9:42 am
Location: Edinburgh, UK

HTPC Specs: Show details

#1807

Post by IT Troll » Tue Mar 24, 2020 7:34 am

Oh interesting, I’ll have to get upgraded. I am guessing those running from SSD will not see much difference though?
Are you a Recorded TV HD user or want to give it a try? Check out the new community-made update; Recorded TV HD v2.1.1

User avatar
garyan2

Posts: 5185
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#1808

Post by garyan2 » Tue Mar 24, 2020 3:00 pm

Not quite as much, but I did see a difference. For an update run that was typically around 22s, I am now around 16s with the .20 version. This is on a circa '16 Mushkin SSD.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

npradeepkumar

Posts: 16
Joined: Sat Nov 26, 2011 10:00 am
Location:

HTPC Specs: Show details

#1809

Post by npradeepkumar » Mon Aug 17, 2020 9:54 pm

I have a question. Will this work with Windows 10, if I buy the media center pack? I had this up and running and was working fine, but recently my machine crashed and I had to re-install the OS and start over. Since Windows 7 is officially not supported anymore, it has been throwing hurdles every step of the way with lots of other software. So, I am debating if I should bite the bullet and switch to Windows 10.

I don't know if there are other issues or gotchas that I need to be aware of (if I am switching to Windows 10).

Thanks

stuartm

Posts: 481
Joined: Mon Nov 05, 2012 8:05 pm
Location: Longmont, CO

HTPC Specs: Show details

#1810

Post by stuartm » Mon Aug 17, 2020 11:27 pm

Be aware that Windows 10 does not support WMC, it will only work with a fair bit of hacking on your part. Also AIUI only OTA works, I don't think cable card will work with the win10 hacks (and definitely not any protected content). There is a whole subforum about WMC and win10 here viewforum.php?f=97. That said, if you go that route EPG123 does work with the hacked in version of WMC for win10.

npradeepkumar

Posts: 16
Joined: Sat Nov 26, 2011 10:00 am
Location:

HTPC Specs: Show details

#1811

Post by npradeepkumar » Mon Aug 17, 2020 11:33 pm

Thanks for the warnings. If protected content and premium channels like ESPN etc. do not work, then it is a no go for me. I guess, I have to stay on Windows 7 and make it work.

I have been looking at other options for whole home media solutions. I briefly looked at Plex, Silicondust (hd homerun). There seems to be no good option out there that can do all the things that WMC can do on Windows 7. What else am I missing? Is Direct Tv a better solution or is there any other solution that is better or at least equivalent with all the features that WMC has (Connectivity in multiple rooms, ability to play all channels including premium, DVR functionality etc.)

Apologize if this is not the right place to ask this question.

sptrainman

Posts: 51
Joined: Thu Jul 06, 2017 9:27 pm
Location:

HTPC Specs: Show details

#1812

Post by sptrainman » Mon Aug 24, 2020 4:39 pm

Hi,

My guide has not updated for the last two nights now. The problem seems to be due to missing channels in the data coming from Schedules Direct. Last night, the mxf imported, but the file is 72 hours old. So I'm not sure what that means.

I sent the trace log to you via email.

Thanks,
Jerry

User avatar
garyan2

Posts: 5185
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#1813

Post by garyan2 » Mon Aug 24, 2020 5:34 pm

sptrainman wrote:
Mon Aug 24, 2020 4:39 pm
Hi,

My guide has not updated for the last two nights now. The problem seems to be due to missing channels in the data coming from Schedules Direct. Last night, the mxf imported, but the file is 72 hours old. So I'm not sure what that means.

I sent the trace log to you via email.

Thanks,
Jerry
Jerry, I haven't received your trace.log file. Please send to support@garyan2.net.

EPG123 has a safety check whereas if the available number of stations being downloaded is less than 95% of the number of stations that were available when you last saved the configuration file, it will abort the update. This avoids a massive loss of channels in your guide that may or may not be real.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

User avatar
garyan2

Posts: 5185
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#1814

Post by garyan2 » Mon Aug 24, 2020 6:53 pm

garyan2 wrote:
Mon Aug 24, 2020 5:34 pm
sptrainman wrote:
Mon Aug 24, 2020 4:39 pm
Hi,

My guide has not updated for the last two nights now. The problem seems to be due to missing channels in the data coming from Schedules Direct. Last night, the mxf imported, but the file is 72 hours old. So I'm not sure what that means.

I sent the trace log to you via email.

Thanks,
Jerry
Jerry, I haven't received your trace.log file. Please send to support@garyan2.net.

EPG123 has a safety check whereas if the available number of stations being downloaded is less than 95% of the number of stations that were available when you last saved the configuration file, it will abort the update. This avoids a massive loss of channels in your guide that may or may not be real.
Got it. When you last saved your configuration file, you had 674 stations selected for download. On 8/21 you were downloading 687 stations meaning there were "new" stations added since then. The problem is on 8/22 that number of stations available for download was decreased to 633 which is 93.9% of the original 674. So it is 41 stations less than what were available when you last saved your configuration file, and 54 less than what was available on 8/21.

I'm not going to be able to tell what disappeared and/or added since you last saved your configuration file, but I suggest you open the configuration GUI and inspect to see if any stations you care about are missing. If you are okay with what is listed, just click the [Save & Execute] and everything will complete. Performing a save re-baselines the number of stations you are expecting to download to 633 instead of 674 and it will pass the safety check.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

Post Reply