EPG123 Log Parser
Forum rules
★ Download the latest EPG123 here: https://garyan2.github.io/ <> Setup guide here: https://garyan2.github.io/install.html ★
★ Download the latest EPG123 here: https://garyan2.github.io/ <> Setup guide here: https://garyan2.github.io/install.html ★
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
EPG123 Log Parser
Hello everyone!
I guess I'm getting too lazy, too old, eyesight going bad, or a combination of all of it. I know I can open the trace.log file and visually scan it or perform a search in notepad for errors and whatnot... But, it all comes back to lazy, old, failing eyesight (and probably other stuff as well).
Anyway, I decided to try and make it easy on myself because Gary has done such a wonderful job on his program that I don't find myself looking at the logs much at all. When I do, I find that they've grown pretty sizable and tedious for an old geezer to navigate. Besides, all I want to look at are the errors, warnings, and stats. So I wrote a little utility to help me with that.
It's called "EPG123 Log Parser" and it's a simple exe that can be dropped anywhere. I dropped it in a folder under my EPG123 directory called "Other Stuff".
The first time it's run, it will create an ini file (saving file names and locations) in the same directory that it's run from. It will also create 2 dll files that give it access to:
wwsop34i.dll - Shell operations (for file operation capabilities)
WBDIB44I.DLL - WIL Interpreter (Interprets the source instructions)
These can be safely deleted and will be created each time the parser is run.
The first time It's run it needs to know where the EPG123 log file is located. I had multiple log files, so I just selected one of them.
After that I created a simple interface that shows me the file that I'm working with and then gives me options for what I'd like to see.
Errors - Creates a concise listing of errors.
Warnings - Creates a concise listing of warnings.
Stats - Creates a concise listing of stats.
Full Log - Opens the currently selected log file.
Select Different Log File - I had multiple log files, so I needed a way to select a different one!
Exit - ...
When a button is pressed (for example Errors) the program will open the selected log file in READ ONLY mode. As it analyzes each line of the log, it will copy only the relevant lines to a new file. When it has analyzed each line in the log, it will open a new Notepad document.
It creates these "trash" files in the same directory as the log file that was selected and names them "OK_TO_DELETE_XXXXXXOutput.txt". Where "XXXXXX" is Error, Stats, or Warning as applicable.
And as it says, they're OK to delete! All in all it creates 6 files (3 text files, 2 dll's, and an ini file) that can be easily deleted. It doesn't write anything to the registry and deleting the exe is all that's needed to "uninstall" it.
I've successfully run it in Windows 10(x64) and Windows 7 Pro(x64).
I guess I'm getting too lazy, too old, eyesight going bad, or a combination of all of it. I know I can open the trace.log file and visually scan it or perform a search in notepad for errors and whatnot... But, it all comes back to lazy, old, failing eyesight (and probably other stuff as well).
Anyway, I decided to try and make it easy on myself because Gary has done such a wonderful job on his program that I don't find myself looking at the logs much at all. When I do, I find that they've grown pretty sizable and tedious for an old geezer to navigate. Besides, all I want to look at are the errors, warnings, and stats. So I wrote a little utility to help me with that.
It's called "EPG123 Log Parser" and it's a simple exe that can be dropped anywhere. I dropped it in a folder under my EPG123 directory called "Other Stuff".
The first time it's run, it will create an ini file (saving file names and locations) in the same directory that it's run from. It will also create 2 dll files that give it access to:
wwsop34i.dll - Shell operations (for file operation capabilities)
WBDIB44I.DLL - WIL Interpreter (Interprets the source instructions)
These can be safely deleted and will be created each time the parser is run.
The first time It's run it needs to know where the EPG123 log file is located. I had multiple log files, so I just selected one of them.
After that I created a simple interface that shows me the file that I'm working with and then gives me options for what I'd like to see.
Errors - Creates a concise listing of errors.
Warnings - Creates a concise listing of warnings.
Stats - Creates a concise listing of stats.
Full Log - Opens the currently selected log file.
Select Different Log File - I had multiple log files, so I needed a way to select a different one!
Exit - ...
When a button is pressed (for example Errors) the program will open the selected log file in READ ONLY mode. As it analyzes each line of the log, it will copy only the relevant lines to a new file. When it has analyzed each line in the log, it will open a new Notepad document.
It creates these "trash" files in the same directory as the log file that was selected and names them "OK_TO_DELETE_XXXXXXOutput.txt". Where "XXXXXX" is Error, Stats, or Warning as applicable.
And as it says, they're OK to delete! All in all it creates 6 files (3 text files, 2 dll's, and an ini file) that can be easily deleted. It doesn't write anything to the registry and deleting the exe is all that's needed to "uninstall" it.
I've successfully run it in Windows 10(x64) and Windows 7 Pro(x64).
-
- Posts: 721
- Joined: Mon Nov 05, 2012 8:05 pm
- Location: Longmont, CO
- HTPC Specs:
If you want to save yourself from ever having to look in your logs you can also follow the instructions I posted here:
viewtopic.php?f=99&t=10984
and get an e-mail if EPG123 has thrown an error or warning.
viewtopic.php?f=99&t=10984
and get an e-mail if EPG123 has thrown an error or warning.
- garyan2
- Posts: 7438
- Joined: Fri Nov 27, 2015 7:23 pm
- Location:
- HTPC Specs:
This is pretty cool, though I may see something different than everyone else.
What I see is a system that has been online at least 10 months and, regardless of cause, has only had 9 days that may not have received an update.
@StinkyImp
Did you intend to attach the utility, or are you still polishing it up? I wouldn't mind scanning my server to see what it says, though I did a complete rebuild about 6 months ago.
EDIT: I'll actually revise that number to 5 days in the last 10 months and 2 of those is SD down for maintenance and a recording in progress.
EDT2: Well, shoot ... it was only for 6 months, not 10.
What I see is a system that has been online at least 10 months and, regardless of cause, has only had 9 days that may not have received an update.
@StinkyImp
Did you intend to attach the utility, or are you still polishing it up? I wouldn't mind scanning my server to see what it says, though I did a complete rebuild about 6 months ago.
EDIT: I'll actually revise that number to 5 days in the last 10 months and 2 of those is SD down for maintenance and a recording in progress.
EDT2: Well, shoot ... it was only for 6 months, not 10.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io
Keeping WMC alive beyond January 2020. https://garyan2.github.io
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
Thanks garyan2! I built this system about 6 months ago, so that's all the logs I have. I should probably pull the logs off my old HTPC and see what they show!garyan2 wrote:This is pretty cool, though I may see something different than everyone else.
What I see is a system that has been online at least 10 months and, regardless of cause, has only had 9 days that may not have received an update.
@StinkyImp
Did you intend to attach the utility, or are you still polishing it up? I wouldn't mind scanning my server to see what it says, though I did a complete rebuild about 6 months ago.
EDIT: I'll actually revise that number to 5 days in the last 10 months and 2 of those is SD down for maintenance and a recording in progress.
EDT2: Well, shoot ... it was only for 6 months, not 10.
Your program is so rock solid that the errors and whatnot are pretty few and far between. I just wanted a way to look at only those. I also didn't know whether anyone (especially you) would be offended by it. Therefore this post was just a "feeler". I figured if anyone was opposed to it, I'd keep it to myself, otherwise I'd share it with anyone that thought it might be useful to them.
Polished? I'm like anyone else. I see things in it I'd like to improve, but will I? It depends... I'm like you though, I wanted to see every error listed by itself in sequential order so I could have an overview. If you'd like I can PM you a copy with the source so you can verify that it won't affect your program or logging in any way.
- garyan2
- Posts: 7438
- Joined: Fri Nov 27, 2015 7:23 pm
- Location:
- HTPC Specs:
Certainly not offended... the log file is there first for a troubleshooting aid, but I also use it to measure how well things are performing. Kind of a health check on my setup. By all means, have fun with it and go as far as you want.
If you can, try to ensure that your I/O access to the file does not lock the file (read only, shared?). If your program has the file locked while epg123 tries to write to it, well nothing bad will happen; it will just fail to write to the file. Of course, the user could simply make sure they didn't run your program during an update.
If you can, try to ensure that your I/O access to the file does not lock the file (read only, shared?). If your program has the file locked while epg123 tries to write to it, well nothing bad will happen; it will just fail to write to the file. Of course, the user could simply make sure they didn't run your program during an update.
- Gary
Keeping WMC alive beyond January 2020. https://garyan2.github.io
Keeping WMC alive beyond January 2020. https://garyan2.github.io
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
Hmmm... For the past two days I've tried to PM you a copy of this. Yesterday I was getting "The connection to the server was reset while the page was loading." error. So this morning I tried changing extensions, zipping, etc. The source is a Notepad txt file and it returned "The extension txt is not allowed." in addition to the "server reset" error.
What kind of attachments are allowed to PM's? Maybe it's a size limit, but the text document is only ~12KB.
What kind of attachments are allowed to PM's? Maybe it's a size limit, but the text document is only ~12KB.
-
- Posts: 1708
- Joined: Fri Aug 24, 2012 7:35 pm
- Location:
- HTPC Specs:
Will it parse channel number changes, removals and additions?
Thanks
Sent from my SM-G930P using Tapatalk
Thanks
Sent from my SM-G930P using Tapatalk
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
I'm going to rewrite this because I found that I wanted other information that I didn't include in the original version, like "execution times". When I was adding in that functionality I realized that the utility was becoming larger and losing efficiency.Sammy2 wrote:Will it parse channel number changes, removals and additions?
Thanks
I'd like to look at other things too (like channel number changes, etc)... So I'm going to scrap the version I was building and start from square one. Right now the things it looks for are based on [BRACKET] tags that garyan2 uses to make certain things pop. I envision a tool that will have a drop down of common queries, but will also allow a custom search string to be used.
So in response to Will it parse channel number changes, removals and additions?... Not at the moment. The next version "should" be able to pull out only the information you want to see.
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
ADMIN PLEASE HELP! Every time I try to upload this utility to a post, or to a PM, I get the "Connection reset by server" error. Can you assist me with this problem? I've tried zipping, changing the extension, etc...StinkyImp wrote:I'm going to rewrite this because I found that I wanted other information that I didn't include in the original version, like "execution times". When I was adding in that functionality I realized that the utility was becoming larger and losing efficiency.Sammy2 wrote:Will it parse channel number changes, removals and additions?
Thanks
ALPHA VERSION Things to know:
- 1. I was happily generating reports with this when (after awhile) I realized I had 25+ Notepad documents open. Therefore... This utility is very Notepad'centric. It checks for the existence of any open Notepad instances and won't continue processing until they are closed.
- 2. On initial startup it will check to see if you have any Notepad instances open and will give you only ONE chance to close them before it shuts itself down.
- 3. Upon your first use, you will be presented with a file browser to select the location of the log file you'd like to analyze.
- 4. This utility never affects your "live" log file. It only works with a temporary copy.
- 5. (Suggestion) Create a directory to stick this in. Then to uninstall, all you need to do is delete that directory.
- 6. This was compiled and tested on Win10(x64) and Win7(x64). I'm not sure if it will run on 32 bit systems (or Win8) because I don't have one.
- 7. TO UNINSTALL - If you used the suggestion above, simply delete the directory you created. This utility is completely portable and makes absolutely no changes to your system.
Section (Working Log File)
- Select Different Log File - If you have multiple log files, you can select a new one to work with.
- Open Working Log - Opens a temporary copy of the current log.
- Predefined Search Terms (Field) - This is a list of common terms found in EPG123.
- Add Terms to List - Allows you to add terms to the predefined list. This is a work in progress. For now it only saves the changes for the current session.
- Generate Predefined - Performs a search query and returns a summary report.
- Enter Custom String (Field) - You can type (or copy/paste) a custom search word or phrase to search for.
- Generate Custom - Performs a search query and returns a summary report.
Delete Temp Files - Allows you to delete all the temporary files that you generate except the temporary copy of the log file. It uses that to work with.
Exit - Will perform cleanup operations (removes all temporary files, including the temporary log file) and cleanly exits the utility.
Section (Credits) - Tries to avert any hassles for garyan2 but also points potential users to his EXCELLENT program.
- Scallica
- Posts: 2797
- Joined: Mon Jun 06, 2011 7:09 pm
- Location: USA!
- HTPC Specs:
No need to shout How large is the file?StinkyImp wrote:ADMIN PLEASE HELP! Every time I try to upload this utility to a post, or to a PM, I get the "Connection reset by server" error. Can you assist me with this problem? I've tried zipping, changing the extension, etc...
HTPC Enthusiast / Forum Moderator - TGB.tv Code of Conduct
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
ROFL! That wasn't so much shouting as it was to be easily noticed.Scallica wrote:No need to shout How large is the file?StinkyImp wrote:ADMIN PLEASE HELP! Every time I try to upload this utility to a post, or to a PM, I get the "Connection reset by server" error. Can you assist me with this problem? I've tried zipping, changing the extension, etc...
Zipped it's 574 KB. But... I tried to attach a 12 KB text file the other day and it failed too.
- Scallica
- Posts: 2797
- Joined: Mon Jun 06, 2011 7:09 pm
- Location: USA!
- HTPC Specs:
Test file.
- Attachments
-
- test-file.zip
- Zipped Text File 39KB
- (323 Bytes) Downloaded 56 times
HTPC Enthusiast / Forum Moderator - TGB.tv Code of Conduct
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
The previous post was in Firefox. This post is using Chrome. I can add your file and png files without any issues. Every time I try to add "EPG123 Log Parser.zip" I get the server reset error. Are there restrictions in place at my user level?
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
The button [Add Terms to List] has been modified to allow changes to be persistent/permanent.
If the file named "EPG123 Predefined Terms.txt" in the program directory is deleted, then this utility will rebuild it using the default search terms and any changes will be lost.
If the file named "EPG123 Predefined Terms.txt" in the program directory is deleted, then this utility will rebuild it using the default search terms and any changes will be lost.
- Scallica
- Posts: 2797
- Joined: Mon Jun 06, 2011 7:09 pm
- Location: USA!
- HTPC Specs:
Ahh, we don't allow EXE files. The ZIP contains an EXE file; that's why it's failing to upload. Rename the EXE to EPG_Log_Parser.zip. Then when a user downloads it, they can rename it back to .exe.
HTPC Enthusiast / Forum Moderator - TGB.tv Code of Conduct
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
I'll give that a shot. The extension on this file needs to be changed from .zip to .exe.Scallica wrote:Ahh, we don't allow EXE files. The ZIP contains an EXE file; that's why it's failing to upload. Rename the EXE to EPG_Log_Parser.zip. Then when a user downloads it, they can rename it back to .exe.
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
I've been working on this thing until my eyes are crossed. This is an entirely different utility based on garyan2's input and will look for trace logs created by EPG123. It will merge together up to three log files and use that to work with. If EPG123 has just one or two files, that's what it will use. The three logs that were created on my system spanned from February 5, 2017 through today and has close to 22,000 lines of data!
This version has a ton of options but still only works off "copies" of the EPG123 log file(s). I've removed the Notepad checking because it created too much overhead and well... the user can close any instances that are created. This utility will give an occasional reminder to close them, but that's up to the user.
Here's the current interface. and here's a sample report. It took 13 seconds to extract the 14 lines that included the search term "client execution time" between April 1st and April 15th by going through almost 22,000 lines of data.
As hard as I tried (putting everything about 5 to 6 directories deep on a USB drive) I couldn't perform a query that took even a minute to perform. The worst timed query was 51 seconds.
NOTE: "client execution time" does not equal "Client Execution Time". The queries are case sensitive.
TIP: Use the "Open Working Log" button to open the full log and copy/paste your search criteria. If it's something you'll use consistently, add it to the predefined list.
To reiterate, it's a simple exe that can be dropped anywhere. I dropped it in a folder under my EPG123 directory called "Log Parser". So to completely uninstall it I can just delete that directory. Nothing is ever written to the registry (or anywhere else) and it keeps itself contained. Change the "zip" extension to "exe" and that's it!
This version has a ton of options but still only works off "copies" of the EPG123 log file(s). I've removed the Notepad checking because it created too much overhead and well... the user can close any instances that are created. This utility will give an occasional reminder to close them, but that's up to the user.
Here's the current interface. and here's a sample report. It took 13 seconds to extract the 14 lines that included the search term "client execution time" between April 1st and April 15th by going through almost 22,000 lines of data.
As hard as I tried (putting everything about 5 to 6 directories deep on a USB drive) I couldn't perform a query that took even a minute to perform. The worst timed query was 51 seconds.
NOTE: "client execution time" does not equal "Client Execution Time". The queries are case sensitive.
TIP: Use the "Open Working Log" button to open the full log and copy/paste your search criteria. If it's something you'll use consistently, add it to the predefined list.
To reiterate, it's a simple exe that can be dropped anywhere. I dropped it in a folder under my EPG123 directory called "Log Parser". So to completely uninstall it I can just delete that directory. Nothing is ever written to the registry (or anywhere else) and it keeps itself contained. Change the "zip" extension to "exe" and that's it!
-
- Posts: 89
- Joined: Mon Feb 22, 2016 8:27 pm
- Location: Bethesda, Maryland
- HTPC Specs:
Cool utility. I'll give it a look but since I just rebuilt my HTPC with a more modern MoBo and CPU 10 days ago I don't have much of a log to look at. Probably going forward it will be useful. thanks for the work on this, when one has a huge log, finding stuff can be a real drag.
- StinkyImp
- Posts: 669
- Joined: Thu May 11, 2017 7:53 pm
- Location:
- HTPC Specs:
Hi Alan.Alan G wrote:Cool utility. I'll give it a look but since I just rebuilt my HTPC with a more modern MoBo and CPU 10 days ago I don't have much of a log to look at. Probably going forward it will be useful. thanks for the work on this, when one has a huge log, finding stuff can be a real drag.
Thanks for your kind words! This is a work in progress. One of the reasons I built this utility is for the exact reason you stated. Garyan2 has designed EPG123 to have up to three log files. When the trace.log file reaches a certain size, he timestamps it, backs it up, and then creates a new trace.log. He does this twice and then deletes the oldest backup. Therefore, you could conceivably have one trace.log and two timestamped backups.
This utility attempts to merge these log files and then give the user various tools to analyze it. As of this morning my "merged log" starts in early February and includes 21,999 (almost 20,000) lines of data.
Attached is a maintenance update that adds some functionality or corrects some issues. I think I'm the only one really using it right now and I find it hard to test every scenario there might be so... any feedback would be useful.
Changes in this version - 1.5.1 (ALPHA)
- 1. Creates a directory under Epg123 called "Log Backups". I did this so that in the future I could manually merge the oldest log backups (or create a routine that recognizes and automatically merges the oldest logs) and maintain a huge history that could conceivably span years (if not decades).
- 2. Fixed a bug concerning quotation marks that, on rare occasions, could hang the utility.
- 3. Report enhancements.
- 4. Better error handling and code refinement.
- 5. Minor stuff that was fixed on the fly.