PSA for failed token requests

[garyan2]

Everyone, this is just a Public Service Announcement. One of the main reasons folks are having problems getting a token and/or downloading updated guide listings, is that everyone is trying to do their updates at the same time. It is okay to do updates at other times of the day other than when you are sleeping. There is no issue with an update being done while watching TV. If you are recording, then EPG123 will wait until the recordings are complete before updating WMC (which can be overridden but not advisable). If everyone can randomize their schedules a bit, maybe we can level out the load on the SD servers so we can all continue to enjoy up-to-date guide listings.

Not only does it benefit us all as a whole to not see the red EPG logo, but it also helps SD avoid additional costs putting more servers online to support the midnight EST to 3AM PST (6 hours) demand. This in turn allows SD to keep the membership fees low (my guess).

Can I programmatically handle a failed token request to delay a random amount of time before trying again? Sure. But if that thought is okay for you, than it is also okay for you to go ahead and change your scheduled update time to a time that works for you... possibly outside the high demand times.

[StinkyImp]

Can I programmatically handle a failed token request to delay a random amount of time before trying again? Sure.

But if that thought is okay for you, than it is also okay for you to go ahead and change your scheduled update time to a time that works for you... possibly outside the high demand times.

I vote for programatically, please?

I've tried to time shift and at some point it fails. 12:15 worked for three years! For me it's a pain because there's a cascade effect in changing the time.

Code: Select all

[8/18/2017 2:00:23 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[3/2/2018 12:15:20 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[3/20/2021 12:18:10 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[5/1/2021 12:45:09 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[6/3/2021 12:07:13 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/8/2021 2:21:00 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.

[Space]

When you get a failed token request, does the server tell you when it's OK to try again?

Sometimes servers are able to determine how busy they are, and give out a "try again" time so that the client knows when it's OK to try again, and everyone isn't retrying at once. Although from the error message, it looks like it is just timing out, so no response at all I assume.

Just as a data point, I do automatic updates every day during the day (not at night) and have not had a single failure.

[garyan2]

The token requests are timing out at 3 seconds, so no response from SD to work off there. I'll have to look, but I believe they do supply a retry again time but only from maintenance. I might be misremembering or thinking of a different API.

[stuartm]

Are there many new users? My update times have been set for a very long time and only started failing recently.

[garyan2]

Not that I have seen. SD will be able to say better whether it is new users or maybe related to some background processes they are running to improve program/season/series images. We've seen in the past that it could be an errant server that needs to be kicked, or just add more servers as a fix action.

[IT Troll]

Hmm, 06:15 BST (22:15 PST / 01:15 EST) here. I wouldn’t have thought that was a common choice and I’ve not had problems before. I’ll try moving it though.

Some folks have posted on the SD forum about performance issues. No official response yet.

[d00zah]

My scheduled update ( EST) has been quite reliable until recently.

Code: Select all

[5/7/2021 4:11:01 AM] [ERROR] Failed to complete request. Exiting
[5/7/2021 4:11:01 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[5/7/2021 4:11:01 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[5/7/2021 4:11:01 AM] [ERROR] Failed to create MXF file. Exiting.
[5/7/2021 4:12:42 AM] [ERROR] The MXF file imported is 24.03 hours old.
===
[5/7/2021 9:52:21 AM] [ERROR] Failed to complete request. Exiting
[5/7/2021 9:52:21 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
===
[7/4/2021 4:11:04 AM] [ERROR] Failed to complete request. Exiting
[7/4/2021 4:11:04 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/4/2021 4:11:04 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[7/4/2021 4:11:04 AM] [ERROR] Failed to create MXF file. Exiting.
[7/4/2021 4:12:26 AM] [ERROR] The MXF file imported is 24.02 hours old.
===
[7/5/2021 4:11:03 AM] [ERROR] Failed to complete request. Exiting
[7/5/2021 4:11:03 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/5/2021 4:11:03 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[7/5/2021 4:11:03 AM] [ERROR] Failed to create MXF file. Exiting.
[7/5/2021 4:12:26 AM] [ERROR] The MXF file imported is 48.02 hours old.
===
[7/6/2021 4:11:00 AM] [ERROR] Failed to complete request. Exiting
[7/6/2021 4:11:00 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/6/2021 4:11:00 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[7/6/2021 4:11:01 AM] [ERROR] Failed to create MXF file. Exiting.
[7/6/2021 4:12:23 AM] [ERROR] The MXF file imported is 72.02 hours old.
===
[7/7/2021 4:11:01 AM] [ERROR] Failed to complete request. Exiting
[7/7/2021 4:11:01 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/7/2021 4:11:01 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[7/7/2021 4:11:01 AM] [ERROR] Failed to create MXF file. Exiting.
[7/7/2021 4:28:37 AM] [ERROR] The MXF file imported is 96.29 hours old.
===
[7/8/2021 4:11:03 AM] [ERROR] Failed to complete request. Exiting
[7/8/2021 4:11:03 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/8/2021 4:11:03 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[7/8/2021 4:11:03 AM] [ERROR] Failed to create MXF file. Exiting.
===
[7/8/2021 5:03:14 AM] [ERROR] Failed to complete request. Exiting
[7/8/2021 5:03:14 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/8/2021 5:04:52 AM] [ERROR] Failed to complete request. Exiting
[7/8/2021 5:04:52 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
===
[7/9/2021 4:11:03 AM] [ERROR] Failed to complete request. Exiting
[7/9/2021 4:11:03 AM] [ERROR] Did not receive a response from Schedules Direct for a token request.
[7/9/2021 4:11:03 AM] [ERROR] Failed to retrieve token from Schedules Direct. message: Did not receive a response from Schedules Direct for a token request.
[7/9/2021 4:11:03 AM] [ERROR] Failed to create MXF file. Exiting.

[StinkyImp]

My scheduled update ( EST) has been quite reliable until recently.

I'm on day six.

When you get a failed token request, does the server tell you when it's OK to try again?

I remember seeing "Trying again" being logged, but it fails immediately after that entry. It doesn't retry.

Code: Select all

EPG123 Log Parser (and Report Generator) - v 1.7.6 (64 bit)
Report Created on 07/09/2021 at 09:40:56
================================================================
[7/4/2021 12:07:04 AM] [ INFO] SD API WebException Thrown. Message: The operation has timed out , Status: Timeout . Trying again.
[7/5/2021 12:07:04 AM] [ INFO] SD API WebException Thrown. Message: The operation has timed out , Status: Timeout . Trying again.
[7/8/2021 12:07:04 AM] [ INFO] SD API WebException Thrown. Message: The operation has timed out , Status: Timeout . Trying again.
[7/8/2021 2:21:00 AM] [ INFO] SD API WebException Thrown. Message: The operation has timed out , Status: Timeout . Trying again.
[7/8/2021 2:28:43 AM] [ INFO] SD API WebException Thrown. Message: The operation has timed out , Status: Timeout . Trying again.
[7/8/2021 2:29:04 AM] [ INFO] SD API WebException Thrown. Message: The operation has timed out , Status: Timeout . Trying again.

===== QUERY INFORMATION (Custom) ===============================
Logs used:		trace.log ONLY
Search String:		Trying again
Date Range:		07/01/2021 thru 07/09/2021 (9 days)
Total results found:	6
Query time:		02 seconds

[garyan2]

I don't do retries for token requests. If the reason is wrong password, then it counts against you toward the account lockout limit.

I was thinking about your situation this morning, and it might seem strange, but you can create a task for the server component to download guide listing at a reliable time (10PM local?) and the epg123_update task could be set up as client mode. Kinda split when you download guide listing and when you actually import them.

Sent from my SM-G965U using Tapatalk

[stuartm]

So for most (all?) of us experiencing problems, we have not changed our times and failures started happening. Most likely is something changed at SD and they either don't have enough token capacity available for some reason or hinted at performance problems are keepng tokens active longer than previously and thus locking out update attempts. (I suppose it's possible something Gary did in a recent EPG123 release holds tokens longer but since people were updating to newer releases trying to fix the token problem it seems unlikely). I had failures again last night so even though they were at 2:32 and 3:11 AM Mountain time, I guess I can try moving them to different times. (a 4:13 AM update did succeed)

[d00zah]

Holding tokens 24 hrs? Seems improbable IMO. Started on the 4th & an EPG123 release came out several days later. More likely just ongoing SD maintenance. Inconvenient, but not the end of the world. YMMV

[garyan2]

EPG123 doesn't hold tokens. There can be only 1 active token for an account and it lasts for 24 hours. Every time you open the configuration gui, you request and receive a new token and the countdown starts again. Same for everything an update is run. The old token is retired/invalid.

It isn't like a license server whereas you have a limited number of active tokens.

Sent from my SM-G965U using Tapatalk

[stuartm]

Thanks for the explanation. So it is really a matter of SD not responding to a token request, not a token limit.

[garyan2]

Correct.

Sent from my SM-G965U using Tapatalk

[StinkyImp]

you can create a task for the server component to download guide listing at a reliable time (10PM local?) and the epg123_update task could be set up as client mode. Kinda split when you download guide listing and when you actually import them.

I have always been excited and up to the challenge to try any tips and tricks you post including installing "test" versions of your software. In this case I'm hesitant.

As you know I'm a HUGE user of, and developer for, your log files. In the beginning you warned encouraged me to leave the logs unmolested in any way, shape, or form. I have held true to that guidance to the extreme. In doing so I have created an automated sequential system that analyzes my personal log environment (before and after the trace.log is updated) through both scheduled tasks and specialized custom programs. I do this to help me refine my log utility and ensure the health of the logs for me and anyone that uses it.

Every part of that is predicated on the update being performed completely between midnight and 1:00AM. If I change the time, I have to change everything before and everything after. I've done that previously and for me it's a huge pita if it's only going to work for a short time.

Performing the "Save & Execute" solution outside those times throws a monkey wrench in my works too...

[FEATURE REQUEST] -> When EPG123 encounters the string "Failed to retrieve token" have it do something, like pause for a bit and retry? It doesn't appear to make it to the "Status request successful. account expires:" phase. Therefore there's no login to count against attempts.

I don't know... All I know is I don't want to have to do anything wonky on my system that may or may not work.

[IT Troll]

I moved my task out of the peak US times (still at a time that works for me in the UK) and it went through fine last night.

It does seem odd that we have suddenly hit a wall rather than see a gradual increase in errors. I wonder if there is some other greedy client app causing the problem. The lack of comms from SD, either here or on their own forum, is disappointing and out of character.

I see they blocked Emby last month as it is now commercial. So if anything they should have spare capacity. Unless they scaled back following this action...

[Space]

I normally update in the evening hours and have had no problems. I tried to update around 4:30am ET this morning as a test and it failed, so it does look like the servers are failing (or very busy) during the early morning hours.

[StinkyImp]

Failed again last night.

If I knew, or had confidence, that changing the time to a different one-hour window would work in the long term, I'd do it in a heartbeat. As it is, it's a crap shoot.

[d00zah]

Failed again last night.

Same.