Adventures with \Playready\MSPR.HDS and DRM

An evolving, supported alternative to Rovi
Forum rules
★ Download the latest EPG123 here: http://epg123.garyan2.net <> Setup guide here: http://epg123.garyan2.net/downloads/epg123_Guide.pdf
Post Reply
DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

Adventures with \Playready\MSPR.HDS and DRM

#1

Post by DSperber » Thu Jun 06, 2019 1:12 am

It works!!! Saving/restoring MSPR.HDS (while in SAFE mode) absolutely seems to retain usability of all of the DRM copy-protected WTV recordings made by my Ceton tuner. JOY!!

For the past week or so, my HTPC has been acting up strangely. I don't know what caused it but its behavior has been deteriorating. Extender RDP sessions seem to "hang" even though I've powered the extender off, so that when I try to re-boot (hopefully to reset things back to their normal behavior) I get a message about currently still-active connections, etc. That's not the only mysteriously should-not-occur symptom, and it's all very unexpected and not right.

Not knowing what might have happened over the past week that triggered this, I decided to restore a "GOLD" system image backup I had specifically taken back on May 28, because I had just finished some software installs and updates and decided since things overall were working so blissfully well with this HTPC that it deserved to be "memorialized" in a special backup, apart and separate from my normal regular ongoing nightly "system image" backups which go back 7 generations in a continuing loop.

So first I booted to SAFE mode, in order to preserve both (a) c:\ProgramData\Microsoft\Playready\MSPR.HDS, as well as (b) my NovaStor NovaBACKUP folder also located in ProgramData which contains data on my regular nightly "data" folder/file backups. If I was going back to a Windows image from 5/28 I didn't want to lose both (a) access to DRM copy-protected recordings made right up to 4PM today, and (b) logs from NovaBACKUP run over the past 8 days. I did a simple COPY from the two source ProgramData folder locations to a "saved data" folder location on my G-drive that I had created many years ago for such use.

Then I booted to Macrium Reflect, and restored the "Gold" image of my Win7 C-partition from 5/28 (only takes 2 minutes).

Then I re-booted back to Boot Manager, and pressed F8 to get into Safe mode. I then deleted the current version of \PlayReady\MSPR.HDS as well as the NovaBACKUP log data folder, and replaced them with the latest up-to-date versions I had just a few minutes earlier preserved on my G-drive from my pre-restore latest Windows.

Then I re-booted normally, got into WMC and verified that I could in fact still play those DRM copy-protected recordings made from just today, before 4PM. AMAZING!!! It works! Thumbnails correctly appear in the recordings list, and the programs themselves play normally..

So, thanks again for this insight regarding MSPR.HDS holding all of the DRM keys. Properly preserving/restoring in SAFE mode guarantees that even an older "system image" backup must be restored, at least 100% of the DRM copy-protected recordings will still be playable after the restore, even though the recordings were made more recently than the date of the "system image" backup that got restored.

Now let's hope that whatever was the cause of the "instability" in my HTPC that seemed to have crept in over the past week or so will no longer be present now that I've restored my GOLD backup from 5/28. I know things were "perfect" back then so much so that I was inspired to take this "GOLD" image. So let's hope that's true now as well, having restored that image.

Space

Posts: 1558
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#2

Post by Space » Thu Jun 06, 2019 1:55 am

Good to see further confirmation of what others have reported. I have not had to do this type of restore myself, but it is reassuring that if I needed to, I know how to do it and that it will work.

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#3

Post by DSperber » Thu Jun 06, 2019 4:47 am

I did check my regular nightly "data" folder/file "incremental" backups (which archive anything created or changed in the past 24-hours), and sure enough c:\ProgramData\Microsoft\Playready\MSPR.HDS is always present on each night's archive. What I don't know is whether or not the backup taken at 1AM every night will successfully copy it (even if it is "open" and/or conceptually "in use"by WMC),. I'm trying to decide if I MUST actually have to be in SAFE mode to know that I'm getting a 100% usable preserved version of that file.

There's certainly no warning or other message from NovaBACKUP regarding "file in use" or giving me any reason to suspect that it might not be the current active version, but an older "original" starting version. I suppose whatever's on disk is what gets backed up, not what might be in-memory and newer. So if the file is written out by WMC whenever a new program gets recorded (which seems very likely and plausible and reasonable), and as long as the file is not "in-use" and thus prevented from being read by NovaBACKUP, I would have a hunch that the regular nightly backup would be usable for restoring from (as long as there weren't yet more additional copy-protected recordings made after that backup was taken, which would thus definitely be "lost" following the back-dated image restore).

Certainly doing the preserve/restore in SAFE mode right before and after the image restore is the absolute most reliable way to ensure no copy-protected recordings would get lost. But I suppose in absolute worst case last night's 1AM backup version could be considered an emergency fallback, with a possible "price" if later newer DRM recordings also exist.

Space

Posts: 1558
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#4

Post by Space » Thu Jun 06, 2019 5:25 am

Yeah, I remember some discussion about the file being in use when copying it, but I don't remember if there was any verdict on if it was safe or not. I seem to recall that if you can copy it without error, it should be good, but don't quote me on that.

Also, when restoring it, I think you can stop one of the services, restore the file and then start the service again instead of doing it in safe mode, but again, I don't remember the details, and since this should be a rare occurrence, doing it in safe mode should not be a big deal.

Sammy2

Posts: 1669
Joined: Fri Aug 24, 2012 7:35 pm
Location:

HTPC Specs: Show details

#5

Post by Sammy2 » Thu Jun 06, 2019 2:56 pm

I do not miss messing with DRM one bit.

jachin99

Posts: 1160
Joined: Wed Feb 24, 2016 3:36 pm
Location:

HTPC Specs: Show details

#6

Post by jachin99 » Thu Jun 06, 2019 3:51 pm

Now I wonder if there is some way you could restore that image & key to a new PC build. I know new hardware would probably require updated drivers but I wonder if you could restore the old image to the new hardware and have drivers work well enough to run everything smoothly.

cwinfield

Posts: 558
Joined: Tue Feb 12, 2013 1:14 am
Location: Monroe, NC

HTPC Specs: Show details

#7

Post by cwinfield » Thu Jun 06, 2019 4:25 pm

jachin99 wrote:
Thu Jun 06, 2019 3:51 pm
Now I wonder if there is some way you could restore that image & key to a new PC build. I know new hardware would probably require updated drivers but I wonder if you could restore the old image to the new hardware and have drivers work well enough to run everything smoothly.
Nope, the mspr.hds file is tied to your hard drive partition data and windows install, this generates the encryption key that identifies that unique computer for digital rights management.

If you had identical hardware you could use a restore image and it may work...

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#8

Post by DSperber » Sat Jun 08, 2019 7:13 am

Ok. The story of which originally prompted me to start this thread regarded a major malfunction:

"For the past week or so, my HTPC has been acting up strangely. I don't know what caused it but its behavior has been deteriorating. Extender RDP sessions seem to "hang" even though I've powered the extender off, so that when I try to re-boot (hopefully to reset things back to their normal behavior) I get a message about currently still-active connections, etc. That's not the only mysteriously should-not-occur symptom, and it's all very unexpected and not right."

So for a week or so I've been unable to start/stop and then re-start extender sessions. And that also means I've been unable to be watching a program with the extender set to 1080i, and then get into settings (which stops the session) to reset to 720p, and then re-start the extender to now watch a new program at 720p. Any "stop" of the first extender RDP session leaves the session "dangling and hung", and not stopped at all.

From a Windows internal perspective this corresponds to a 115 Event Log entry when the session starts, but no corresponding 117 entry when the session ends.

And this is why I thought something in my Windows had gotten corrupted, with the easiest way to recover being to restore the May 28 "Gold" image backup I'd taken, and then reinsert the latest MSPR.HDS prior to the restore.

Well, after a frantic week of trying lots of things which should have had beneficial effect but surprisingly had no success, I finally decided that I might need to rebuild this failing HTPC again (but salvaging current copy-protected recordings by use of the MSPR.HDS trick), and to give me the opportunity to do this I decided to re-activate my second "currently spare" and turn it into my "production" machine. So I went to the machine, and before doing anything just wanted to confirm that it still "worked perfectly" and didn't exhibit this mysterious "dangling extender RDP session" symptom I've been fighting. Imagine my surprise to discover that even this second HTPC, which hadn't been used for months, was also exhibiting the exact same "dangling extender RDP session" symptom!!

How can this be? Only some common hardware or common software with the other failing HTPC exhibiting the same anomaly. I do have a spare router, and decided that as inexplicable as it sounded I suppose I could try replacing my current main router with this spare... just to be sure it wasn't somehow part of the story. But before spending any time on that activity I decided to back out the current nVidia 430.86 driver I'd recently updated to. Not surprisingly, that had no effect.

Also, something was clearly seriously wrong, as QWINSTA and RWINSTA commands were not accomplishing the manually forced "kill" termination of an open active RDP session.

Image

I then decided to act on my real main suspicion, which was some harmful effect of this past week's "product feature enhancement" of BitDefender 2019 Total Security. You may remember that back around four months ago I had discovered a software conflict between Malwarebytes and Macrium Reflect, which had been responsible for the "freeze" symptom I'd been fighting on both of my HTPC machines for eight months. Until the software conflict was resolved (by a fix from Macrium Reflect) I uninstalled Microsoft Security Essentials and Malwarebytes and instead went to BitDefender as my anti-virus product. Of course this "fixed" the freeze problem (since Malwarebytes was no longer on my system, and thus could no longer bump heads with Macrium Reflect) so I've been running with BitDefender ever since.

Because of the curious recent timing of my symptom's birth, along with the recent release of a significant feature update to BitDefender, I decided on a flyer to temporarily disable (but not fully uninstall) BitDefender, just to see if that cured the RDP session problem. It did not.

I then decided that just to be absolutely positive, I would fully uninstall BitDefender and reinstall Microsoft Security Essentials. Amazingly... THIS FIXED IT!!! Sure enough, extender sessions were now correctly ending with power-off as they should and as they always have until this past week. I then went to my second HTPC (where the same symptom had surprisingly also been occurring) and repeated my "cure": uninstall BitDefender and reinstall MSE. And sure enough, once again THIS FIXED IT!!!

So that's how I'm still running at the moment. I've obviously reported the issue to BitDefender, and haven't yet reinstalled Malwarebytes (only because I canceled my Malwarebytes licenses when I purchased BitDefender a few months back) but will make a decision on that soon. I plan to make one of my machines available to BitDefender as a "lab rat" so that we can reproduce the failure and through their diagnostic trace code chase down what is going wrong and then test out the proposed "fix".

Note that there not only was this extender RDP session problem I was observing this past week. There were several other graphics and audio and memory and CPU anomalies I had noticed, and these too have all disappeared with the uninstall of BitDefender. These may or may not be tied to the same new product features responsible for the extender RDP session issue, but for sure were clearly tied to the presence of the current version of BitDefender and are gone now that BitDefender is uninstalled.

I'm now going to make a new "Gold" version as of right now... absent BitDefender and including MSE.

The happy ending is that it turns out I do NOT have to do anything additional to both of my HTPC's to get them back to "normal". There is no hardware issue, no WMC or extender-related software issue, no Windows corruption. It was BitDefender's recent product enhancement which was 100% the culprit.

Happy ending, thankfully. Paired 115/117 events are once again occurring. Now I can get some sleep again.

Image

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#9

Post by DSperber » Tue Jun 25, 2019 4:39 pm

Space wrote:
Thu Jun 06, 2019 5:25 am
Yeah, I remember some discussion about the file being in use when copying it, but I don't remember if there was any verdict on if it was safe or not. I seem to recall that if you can copy it without error, it should be good, but don't quote me on that.

Also, when restoring it, I think you can stop one of the services, restore the file and then start the service again instead of doing it in safe mode, but again, I don't remember the details, and since this should be a rare occurrence, doing it in safe mode should not be a big deal.
So here's a question: why is the size of MSPR.HDS always the same???

Image

I happened to be curious about whether or not my nightly backups (which of course include the updated MSPR.HDS available at the time the backups get run) were successfully capturing the up-to-date version of this file, so that in case I needed to restore it would it be usable and up-to-date. I wanted to know if it was necessary to copy/backup and/or restore only in SAFE MODE in order to ensure that the results were as expected. Or, could I just treat this as an ordinary file, even though WMC is active 24/7.

Now back some years I used to have a problem with WMC unable to get to the Ceton tuners (which are network devices, accessed through TCPIP and 192.168.200.1/2) whenever for my work I was VPN-connected to a remote customer computer, through assorted VPN software products. They all seemed to shut down some/all outside network and internet connectivity, probably in the interests of "security" and how VPN facilitated that security. So while VPN connected, I was for example not also able to check my email or browse the internet, and certainly could not be recording TV programs using the Ceton tuners. My workaround was simply to use my laptop to do this VPN-related remote work, rather than using my HTPC.

Anyway, one of the side consequences of using VPN on the HTPC was that once the VPN connection was closed, for some reason the Ceton drivers did not just bring back access and normal functionality to the Ceton tuners. I had reported this to Ceton (back years ago when they were still alive) but they were baffled, and certainly never provided any solution.

Now initially the only solution I had to this "loss of Ceton tuners after VPN" was to re-boot the machine. Obviously annoying and undesirable, but I didn't know any other way to kick-start the Ceton tuners back into a usable state, and it definitely was not happening "naturally" through any number of actions I might take to try and wake it back up.

And then I discovered a "solution", which worked successfully 99.9% of the time. Only in the rarest of circumstances was it unsuccessful, requiring me to re-boot. The "solution" was a series of two commands (which I named VPN1.bat and VPN2.bat) performed in sequence:

VPN1:

Code: Select all

taskkill /im ehrecvr.exe /f
VPN2:

Code: Select all

net start ehrecvr
%systemroot%\ehome\ehprivjob.exe /ocurdiscovery /ex
Magically, this would first STOP and then START the WMC process, and in doing so would re-establish all TCPIP network connectivity to the Ceton tuners which were "rediscovered" by the second command.

I am guessing that doing the "taskkill" for ehrecvr.exe in the VPN1 command would also CLOSE all open files, including MSPR.HDS if it were open (as it always is, presumably). Then my NovaBACKUP could definitely for sure back it up with integrity. And then after the backup completes the VPN2 command could be issued to bring back WMC, and thereby re-open MSPR.HDS, etc. Of course this would be problematic if I really was using WMC to record something in the middle of the night when NovaBACKUP kicked off. And it might actually be completely unnecessary if the file could be read/backed-up successfully even if open in WMC as it seems to be, as I really do have non-null backup versions of it taken every night by NovaBACKUP.

So I was thinking about all of this, and just investigating and poking around. And that's when I noticed that at least for this particular HTPC, the size of MSPR.HDS doesn't seem to have changed in almost two months! I'ts been 18,354,176 bytes and despite the fact that many additional copy-protected recordings have been made, this "file" does not seem to change in size over time!! Why is that? Anybody else see a similar symptom?

Now that's not always the size of MSPR.HDS, just on this particular HTPC for the past two months. I actually have two other HTPC machines (one of which hasn't been used at all in two years, and the other of which hasn't been used to do any new WMC recordings for six months although it has been up and running Win7) and their MSPR.HDS file sizes are definitely different (although I don't know yet if they, too, remained unchanged over the last month or two of their active use, but will try and investigate that).

Image

Image

Anybody have more in-depth knowledge about MSPR.HDS, why it seems to remain the same size for extended periods despite new recording data injecting into it daily?

Space

Posts: 1558
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#10

Post by Space » Tue Jun 25, 2019 10:46 pm

The file size not changing is an interesting observation. Does the size remain the same even after a new copy protected recording is made and then the system is rebooted? It would seem to be a bad idea to not flush the buffers to the file whenever it is written to, as it would be vulnerable to a system crash/power outage.

Perhaps it allocates space to the file in blocks and only increases the file size when new blocks are needed. I know that with some databases you specify a database size at the start and it creates a file of that size, even if it is "empty". I have no idea if this is what is happening here, but it could allocate initial space and then only when the existing data fills that allocated space does it allocate more "free" space in the file. But this is all just supposition, as I am no database expert.

One thing you can do is do a binary comparison of the equal-size backup files to see if they are the same or if they are different. If you have Win7 (or perhaps Win10 as well) you can use the "FC" command to compare binary files. The output is just a list of each byte that differs in the two files:

FC /B file1 file2

Be sure to use the /B (binary) option or you may get a wall of binary characters sent to the screen.

There are also binary file compare tools with a visual interface, but I have not used them. This one seems to be free:
https://www.cjmweb.net/vbindiff/
Note that I think this still requires you to use the command line to specify the files as arguments, so it is not a true Windows GUI type program (it was ported from Linux).

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#11

Post by DSperber » Wed Jun 26, 2019 2:12 am

Well, here's interesting new information.

I used Beyond Compare to to a hex compare of those two MSPR.HDS files which were in my \Playready folder (and had the date suffixes), and they were absolutely significantly but not totally different inside. So the theory that probably in the interest of efficiency a large "block" of storage is used and re-used as appropriate, with entries getting populated when new recordings are made and then vaporized when those recordings are deleted, that theory makes very good sense and seems to be borne out by this comparison. And as you hypothesize, perhaps it only gets larger (by another large incremental amount) if the current storage is 100% full. Seems reasonable.

I then tried to compare the "live" current MSPR.HDS which is at this moment actually in-use by WMC as a copy-protected recording is being made. The comparison could not open MSPR.HDS because it was in-use by another process. This almost suggested that if a copy-protected recording were going on at 1AM every morning when my nightly backup takes place, that NovaBACKUP also would not be able to open the file in order to copy it. But then NovaBACKUP doesn't actually copy files "live" from where they exist. Rather, it uses VSS Shadow Copy to take an "instant snapshot" of the disk, and then takes its backup from the VSS Shadow Copy. This actually is what allows it to perform backups of open files, even though they're actually open and in-use. Now as to what the backed up data looks like for such an open-file situation, I don't know. But at least the backup can technically be performed. Hard to believe the VSS-supported backup of such a file could have "integrity", or is it simply whatever the file truly looks like at the exact instant when instant-copied into the VSS Shadow to to it? I don't know.

Anyway, I then restored (to another location) the MSPR.HDS which got backed up by NovaBACKUP last night, which is still the same size as all the others. And once again the comparison (which I obviously could now do since this file was not "in-use") showed that it is again significantly but not totally different from the other files.

This suggests that the apparent fixed size of MSPR.HDS on one system is probably just the way it is. Chances are excellent that if the number of copy-protected recordings were to grow significantly that MSPR.HDS would suddenly grow by several MB. And then it would remain there, either forever or until even more copy-protected recordings justified needing more storage again.

I will have to perform some careful experimenting at some off-hour time, when I'm not doing any copy-protected recording and can afford to re-boot to safe mode, take a backup of MSPR.HDS, etc., in order to compare the contents of the NovaBACKUP version of MSPR.HDS taken at 1AM vs. what it actually looks like when closed and not in-use, as would be the case when viewed in safe mode. I think that's the only way I will be able to prove that my regular nightly backups are sufficient protection, exactly as things are produced right now with no additional or special protections required. Or that they're not, and I need to do more in order to guarantee that I can always recover the latest MSPR.HDS if a disaster strikes, so as not to lose the access to all the copy-protected recordings reflected in that MSPR.HDS.

Space

Posts: 1558
Joined: Sun Jun 02, 2013 9:44 pm
Location:

HTPC Specs: Show details

#12

Post by Space » Wed Jun 26, 2019 4:31 am

I certainly hope that the license for deleted copy protected files is not removed from the mspr.hds file (at least for files that are "missing" and not deleted directly by WMC). I have several terabytes of copy protected recordings stored off-line. If WMC noticed those recordings missing and removed the license from the mspr.hds, that would not be good. I suppose, however, that if you delete the recording from within WMC, it may be OK to delete the license in that case, although I would bet that once the license is in there, it is never removed.

This can be tested by making a copy of a copy protected WTV file and then deleting the recording from within WMC. Then put the copy of that file back where the original was and see if you can play it.

As for shadow copy, integrity of open files is not maintained. Whatever was on the disk at the time the shadow copy was initiated is what gets backed up. Although I would assume the window for destroying the integrity of the file would be very short, as I assume the license is written after the recording is finished and only takes a short time (but this it just a total guess, and the fact that the file is locked while the recording is taking place may be a clue that I am wrong).

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#13

Post by DSperber » Wed Jun 26, 2019 6:21 pm

Well, had a chance to do a little testing.

First, it appears that once WMC starts and opens MSPR.HDS for whatever reason (e.g. to make a new recording, or perhaps even to view "live" TV for a copy-protected program), the file remains open and in-use indefinitely thereafter, even when the recording ends and nothing is happening.

So the only way to actually read it in Beyond Compare for the hex compare (against last night's 1AM backup) is to run the VPN1 command (which I showed previously). That kills EHRECVR.EXE and apparently that program/task is what had the lock on MSPR.HDS. So as soon as the task was killed I was now able to do the compare.

The results of the compare showed only about 80 bytes that differed in one way or another between last night at 1AM and now. The rest of the 18MB was identical.

Now I haven't made any new recordings in that window, but I did view and then delete four copy-protected recordings as I recall. So if we theorize that there are umpteen entries in this big file and they get deleted if I delete the recording through WMC, and then the empty entries get re-used (only growing the file by some large chunk if it ever fills up or maybe gets full above some threshold), maybe that implies each entry might be 16-bytes? Don't know. I don't know what would have re-filled those bytes with something new (other than perhaps some default string implying "available slot").

Nevertheless, all but those 80 bytes or so were identical. This certainly implies NovaBACKUP definitely WAS able to perform the backup via the VSS shadow copy, even as the genuine MSPR.HDS was itself still open and in-use (and would remain so forever, apparently, even if no recording is going on). I will perform another experiment that again "kills" EHRECVR to free up the file, shortly after 1AM when the backup took place. Then, without deleting anything or recording something new I will again do a hex compare against the backup out of NovaBACKUP. If the two files are in fact identical that's terrific. That means backups of MSPR.HDS can be done 24/7 and don't need to be run only in safe mode.

I would also like to perform one new recording, and then "kill" again, and compare the new MSPR.HDS against the backup copy, to presumably see exactly one new entry that got added somewhere inside the file. And I should also perform a "delete" of a single recording through WMC to see how that gets reflected in MSPR.HDS.

Your additional experiment of copying a copy-protected WTV recording, then deleting it through WMC, then copying it back, to see if it could still be read or not... I will get to that as well.


In passing, I mention one other thing.

After killing EHRECVR (and un-locking MSPR.HDS) using my VPN1 command, I then issued VPN2 to restart EHRECVR. Everything seemed fine and I assumed MSPR.HDS would once again get locked. It didn't. In fact I could still to the hex compare. Unexpected.

I then thought well maybe I need to kick-start it back to life, by doing something that would open it. This would probably include doing a recording of a copy-protected program, or maybe even just trying to live-view a copy-protected program. Well, when I tried to "live" view a copy-protected program (not a recording, but a live program) I got a copy-protected complaint from WMC (which seemed from its wording that it would be much more appropriate for trying to PLAY a copy-protected recording, rather than just viewing a copy-protected recording).

Anyway I couldn't view or record the program. I had to re-boot the machine in order to get EHRECVR started correctly and PlayReady working properly again. And of course now MSPR.HDS was once again locked.

I was sure my VPN2 command used to work properly and didn't produce the symptom I saw in tonight's test, but I will give it another shot before coming to any conclusion about what might have happened.

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#14

Post by DSperber » Fri Jun 28, 2019 6:51 pm

Well, I'm a bit confused.

For sure, it appears that once a copy-protected program has been recorded, its entry in MSPR.HDS appears to be retained even if the WTV file is then deleted through WMC. Because after deleting it this way, and then copying it back from where I'd preserved it I can once again use WMC to view it. Shows up in the list of recordings, proper thumbnail, can be played perfectly, everything totally normal.

Now my experiments this morning started from last night's backup, with no changes of any kind between last night at 1AM and now. And when I re-ran another "backup" of MSPR.HDS this morning sure enough there were no differences from last night's backup.

Also, whatever anomalies occurred the other day when using my VPN1 and VPN2 commands (to kill and re-start EHRECVR) well they didn't happen today (I have rebooted several times over the past few days, so I guess that must have just naturally cleaned up whatever was the issue a few days back). I used VPN1 to (I believe) ensure that MSPR.HDS was properly "closed" before taking the backup, just in case there was any in-memory buffered data not yet reflected on disk (seems unlikely) that needed to be flushed out to disk. I don't believe this should have really been necessary, but I don't think VPN1/VPN2 hurts anything and makes me more comfortable that any copying of MSPR.HDS would be up-to-date.

Now mysteriously, I actually deleted TWO copy-protected recordings. I had set up my test backup to include MSPR.HDS as well as the one copy-protected WTV file I was going to use as my test data for deleting and then restoring to see if it could still be played. Well, for some reason, NovaBACKUP failed to back up the copy-protected WTV file! Didn't tell me it skipped it, just didn't copy it. I don't know why yet but I'm researching. I didn't double-check to ensure it had been backed up, so when I deleted it I actually permanently lost it. Fortunately it was a very recent recording and there's another airing on Saturday that I've already scheduled in order to recover the program recording. So that's why I had to work with a second copy-protected recording, but this time manually COPY'd myself to preserve it to play with.

So what this means is that I was tracking MSPR.HDS between last night's backup, this morning's "duplicate", and then after deleting the first copy-protected WTV file, and then after deleting the second copy-protected WTV file. Definitely doesn't seem to be one or two entries which "disappeared" as a result of WMC's deleting of the two recordings, and this is confirmed by being able to play the second deleted recording (which I'd manually preserved myself) after manually copying it back into \Recorded TV. Hence my apparent confirmation that once a copy-protected recording is made, its Playready key data seems retained forever (or so it seems). And that means it can be played forever, as long as that MSPR.HDS which contains its key is preserved.

What's mysterious is that there still was that 80 bytes or so of data which appears to have changed, immediately upon deleting the first recording. I thought this meant its entry was actually "emptied out" or otherwise marked for re-used, which would suggest it would then not be playable again after restoring it. Unfortunately I'd lost the file (since NovaBACKUP hadn't actually backed it up), so I couldn't perform any further experiments using this file to see what happened after restoring it. Hence why I moved on to the second recording, this time being more careful to preserve it first.

Amazingly, even after deleting the second entry, again there was no "second 80 bytes" that to changed. In fact the same first 80 bytes was changed again! This kind of suggests maybe this area of data is holding some information about the overall catalog of data represented by MSPR.HDS? Even after tracking MSPR.HDS following first one and then a second delete (using WMC to delete recordings, not me deleting WTV files manually) I still can't explain what's happening with these 80 bytes, and whether or not to expect some change here if I do something in WMC.

Even more remarkable, I then added a short 1-2 minute new copy-protected recording made just now. I honestly expected to see a change to MSPR.HDS, reflecting the addition of a new key so that this recording could be played. Well, no change to MSPR.HDS!! Seems impossible. I used VPN1/VPN2, but there still doesn't seem to have been a difference between MSPR.HDS before and after adding this brand new copy-protected recording. I'm baffled. Perhaps I'm not really getting up-to-the-minute copies of MSPR.HDS, because it does seem impossible that there wouldn't be a change after creating a new recording... and yet it seems there wasn't.

For the moment I'm going to take a break for a day, and then repeat this all over again tomorrow. I may just decide to use re-booting, Safe mode, etc., to get the sequence of copies of MSPR.HDS that I need to 100% guarantee are what I expect them to contain, in order to draw objective factually accurate conclusions about what the internal contents are of MSPR.HDS, and how it gets used when copy-protected recordings are either deleted or added.

So for now there's apparently still just that one seemingly absolutely proven question regarding "can I play a copy-protected recording after deleting it with WMC, if I restore it manually from some backup/copy"? And the answer seems to be yes.

And I need to pursue why NovaBACKUP doesn't actually backup a copy-protected WTV file. Will it backup an unprotected WTV file? Does it have to do with the \Recorded TV folder being "special" and off-limits for backing up (for example, the program will not back up anything in its own configuration folder for some inexplicable reason)?

More as the story develops.

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#15

Post by DSperber » Fri Jun 28, 2019 6:59 pm

Well, answers to the NovaBACKUP mysteries... even if unsatisfying.

Turns out backing up a WTV file, either copy-protected or unprotected, will not occur from \Recorded TV. This appears identical to what happens when trying to back up the configuration folder of NovaBACKUP, namely that all of the "checked" files are simply ignored. They don't count as "skipped" or "selected", but are simply ignored as if you hadn't ever checked them.

In contrast, if I do a backup of the identical two WTV files (i.e. samples of both copy-protected and unprotected) from my "preservation folder" (i.e. not \Recorded TV), well now sure enough both files are backed up normally.

I need to take this up with NovaStor, to find out if they are aware of this and if it was "by design", or if it is somehow the result of something Win7 is doing.

DSperber

Posts: 228
Joined: Thu Jan 16, 2014 1:35 am
Location: Marina Del Rey, CA

HTPC Specs: Show details
2+ Yrs TGB Veteran

#16

Post by DSperber » Fri Jun 28, 2019 8:25 pm

Heard back from NovaStor. Turns out the issue is that their backups are performed from the VSS shadow copy, and the omission of \Recorded TV is specified by Microsoft in the Registry entry for this:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshot

This is documented in this MS article.

And, sure enough, that's what my Registry entry looks like, with \Recorded TV excluded:

Image

I guess that explains this particular anomaly. I still think they should "warn" me if file I've checked as wanting to be backed up wasn't found in the VSS shadow copy and so isn't on the backup, you'd think I would want to be told about that now rather than later when I go to restore it in order to recover from a disaster.

Anyway, at least I now know what is causing it.

Post Reply