The Shadow Key Companion

Shadow Keys have been around for a very long time and many (including myself) have written about this topic. I feel it is time to compile all the information about shadow keys in one place to provide a comprehensive overview. This article will tell you about the concept of shadow keys, how they affect x64 and why some applications get around writing shadow keys at all.

What Shadow Keys are and How They Work

Whenever a RDS server (formerly Terminal Server) detects an installation, it switches to install mode and captures all new settings writting into HKCU. They are duplicated (shadowed) into HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Terminal Server\Install. The RDS server can be forced into install mode by [change user /install](http://support.microsoft.com/kb/186504/EN-US/) and back into execute mode [change user /execute](http://support.microsoft.com/kb/186504/EN-US/).

On 64 bit servers, shadow keys are written to different locations based on the bitness of the executable:

When conflicting keys are present in those two locations, those in the 32 bit location win.

Update: Helge Klein has tested shadow keys on Windows 8 / Windows Server 2012 to confirm that they work as before.

How Shadow Keys are Merged in Detail

Whether shadow keys are merged into a user profile is determined by comparing a timestamp in the user profile and a second timestamp stored with the shadow keys:

When the user profile contains an outdated copy of the shadow keys, only those are applied again that have a larger timestamp on the individual shadow key.

Issues Introduced by Shadow Keys

This heading is actually misleading because shadow keys are not an issue if your servers are never re-installed. But sooner or later you will migrate to a now OS or need to rebuild a server due to some hardware or software issue. At that point in time, shadow keys may begin to cause trouble.

When a server is re-installed, shadow keys are created from scratch in the registry of the server. They also contain a current timestamp which is newer that the timestamp in the profiles of your users. Consequently, whenever a user logs on to the new server, the shadow keys ar eapplied to his user profile yet again causing existing values to be overwritten. Microsoft has documented this a long time ago. In some cases, users may even see registry settings of administrators who have configured applications during the installation of another application causing additional registry keys to be included in the shadow keys.

For a long time, the remedy has been resetting the timestamp of shadow keys on new servers to a ridiculously early date. This effectively causes shadow keys to be applied once for new user profile and never again for existing user profiles. Microsoft has provided a tool called sdt.exe. Unfortunately, this tool is awfully hard to get your hands on.

In addition, sdt.exe does not handle 64 bit systems correctly. Therefore, shadow keys will become an issue again when 64 bit applications produce shadow keys. Why only 64 bit apps? Because sdt.exe is a 32 bit executable it will reset 32 bit shadow keys.

How to Handle Shadow Keys

The old approach: For some time now, many installations of RDS and XenApp servers have used sdt.exe to reset the timestamp stored with the shadow keys. But this does only address those issues for 32 bit applications. And we will see a growing number of 64 bit applications in the future as vendors adopt to the (once new) platform.

Update: Holger Adam from sepago’s Research & Development has created two free tools to read and set the timestamps of shadow keys which do not have the limitation mentioned above.

The radical approach: I have written about a very radical approach a few years ago stating that shadow keys are to be replaced by Group Policy Preferences (GPP). Again this is not an ideal solution because it requires capturing the default registry settings and transferring them into a group policy. Those settings are then deployed separately from the application package. This results in maintaining two separate locations when the application changes.

The obscure approach: I have also written about a very poorly documented parameter called TermSrvCopyKeyOnce that can be used to force certain shadow keys to be copied only once. When a REG_DWORD called TermSrvCopyKeyOnce is created and set to 0x1, all values contained in this key are copied only once per user. Note that TermSrvCopyKeyOnce does not work recursively but only affects the values next to it.

The laborious approach: For some time now, Microsoft has published another approach to handling shadow keys. In the section “Registry Paths”, KB 186499 describes compatibility flags that can be created for individual registry paths. The behvious is the same as for TermSrvCopyKeyOnce except that it is maintained in a separate location. By the way, this approach was first mentioned as comment to Brian Madden’s article about sdt.exe.

How to proceed: Any of the approaches presented above may work for you – just as you will have to decide for yourself and at your own risk. I would rather go with the documented approach in KB 186449 (laborious approach) but I like TermSrvCopyKeyOnce better.

Why You May Not See Shadow Keys

There has been some confusion why some applications produce shadow keys while other do not. Helge Klein has recently blogged about the flag TS_AWARE for executables. When this flag is set, the executable is excluded from shadowing user settings in install mode. Also note Helge’s tool for reading the TS_AWARE flag from executables.

Update: System tools like reg.exe, regedit.exe, cmd.exe and powershell.exe are flagged TS_AWARE which may cause your installation scripts to produce unexpected results as no shadow keys are created by them!

References

Brian Madden’s detailed but lengthy description of shadow keys

Feedback is always welcome! If you'd like to get in touch with me concerning the contents of this article, please use Twitter.