Cryptographic Improvements in ASP.NET 4.5, pt. 2

Thanks for joining us for day two of our series on cryptography in ASP.NET 4.5! In yesterday's post, I discussed how ASP.NET uses cryptography in general, where key material is pulled from and how it is stored, and various problems that the APIs have introduced over the years. In today's post, I'll discuss how we're mitigating those issues using 4.5's opt-in model. The series outline is copied below for quick reference.

1. Background regarding the use of cryptography in ASP.NET 4.
2. Changes that were introduced in ASP.NET 4.5 (today's post).
3. Usage notes and miscellaneous Q&A.

Throughout the series I'll refer to a sample solution. This Visual Studio 2012 solution contains projects that demonstrate many of the core concepts mentioned here. It can be downloaded from http://sdrv.ms/T4aMyg.

The world in ASP.NET 4.5

Please keep in mind that everything discussed in this section is opt-in. New ASP.NET applications created using the 4.5 project templates will get the behavior described here. Existing ASP.NET 4 applications will retain their existing behavior, even when running on a machine with 4.5 installed. See the compatibility section later in this post for more information.

Transforming the auto-generated machine key, redux

When ASP.NET is configured to use 4.5 machine key compatibility mode, the manner in which keys are generated and transformed changes drastically. I'll discuss the symmetric encryption key transformation in detail, but the same applies to the validation key transformation.

For starters, using an auto-generated key causes us to read a full 256 bits of key material from the value stored in HKCU, compared with 192 bits in ASP.NET 4. (Recall from the earlier discussion that the default symmetric encryption algorithm is AES.) Assume that those bytes are:

82 7d b1 00 6c 22 4a 21 eb 81 33 1a 1f 85 19 b0 68 1f fb e1 bb 08 be f0 48 4e 27 a9 fe e3 c8 6f

This key is not used directly as a cryptographic key in the product. Instead, it is used as the key derivation key (KDK) into a key derivation function (KDF). We internally use the NIST SP800-108 [PDF link] counter-mode KDF with HMACSHA512 serving as the pseudo-random function (PRF), and this KDF has been baked into almost all of the 4.5 core crypto code paths. The label and context inputs are populated with the application virtual path or application ID if IsolateApps or IsolateByAppId is specified. The KDF is then run to generate what we refer to as an application master key. In the above example, the application master key might come out as:

db d5 30 f4 9c 5e d7 dc 5e a9 40 a3 dd 0b 06 1a 9d 68 f1 a2 60 34 59 e5 3a a9 83 b0 20 b0 aa 93

Note that the algorithm has preserved all 256 bits of entropy, even though IsolateApps or IsolateByAppId may have been involved in the conversion from auto-generated key to application master key. Because we're using a proper KDF, one can envision future support for additional transformation flags, and we could support limitless such flags without further reducing the strength of the auto-generated key. Alternatively, if an explicit key has been specified in Web.config, we'll just use that as the application master key rather than running it through the KDF to generate a master key.

Terminology note: the application master key is actually a set of keys: one used for a symmetric encryption algorithm, the other used for a message authentication algorithm. When I mention transforming or consuming them, imagine the transform being done to each in parallel.

Of primitives and derived keys

In ASP.NET 4's crypto code paths, symmetric encryption and HMAC calculation were kept separate. The APIs were disjointed, and callers had to verify that they were calling into the appropriate primitives in the correct order. (That we left this up to the individual callers rather than putting it in a centralized code path was one of the factors that led to MS10-070.) In ASP.NET 4.5's code paths, these two operations are now inextricable. The caller does not specify if he would like the data to be encrypted or MACed; the code paths automatically handle the correct invocation of both.

One immediate implication of the above is that payloads run through the new system can no longer be only MACed or only encrypted; they must be both or neither. Thus even though the default behavior of ViewState is MAC-only, when run through the 4.5 code paths it will always end up being both encrypted and MACed. If ViewState MACing is disabled by setting EnableViewStateMac to false, then ViewState will be afforded no protections.

Never set EnableViewStateMac to false in production. Not even for a single page. No exceptions! The EnableViewStateMac switch will be removed in a future version.

We also modified all of our internal call sites to pass one additional piece of information to the core crypto routines: their purpose. These purpose objects are basically strings that describe the caller. Some example purposes used internally by ASP.NET are equivalents for "ScriptResource.axd", "FormsAuth ticket", and "ViewState for ~/default.aspx". Importantly, if a cryptographic payload was generated with a particular purpose string, that same string must be provided when trying to verify and decrypt the payload, otherwise the operation will fail.

This is implemented internally by using the same NIST SP800-108 KDF mentioned earlier. Recall that each application has an application master key. That master key is itself used as a KDK, and label and context parameters are populated from the provided purpose string. The KDF is then used to generate a new derived key, and this derived key is used to carry out the requested cryptographic operation. Thus even a trivial change to a call site's purpose string will result in the generation of wildly different key material, the end result being that a purpose of "ViewState for ~/about.aspx" cannot be used to decipher a payload protected with "ViewState for ~/default.aspx" or any other non-matching purpose.

We expect that this feature alone will provide considerable defense in depth. By isolating the cryptographic consumers from one another we hope to contain any future bugs or flaws that are found in the product. For example, if a bug is found in ScriptResource.axd or ViewState, these changes drastically reduce the risk that the bug can be used by an adversary to attack the FormsAuthentication component.

Introducing DataProtector

In ASP.NET 4, we provided the ability to replace the symmetric encryption and message authentication algorithms used by the cryptographic pipeline. This is still supported in 4.5, and we will use the specified algorithms when encrypting and MACing data. To specify your own algorithms, change the <machineKey> element like so:

<machineKey decryption="alg:typename" validation="alg:typename" />

To specify a custom symmetric encryption algorithm, replace typename in the decryption attribute above with the assembly-qualified name of a type subclassing SymmetricAlgorithm. Similarly, to specify a custom message authentication algorithm, replace typename in the validation attribute above with the assembly-qualified name of a type subclassing KeyedHashAlgorithm. We will create instances of these types and set their Key properties as appropriate. (The Key properties are populated with the output of the KDFs.)

In addition to replacing the individual algorithms, ASP.NET now also allows wholesale replacement of the entire crypto pipeline. This functionality is provided by the new .NET 4.5 DataProtector type. Subclassed types implement the abstract ProviderProtect and ProviderUnprotect methods; these methods are responsible for black-box protection of arbitrary data. To use a DataProtector-derived type instead of ASP.NET’s built-in KDF / encrypt-then-MAC logic, specify the type in Web.config:

<machineKey applicationName="applicationName" dataProtectorType="typename" />

The meaning of these attributes is as follows:

The sample solution includes a project DpapiProtectorDemo which demonstrates use of a DataProtector type for protection. That project uses the built-in DpapiDataProtector type, which uses the DPAPI functionality provided by Windows to encrypt and tamper-proof data using keys specific to the current local Windows user account. The DataProtector constructor parameters are populated from a combination of the application name specified in <machineKey> and the purpose string passed to the cryptographic APIs. One benefit to using DPAPI in this manner is that the OS itself is responsible for key management, relieving ASP.NET developers of this need. But DPAPI keys are local to the current user on the current machine, making this technique unsuitable for web farm deployments. There are some ways around this; tomorrow's post contains a section on advanced usage and speaks further to this point.

Since DataProtector behavior may be tied to a particular Windows user account, the ASP.NET runtime will impersonate the application identity before instantiating and calling into a DataProtector. This impersonation is reverted immediately after the call, restoring any impersonation that existed before the call.

There is a caveat to replacing the stack: WebResource.axd and ScriptResource.axd URLs will not go through any configured DataProtector. The payloads have unique caching requirements that require any given plaintext to always result in the same ciphertext, and since DataProtector implementations are expected to include some type of randomness in their protected output this makes DataProtector ill-suited for protecting these particular payloads. WebResource.axd and ScriptResource.axd will honor custom SymmetricAlgorithm or KeyedHashAlgorithm configurations, however, and they will continue to go through the same KDF transformations that the application master keys are otherwise subject to.

MachineKey API changes

New for ASP.NET 4.5 are additional APIs on MachineKey: Protect and Unprotect. These APIs are similar to the original Encode and Decode methods, but they take advantage of the features mentioned so far in this section:

• The caller no longer needs to know whether something needs to be encrypted or MACed. The payload is simply "protected".
• The caller can supply one or more purpose strings to isolate this specific consumer from others. The MSDN documentation for MachineKey.Protect provides good guidance for choosing purpose strings.
• If a DataProtector is configured, calls to MachineKey.Protect / Unprotect will be routed through the configured DataProtector.

When unprotecting data, the same purpose string that was used to protect it must be provided. Consider the following:

byte[] plaintext = ...; 
byte[] ciphertext = MachineKey.Protect(plaintext, "foo component"); 
byte[] deciphered = MachineKey.Unprotect(ciphertext, "bar component");

The last line above will throw a CryptographicException since a different purpose string was used for protection and unprotection, signaling that the decryption is taking place in the wrong context. The end result is that callers gain automatic payload differentiation. The Protect / Unprotect APIs automatically prevent the caller from passing in a purpose string that is used by the ASP.NET runtime itself (e.g., for FormsAuthentication).

The Encode and Decode APIs have been deprecated, and we intend for these new APIs to be their long-term replacements. We hope that you find these APIs useful, that you feel confident in using them, and that they lend themselves well to the pit of success.

Algorithmic implementations

Finally, since .NET 4.5 requires Windows Vista / Server 2008 or higher, we are able to make some assumptions that simplify the code and lead to improved performance. One result is that we are able to standardize on CNG almost everywhere. This is beneficial since this layer is where the Windows team intends to add most of their low-level extensibility hooks and focus most of their performance work. For example, many CNG routines contain hand-rolled assembly instructions that take full advantage of the particular instruction sets available on the target processor.

Compatibility

Opting in or out of the 4.5 code paths

As you might imagine, such drastic changes to the crypto pipeline come at the expense of compatibility. And since .NET 4.5 is an in-place update to .NET 4, we cannot enable these new behaviors by default, otherwise we run the unacceptable risk of breaking existing applications.

To opt in to the new ASP.NET 4.5 behaviors, all that need be done is to set the following in Web.config:

<machineKey compatibilityMode="Framework45" />

Alternatively, you can set the following switch, which is what the ASP.NET 4.5 project templates do:

<httpRuntime targetFramework="4.5" />

The above switch is responsible for a slew of runtime behavioral changes, but that is a blog post for another day. The important bit here is that setting the target framework to 4.5 in the <httpRuntime> element automatically implies a default setting of Framework45 for the <machineKey> compatibility mode unless the machine key compatibility mode has been explicitly specified.

ASP.NET has historically supported sharing forms authentication tickets between different versions of the framework. This allows tickets to be generated by an application running ASP.NET 2.0 and validated by an application running ASP.NET 4, for example. If you are writing an application targeting ASP.NET 4.5 (you have set <httpRuntime targetFramework="4.5" />) and you need to share tickets with applications running earlier versions of ASP.NET, you must set the following in the 4.5 project's Web.config:

<machineKey compatibilityMode="Framework20SP1" />

The value Framework20SP1 is the default machine key compatibility mode for all ASP.NET versions. This has the effect of using the legacy crypto code paths, even if .NET 4.5 is installed on the machine. An existing ASP.NET 4 application that happens to be running on a machine with 4.5 installed will not get the new behaviors automatically since neither <httpRuntime targetFramework="4.5" /> nor <machineKey compatibilityMode="Framework45" /> would be present in that application's Web.config. If, however, you have made a new application targeting 4.5 (and as such it has those config settings) and need to maintain forms authentication ticket compatibility with existing applications, you can set Framework20SP1 to be interoperable with earlier versions of ASP.NET

Behavioral miscellany

The following consumers and APIs will always go through the legacy code paths, regardless of whether the application is otherwise operating in 4.5 machine key compatibility mode:

• Membership, if configured to use reversible encryption when storing passwords in the database (please don't do this!). This was done to preserve backward compatibility.
• MachineKey.Encode / Decode, since developers may depend on their ability to decode arbitrary payloads that have been generated by other ASP.NET runtime components. It is also conceivable that the output of the Encode method might be persisted, and we did not want to render these payloads unreadable.

The following consumers and APIs will always go through the new code paths, regardless of whether the application is otherwise operating in 4.5 machine key compatibility mode:

• MachineKey.Protect / Unprotect, as these are new APIs and are not bound by the same compatibility requirements as the legacy APIs.

We also apply some restrictions on the <machineKey> configuration when the compatibility mode is set to 4.5:

• The <machineKey> validation value must be SHA1 (which we interpret as HMACSHA1), HMACSHA256, HMACSHA384, HMACSHA512, or a reference to a KeyedHashAlgorithm-derived type. The values AES, 3DES, and MD5 are no longer allowed, as they were in ASP.NET 4.
• The decryptionKey and validationKey values – if specified – must be well-formed hex strings. The 4.5 routines are stricter about this than the downlevel routines were.

Tomorrow's post will provide usage notes and go over some advanced scenarios for these new APIs. I will demonstrate how these APIs can be used with new cryptographic routines in Windows 8 to simplify key management for site administrators. I'll also go over some miscellaneous Q&A regarding this design and some reasons why this particular design was chosen over alternatives.