SQL and more with KRUTI: SQL Server Encryption

Encryption is a methodology used to hide confidential information from any illegitimate user is such a way that they do not see the actual contents of the information. Rather they get access to some cipher text. This text can be decrypted to its original form by legitimate users by the use of certain algorithms and secret code known as encryption keys or passwords.

Windows OS Encryption Interface

Starting windows 2000 an API is provided by the operating system to accommodate such data protection mechanism.

How does the operating system do this? In a nutshell there are 2 functions namely CryptProtectData() & CryptUnprotectData(). As the name suggests first one will be used to encrypt the data the latter will be used to decrypt the cipher text.

Both these functions are available under Crypt32.lib library. If the library is not linked to the application that needs to use these windows functions one can dynamically call the Crypt32.dll dynamic link library in their application.The functions run in the security context of the Local Security Authority (LSA). For those who are unaware LSA is a system process that loads when the system is started and runs till the system is shut down.

The CryptProtectData() requires a password to create the cipher. The LSA provides the existing user's password hash. Obviously all applications running under the same user account will be able to access these functions and decrypt any protected data if aware of the location under the same context.The operating system however only provides the mechanism for data protection. This implies that the applications calling these functions should implement their own storage for protected data. When you aren’t aware of the location where the cipher text is stored; the functions and password hash serve no purpose to get unauthorized access to sensitive information. But it’s certainly possible hence DPAPI allows an application to use an additional secret when protecting data.

This additional secret is then required to decrypt the data.

DPAPI generates MasterKeys which are in turn used to generate session keys. These keys along-with some random data & encryption algorithm are used for data encryption. The operating system stores this random data and on discovery of any cipher text uses this random text to generate the session key and decrypt the actual data.

One may wonder how does DPAPI work and get the same MasterKey back if the password is changed. This has also been taken care of by the operating system so we need not worry. I’d recommend you read the below article to understand in depth the WindowsOperating System DPAPI functionality.

This master key is used by SQL server to create something called as Service Master Key (SMK). The SMK sits at the top of hierarchy of SQL server encryption.

Encryption in SQL server.

The SQL server service master key (SMK) is generated at the time of SQL server setup with the master key from Windows DPAPI and the service account configured credentials provided for setup. One service master key per instance.

Note: SMK is a symmetric key.

One will find its entry in the below table

USE master;

SELECT *

FROM sys.symmetric_keys

WHERE name = '##MS_ServiceMasterKey##';

The service master key is used to encrypt the Database Master Key (DMK). The DMK is used to generate asymmetric keys, symmetric keys and certificates in SQL server. Microsoft depicts this beautifully in the below Microsoft diagram.

Thus we have 3 techniques to encrypt data in SQL server

1) Symmetric Keys
Symmetric keys use the same key to encrypt and decrypt data.

2) Asymmetric Keys
Asymmetric key ideally uses a public key to encrypt data and a matching private key is used to decrypt the data. It can also be used vice versa but only the matching pair of keys can decrypt the data encrypted by other.

3) Certificates.
Certificates are containers for the user's public key which store additional data identifying the organization that issued the certification.

There is a utility called certmgr.msc used to view/install certificates on the current machine.

Until SQL server 2005 these were the only 3 techniques for encryption. With the advent of SQL server 2008 SQL server introduced 2 new techniques for encryption namely

4) Transact-SQL functions

This feature is available in all editions of SQL server. This is also known as Cell-Level Encryption /Column Level Encryption in SQL server. SQL server makes use of the below encrypt,decrypt functions to protect data within columns of record.

The encryption functions are:

· ENCRYPTBYKEY, which uses a symmetric key to encrypt data

· ENCRYPTBYCERT, which uses the public key of a certificate to encrypt data

· ENCRYPTBYPASSPHRASE, which uses a passphrase to encrypt data

· ENCRYPTBYASYMKEY, which uses an asymmetric key to encrypt data

The decryption functions are:

· DECRYPTBYKEY, which uses a symmetric key to decrypt data

· DECRYPTBYCERT, which uses the private key of a certificate to decrypt data

· DECRYPTBYPASSPHRASE, which uses a passphrase to decrypt data

· DECRYPTBYASYMKEY, which uses an asymmetric key to decrypt data

· DECRYPTBYKEYAUTOASYMKEY, which decrypts data by using a symmetric key that's automatically decrypted with an asymmetric key

· DECRYPTBYKEYAUTOCERT, which decrypts data by using a symmetric key that's automatically decrypted with a certificate

5) Transparent Data Encryption

The entire database is encrypted both the data and the log files. Data at rest is encrypted. Data transformation happens in memory by SQL server; thus the application using the database need not bother about the encryption decryption process. This feature is only available in Enterprise Edition. Backups of these databases are also encrypted.

In the continued blogs we learn how to encrypt data using each of these techniques.

Windows DPAPI

https://msdn.microsoft.com/en-gb/library/ms995355.aspx

http://www.overclock.net/t/1293731/windows-data-protection-api-c-and-c

Encryption Hierarchy

https://msdn.microsoft.com/en-us/library/ms189586(v=sql.130).aspx

https://msdn.microsoft.com/en-us/library/ms189586(SQL.90).aspx

Choosing and algorithm

https://msdn.microsoft.com/en-us/library/ms345262.aspx

Encryption types

http://sqlmag.com/database-security/sql-server-encryption

http://sqlmag.com/database-security/sql-server-encryption-options