Designer's Favorite New Features in SQLServer
0 likes134 views
The document discusses new SQL Server features from the perspective of database design, focusing on security enhancements like Always Encrypted and Dynamic Data Masking. It emphasizes the importance of data governance, privacy, and risk management in database design, alongside tools like Azure Data Catalog and Azure Purview. The presentation highlights that every design decision should consider costs, benefits, and risks while maintaining a strong emphasis on data protection and security principles.
![AZURE SQL DB LEDGER TABLE – APPEND ONLY
CREATE SCHEMA [AccessControl]
CREATE TABLE [AccessControl].[KeyCardEvents]
(
[EmployeeID] INT NOT NULL,
[AccessOperationDescription] NVARCHAR (MAX) NOT NULL,
[Timestamp] Datetime2 NOT NULL
)
WITH (LEDGER = ON (APPEND_ONLY = ON) );](https://image.slidesharecdn.com/karensfavfeaturessqlserver2023-230613230851-8a5cd298/85/Designer-s-Favorite-New-Features-in-SQLServer-43-320.jpg)
![AZURE SQL DB LEDGER TABLE – UPDATABLE
CREATE TABLE [Account].[Balance]
(
[CustomerID] INT NOT NULL PRIMARY KEY CLUSTERED,
[LastName] VARCHAR (50) NOT NULL,
[FirstName] VARCHAR (50) NOT NULL,
[Balance] DECIMAL (10,2) NOT NULL
)
WITH
(SYSTEM_VERSIONING = ON, LEDGER = ON);](https://image.slidesharecdn.com/karensfavfeaturessqlserver2023-230613230851-8a5cd298/85/Designer-s-Favorite-New-Features-in-SQLServer-45-320.jpg)
Designer's Favorite New Features in SQLServer
- 1. A DATABASE DESIGNER’S FAVOURITE NEW SQL SERVER FEATURES WITH SOME AZURE STUFF, TOO KAREN LOPEZ @DATACHICK
- 2. KAREN LOPEZ Karen has 20+ years of data and information architecture experience on large, multi-project programs. She is a frequent speaker on data modeling, data-driven methodologies and pattern data models. She wants you to love your data.
- 3. “Every design decision comes down to cost, benefit and risk.” - Karen Lopez
- 5. DATABASE DESIGNER Data architect Data Modeller Development DBA Accidental DBAs Data Steward/Curator ….
- 6. 10 TOP WEB SECURITY RISKS https://owasp.org/www-project-top-ten/
- 7. Azure Data Catalog - https://azure.microsoft.com/en- ca/products/data-catalog Azure Purview - https://azure.microsoft.com/en- in/products/purview/ WHAT’S HAPPENING WITH DATA GOVERNANCE CATALOGS AND COCKTAILS?
- 8. Require Data Governance Programs Require Require Chief Data Officer like roles Require Require Data Inventories Require Require Data Lineage from data source to data use Require PRIVACY AND DATA PROTECTION LEGISLATION NOT JUST ABOUT BACKUPS OR ENCRYPTIN
- 9. GOVERNANCE Security at the data level Models capture security & privacy requirements Management reports of reviews Measurement In other words, Governance
- 10. DATA MODELS • Karen’s Preference • Track all kinds of metadata • Live • Advanced Compare features • Support DevOps and Iterative development • Support Conceptual, Logical and Physical design
- 11. DATA QUALITY IS ALSO DATA PROTECTION 11
- 12. 12
- 13. ROI 13
- 14. SECURITY IN DDL BECAUSE IT’S 2023
- 15. SECURITY – SQL 2016 Cell level TDE Always Encrypted Data Masking* Row Level Security 2016!
- 16. ALWAYS ENCRYPTED
- 17. Always! Security – Always Encrypted
- 19. SECURITY – ALWAYS ENCRYPTED Enabled at column level Protects data at rest *AND* in memory Uses Column Master Key (client) and Column Encryption Key (server)
- 21. SECURITY – ALWAYS ENCRYPTED Foreign keys must match encryption types Client code needs to support AE (currently this means .NET 4.x or above) 21
- 22. WHY WOULD A DB DESIGNER LOVE IT? Always Encrypted, yeah. Allows designers to not only specify which columns need to be protected, but how. Parameters are encrypted as well Built in to the engine, easier for Devs
- 23. DYNAMIC DATA MASKING REALLY MORE OF A PRIVACY FEATURE THAN A SECURITY ONE
- 24. SECURITY – DYNAMIC DATA MASKING
- 25. DATA MASKING EXAMPES XXXX XXXX XXXX 1234 kxxxxxx@ixxxxx.com $99,9999 June, 99, 9999 KXXXXX Lopez 25
- 26. SECURITY – DYNAMIC DATA MASKING CREATE TABLE Membership( MemberID int IDENTITY PRIMARY KEY, FirstName varchar(100) MASKED WITH (FUNCTION = 'partial(1,"XXXXXXX",0)') NULL, LastName varchar(100) NOT NULL, Phone# varchar(12) MASKED WITH (FUNCTION = 'default()') NULL, Email varchar(100) MASKED WITH (FUNCTION = 'email()') NULL);
- 27. DYNAMIC DATA MASKING Done at column level (NOT ENCRYPTION!) Data in the database, at rest, has no protection. Meant to complement other methods Performed at the end of a database query right before data returned Performance impact small
- 28. DYNAMIC DATA MASKING 4 functions available. today • Default • Email • Custom String • Random
- 29. DDM FUNCTIONS Function Mask Example Default Based on Datatype String – XXX Numbers – 000000 Date & Times - 01.01.2000 00:00:00.0000000 Binary – Single Byte 0 XXXX 0 01.01.2000 00:00:00.0000000 0 Email First character of email, then Xs, then .com Always .com Kxxx@xxxx.com Custom First and last values, with Xs in the middle kxxxn Random For numeric types, with a range 12 29
- 30. DYNAMIC DATA MASKING Data in database is not changed 01 Ad-hoc queries *can* expose data 02 Does not aim to prevent users from exposing pieces of sensitive data 03 30
- 31. DYNAMIC DATA MASKING Cannot mask an encrypted column (AE) Cannot be configured on computed column But if computed column depends on a mask, then mask is returned Using SELECT INTO or INSERT INTO results in masked data being inserted into target (also for import/export) 31
- 32. NEW: GRANULAR PERMISSIONS SPECIFIC COLUMNS TABLE DATABASE SCHEMA
- 33. WHY WOULD A DB DESIGNER LOVE IT? Allows central, reusable design for standard masking Offers more reliable masking and more usable masking Removes whining about “we can do that later”
- 34. SECURITY – ROW LEVEL SECURITY
- 35. ROW LEVEL SECURITY Filtering result sets (predicate based access) Predicates applied when reading data Can be used to block write access User defined policies tied to inline table functions
- 36. ROW LEVEL SECURITY 36 No indication that results have been filtered If all rows are filtered than NULL set returned For block predicates, an error returned Works even if you are dbo or db_owner role
- 37. WHY WOULD A DB DESIGNER LOVE IT? Allows a designer to do this sort of data protection IN THE DATABASE, not just rely on code. Many, many pieces of code.
- 39. DATA CLASSIFICATION RESULTS AND
- 41. LEDGER DATABASES AND TABLES YOU HAD TO KNOW THAT BLOCKCHAIN WOULD SHOW UP AT SOME POINT.
- 43. AZURE SQL DB LEDGER TABLE – APPEND ONLY CREATE SCHEMA [AccessControl] CREATE TABLE [AccessControl].[KeyCardEvents] ( [EmployeeID] INT NOT NULL, [AccessOperationDescription] NVARCHAR (MAX) NOT NULL, [Timestamp] Datetime2 NOT NULL ) WITH (LEDGER = ON (APPEND_ONLY = ON) );
- 44. LEDGER TABLES IN AZURE SQL DB AND SQL SERVER https://learn.microsoft.com/sql/relational-databases/security/ledger/ledger-overview
- 45. AZURE SQL DB LEDGER TABLE – UPDATABLE CREATE TABLE [Account].[Balance] ( [CustomerID] INT NOT NULL PRIMARY KEY CLUSTERED, [LastName] VARCHAR (50) NOT NULL, [FirstName] VARCHAR (50) NOT NULL, [Balance] DECIMAL (10,2) NOT NULL ) WITH (SYSTEM_VERSIONING = ON, LEDGER = ON);
- 46. KEY FEATURES AZURE LEDGER TABLES Ledger Databases Database Digests Ledger Tables Updatable Append only Immutable storage for transaction recording Ledger Verification
- 47. SAMPLE UPDATABLE LEDGER TABLE https://docs.microsoft.com/en-us/azure/azure-sql/database/ledger-how-to-updatable-ledger-tables
- 48. DEPLOYING A LEDGER TABLE Property of a Table at creation time Set update (versioning) or append only
- 49. DEPLOYING A LEDGER DATABASE Property of a Database at creation time All tables are Ledger Tables
- 50. CHANGING EXISTING TABLE TO A LEDGER TABLE Alter? No. Migrate Create new Ledger Table Copy Data to New Table Clean up previous Table
- 51. SYS.TABLES
- 53. WHY WOULD ONE USE A LEDGER TABLE? More trustworthy More protection from DBA/SysAdmin tampering Don’t need or want full blockchain functionality
- 54. ARC ENABLED SQL SERVER Single point of control for SQL Servers (On-prem, in Azure, or in other clouds) • Dashboards • Best Practices Assessments • AAD authentication • Microsoft Defender • Microsoft Purview • PAYG for SQL Server https://learn.microsoft.com/en-us/sql/sql-server/azure-arc/overview
- 57. SUPPORTED VERSIONS Windows Server 2012 and later versions Ubuntu 20.04 (x64) Red Hat Enterprise Linux (RHEL) 8 (x64) SUSE Linux Enterprise Server (SLES) 15 (x64) SQL Server running in containers. SQL Server Failover Cluster Instances (FCI). SQL Server roles other than the Database Engine, such as SSAS, SSRS, or SSIS) SQL Server editions: Business Intelligence. SQL Server 2008 (10.0.x), SQL Server 2008 R2 (10.50.x), and older versions. SQL Server in Azure Virtual Machines. SQL Server Azure VMware Solution VMs in other Clouds
- 59. VULNERABILITY ASSESSMENTS LET THE ROBOTS DO WHAT THEY ARE GOOD AT DOING
- 62. MICROSOFT DEFENDER FOR SQL DATA IN AZURE
- 63. MICROSOFT DEFENDER FOR SQL Azure SQL Database Azure SQL Managed Instance Dedicated SQL pool in Azure Synapse
- 64. MICROSOFT DEFENDER FOR SQL SERVER MACHINES SQL Server on Virtual Machines On-Prem SQL Server (Arc-enabled SQL Server) On-Prem SQL Server (running on Windows)
- 65. MICROSOFT DEFENDER FOR OPEN-SOURCE RELATIONAL Azure DB for PostgreSQL Azure DB for MySQL Azure DB for MariaDB
- 67. SECURE SCORE HOW DOES MICROSOFT THINK YOU ARE DOING…MAYBE
- 68. SECURE SCORE • Based on: • your progress against Recommendations • Microsoft best practices • Industry best practices
- 69. SECURE SCORE Improve ◦ Remediate ◦ Apply policies ◦ Use blueprints/templates ◦ Ensure new deployments comply
- 71. AZURE MONITOR • Most resources include Monitor-collected data in the Overview page of Azure Portal • Portal option to see Monitor for all services • Can monitor Azure and on-premises resources
- 72. SUMMARY Data quality? Data availability? Data recovery? Query performance? Legal requirements? Which one is right for you?
- 73. Microsoft Defender for Cloud—Databases Protection Protect SQL workloads through security posture management and allow timely responses to threats • SQL security misconfigurations • SQL injection attacks • Brute-force attacks • Unusual data exfiltration • Suspicious access or queries Cloud native security 1-click enablement of protect different type of SQL workloads (IaaS or PaaS) Security posture management Discover, track, and remediate SQL workloads security misconfigurations Advanced threat protection Detect and response unusual and harmful attempts to breach SQL workloads Centralized and integrated Centralize security across all data assets managed by Azure and built-in integration with Sentinel and Purview 1 2 3 https://github.com/microsoft/sqlworkshops-sql2022workshop/blob/main/sql2022workshop/slides/The%20SQL%20Server%202022%20Workshop.pptx
- 75. FINALLY… Performance is important. But it doesn’t override security and privacy.
- 76. ONE MORE TIME… Every Design Decision must be based on Cost, Benefit and Risk
- 78. ONE MORE TIME… Every Design Decision must be based on Cost, Benefit and Risk
- 79. THANK YOU! GO OUT AND BE GREAT…AND DESIGN SECURE DATABASES @DATACHICK DATACHICK@MSTDN.CA