Programming Microsoft SQL Server 2005

Transcription

1 Programming Microsoft SQL Server 2005 Andrew J. Brust, Stephen Forte To learn more about this book, visit Microsoft Learning at Publication Date: December 2004

2 Table of Contents Acknowledgements xix Introduction xxiii Who This Book Is For xxiv How This Books is Organized xxv System Requirements xxvi Using the Samples xxvii Support for This Book xxix Questions and Comments xxix Part I Design Fundamentals and Core Technologies 1 Overview A Tough Act to Follow The Software Industry and Disruptive Change Industry Trends, SQL Server Features, and a Book to Show You the Way Programming the Server Application Code and SQL Server: Extending Your Database s Reach It s the Strategy, Stupid A Collaborative Effort for, and by, Developers Exploring the T-SQL Enhancements in SQL Server Introducing SQL Server Management Studio Common Table Expressions Recursive Queries with CTEs The PIVOT and UNPIVOT Operators Using UNPIVOT Dynamically Pivoting Columns The APPLY Operator TOP Enhancements Ranking Functions ROW_NUMBER() RANK() DENSE_RANK() and NTILE(n) Using All the Ranking Functions Together Ranking over Groups: PARTITION BY What do you think of this book? We want to hear from you! Microsoft is interested in hearing your feedback about this publication so we can continually improve our books and learning resources for you. To participate in a brief online survey, please visit: vii

3 viii Table of Contents Exception Handling in Transactions New Data Types varchar(max) Data Type xml Data Type The WAITFOR Command DDL Triggers and Notifications SNAPSHOT Isolation Statement-Level Recompile Summary An Overview of SQL CLR Getting Started: Enabling CLR Integration Visual Studio/SQL Server Integration SQL Server Projects in Visual Studio Automated Deployment SQL CLR Code Attributes Your First SQL CLR Stored Procedure CLR Stored Procedures and Server-Side Data Access Piping Data with SqlDataRecord and SqlMetaData Deployment Deploying Your Assembly Deploying Your Stored Procedures Testing Your Stored Procedures CLR Functions CLR Triggers CLR Aggregates CLR Types Security Examining and Managing CLR Types in a Database Best Practices for SQL CLR Usage Summary XML and the Relational Database XML in SQL Server The XML Data Type Working with the XML Data Type as a Variable Working with XML in Tables XML Schemas XML Indexes FOR XML Commands FOR XML RAW FOR XML AUTO FOR XML EXPLICIT

4 Table of Contents ix OPENXML Enhancements in SQL Server XML Bulk Load Querying XML Data Using XQuery XQuery Defined SQL Server 2005 XQuery in Action XML DML Converting a Column to XML Summary Introducing SQL Server Management Studio The New Management Studio Interface Overview of New Features Window Types Positioning a Docking Window Window Customization Options Connecting to a Database Server Using Object Explorer Object Explorer Filters Management Studio Solutions, Projects, and Files Code and Text Editor Track Changes Indicator Bookmarks Creating Objects Creating Tables Creating Table-Related Objects Creating Indexes Setting Properties for New Users Generating Scripts from Objects Creating Queries Executing Queries Using Templates Maintenance Features Using a Maintenance Plan Performance Tools SQL Server Profiler Database Engine Tuning Advisor Summary Using SQL Server Management Objects (SMO) What Is SMO? What About SQL-DMO? New Features in SMO Working with SMO in Visual Studio

5 x Table of Contents Iterating Through Available Servers Retrieving Server Settings Creating Backup-and-Restore Applications Performing Programmatic DBCC Commands with SMO Summary SQL Server 2005 Security Four Themes of the Security Framework Secure by Design Secure by Default Secure by Deployment Communications SQL Server 2005 Security Overview SQL Server Logins Database Users The Guest User Account Authentication and Authorization How Clients Establish a Connection Password Policies User-Schema Separation Execution Context Encryption Support in SQL Server Encrypting Data on the Move Encrypting Data at Rest Protecting SQL Server Reducing the Surface Area for Attack How Hackers Attack SQL Server Direct Connection to the Internet Weak SA Passwords SQL Server Browser Service SQL Injection Intelligent Observation Summary Part II Applications Development and Reach Technologies 8 ADO.NET 2.0, Typed DataSet Objects, and.net Data Binding A Brief History of Data Access Object Models DAO: A Golden Oldie RDO: A Thin API Wrapper Enter OLE DB and ADO Classic ADO +.NET = ADO.NET What s New in ADO.NET 2.0?

6 Table of Contents xi New Typed DataSet Members Other Enhancements Typed DataSet Enhancements DataTable Objects in the Typed DataSet Designer TableAdapter Objects Connection String Management Using the TableAdapter Configuration Wizard More on Queries and Parameters Adding Query Objects DBDirect Methods and Connected Use of Typed DataSet Objects Standalone DataTable Objects Pure ADO.NET: Working in Code Querying Keeping Data Up to Date Responsive UIs: Executing Queries Asynchronously Not for Servers Only: Client-Side Bulk Copy It s Not Just Text: Processing XML Columns with ADO.NET and the System.Xml Namespace Nothing but.net: Consuming SQL CLR UDT Data as Native.NET Objects Back to the Drawing Board Embedding SQL CLR Objects in Typed DataSet Objects Adding a CLR Stored Procedure to a Typed DataSet TVFs: Easy Living Aggregates and UDTs Windows Forms Data Binding DataGridView Binding Details View Binding Smart Defaults Binding to Stored Procedures and Views SQL CLR Binding Master-Detail and Lookup Binding Parameterized Query Data Binding Data Binding on the Web Typed DataSet Objects and the Web.config File The Data Source Configuration Wizard, the ObjectDataSource Control, and the New Data-Bound Controls Summary Debugging About the Sample Code Ad Hoc Debugging Creating Database Connections

7 xii Table of Contents T-SQL Step Into Debugging Application Debugging Entering Debugging Mode Debugging SQL CLR Code Breakpoints and Context Switching Mixing SQL CLR and T-SQL Code Test Script Debugging Debugging Queries External to Visual Studio Remote Debugging Server Configuration Server Firewall Considerations Back to the Client Client Firewall Configuration Attaching to a Remote Process Summary SQL Server 2005 Native XML Web Services Understanding Native XML Web Services Comparing Native XML Web Services and SQLXML Exposing SQL Programmability as Web Services Stored Procedures and User-Defined Functions SQL Batch Reserving URLs with Http.sys Creating and Managing Endpoints Granting Endpoint Permissions Calling Native XML Web Service Endpoints from Client Applications Example Native XML Web Services Project Creating the SQL Server Functionality Registering the URL with Http.sys Exposing the Endpoints Granting Security Access to the Endpoints Creating the Client Application Best Practices for Using Native XML Web Services Advantages of Native XML Web Services Limitations of Native XML Web Services Security Recommendations Performance Recommendations When to Avoid Native XML Web Services When to Use Native XML Web Services Summary

8 Table of Contents xiii 11 Transactions What Is a Transaction? Understanding the ACID Properties Local Transactions Support in SQL Server Autocommit Transaction Mode Explicit Transaction Mode Implicit Transaction Mode Batch-Scoped Transaction Mode Using Local Transactions in ADO.NET Transaction Terminology Isolation Levels Isolation Levels in SQL Server Isolation Levels in ADO.NET Distributed Transactions Distributed Transaction Terminology Rules and Methods of Enlistment Distributed Transactions in SQL Server Distributed Transactions in the.net Framework Using a Resource Manager in a Successful Transaction Transactions in SQL CLR (CLR Integration) Putting It All Together Summary SQL Server Service Broker: The New Middleware What Is Middleware? What Is SQL Server Service Broker? Comparing Service Broker and MSMQ What Is a SQL Server Service Broker Application? Service Broker Architecture Integrated Management and Operation Routing and Load Balancing Service Broker Programming in T-SQL A Word About Programming Languages Enabling Service Broker Defining Service Broker Objects The Sending Service Program The Receiving Service Program Running the Application A More Robust, Real-World Application Service Broker and Query Notification Service Broker s Place in the Middleware World Summary

9 xiv Table of Contents 13 Using SQL Server 2005 Notification Services What Is a Notification Application? Notification Services Components Notification Services Deployment Strategies Working with Notification Services Creating Notification Applications A Sample Notification Application Flight Price Notification Sample Application Summary Developing Desktop Applications with SQL Server Express Edition What Is SQL Server Express Edition? Licensing Feature Review SQL Server 2005 Express Edition with Advanced Services Configuration Working with SQL Server Express Edition SQLCMD Command-Line Tool User Instances SSEUTIL Installing SQL Server Express Edition Using the Setup Wizard to Manually Install Express Edition Installing via Command-Line Parameters or a Configuration File Deploying Express Edition Applications Using a Wrapper Deploying Express Edition Applications Using ClickOnce Updating ClickOnce Deployments That Use Express Edition Summary Developing Applications with SQL Server 2005 Everywhere Edition and SQL Server Merge Replication SQL Everywhere Integration with SQL Server Working with SQL Everywhere Databases in Management Studio Working with SQL Everywhere Data in Management Studio Creating a SQL Everywhere Application with SQL Server Replication and Visual Studio Creating a Publication Installing and Configuring SQL Everywhere Server Components for IIS Creating a Subscription Using Management Studio Creating a Mobile Application Using Visual Studio Summary

10 Table of Contents xv Part III Reporting and Business Intelligence 16 Using SQL Server 2005 Integration Services History of Data Transfer in SQL Server DTS Packages Working with Integration Services Packages Control Flow Data Flow Using Integration Services Packages Creating Packages Using the Import And Export Wizard Creating Packages Using BI Development Studio Managing Packages Using Management Studio Using the Command Line to Execute and Manage Packages Scheduling Packages Using SQL Server Agent Configuring and Deploying Packages Overview of Programming Package Extension Security Dealing with Sensitive Information and Assets Considerations for Working on a Single Development Machine Considerations for Workgroups Programming Integration Services Programming in Visual Studio Loading and Executing Packages in Applications Creating Packages Programmatically Extensibility Script Tasks Custom Tasks Custom Components Script Components Custom Connection Managers Log Providers Foreach Enumerator Summary Basic OLAP Wherefore BI? OLAP OLAP Vocabulary Dimensions, Axes, Stars, and Snowflakes Building Your First Cube Preparing Star Schema Objects A Tool by Any Other Name

11 xvi Table of Contents Creating the Project Adding a Data Source View Creating a Cube with the Cube Wizard Using the Cube Designer Using the Dimension Wizard Using the Dimension Designer Working with the Properties Window and Solution Explorer Processing the Cube Running Queries Summary Advanced OLAP What We ll Cover in This Chapter MDX in Context And Now a Word from Our Sponsor Advanced Dimensions and Measures Keys and Names Changing the All Member Adding a Named Query to a Data Source View Parent-Child Dimensions Member Grouping Server Time Dimensions Fact Dimensions Role-Playing Dimensions Advanced Measures Calculations Calculated Members Named Sets More on Script View Key Performance Indicators KPI Visualization: Status and Trend A Concrete KPI Testing KPIs in Browser View KPI Queries in Management Studio Other BI Tricks in Management Studio Actions Actions Simply Defined Designing Actions Testing Actions Partitions, Aggregation Design, Storage Settings, and Proactive Caching Editing and Creating Partitions Aggregation Design Partition Storage Options

12 Table of Contents xvii Proactive Caching Additional Features and Tips Perspectives Translations Roles Summary OLAP Application Development Using Excel Working Within Excel Using PivotTables and Charts in Applications and Web Pages Beyond OWC: Full-On OLAP Development MDX and Analysis Services APIs Moving to MDX Management Studio as an MDX Client OLAP Development with ADO MD.NET XMLA at Your (Analysis) Service Analysis Services CLR Support: Server-Side ADO MD.NET Summary Extending Your Database System with Data Mining Why Mine Your Data? Getting Started Preparing Your Source Data Creating Training and Test Samples Adding an SSAS Project Using the Data Mining Wizard and Data Mining Designer Creating a Mining Structure Creating a Mining Model Editing and Adding Mining Models Deploying and Processing Data Mining Objects Viewing Mining Models Validating and Comparing Mining Models Nested Tables Using Data Mining Extensions Data Mining Modeling Using DMX Data Mining Predictions Using DMX DMX Templates Data Mining Applied Data Mining and API Programming Using the WinForms Model Content Browser Controls Executing Prediction Queries with ADO MD.NET

13 xviii Table of Contents Model Content Queries ADO MD.NET and ASP.NET Using the Data Mining Web Controls Developing Managed Stored Procedures XMLA and Data Mining Data Mining and Reporting Services Summary Reporting Services Report Definition and Design Data Sources Report Layouts Report Designer Report Builder Report Definition Language Report Management Publishing Report Manager SQL Server Management Studio Command-Line Utilities Programming: Management Web Services Report Access and Delivery Delivery on Demand Subscriptions Presentation Formats Programming: Rendering Report Server Architecture Deployment Modes Extensibility Report Integration Summary Index What do you think of this book? We want to hear from you! Microsoft is interested in hearing your feedback about this publication so we can continually improve our books and learning resources for you. To participate in a brief online survey, please visit:

14 Chapter 3 An Overview of SQL CLR Andrew Brust In this chapter: Getting Started: Enabling CLR Integration Visual Studio/SQL Server Integration Your First SQL CLR Stored Procedure CLR Stored Procedures and Server-Side Data Access Deployment CLR Functions CLR Triggers CLR Aggregates CLR Types Security Examining and Managing CLR Types in a Database Best Practices for SQL CLR Usage Summary The banner headline for Microsoft SQL Server 2005 is its integration of the Microsoft.NET common language runtime (CLR). This architectural enhancement means that SQL Server can use certain.net classes as basic data types and can accommodate the use of.net languages for the creation of stored procedures, triggers, and functions, and even user-defined aggregates. Note Throughout this chapter, we will refer to the CLR integration in SQL Server as SQL CLR features, functionality, or integration, and we will refer to stored procedures, triggers, functions, aggregates, and user-defined types as the five basic SQL CLR entities. Let s face facts: Transact SQL (T-SQL) is essentially a hack. Back when SQL Server was first developed, Microsoft and Sybase took SQL a declarative, set-based language and added variable declaration, conditional branching, looping, and parameterized subroutine logic to make it into a quasi-procedural language. Although extremely clever and useful, T-SQL lacked, and still lacks, many of the niceties of a full-fledged procedural programming language. 45

15 46 Part I: Design Fundamentals and Core Technologies This shortcoming has forced some people to write T-SQL stored procedures that are overly complex and difficult to read. It has forced others to put logic in their middle-tier code that they would prefer to implement on the database. And it s even forced some people to abandon stored procedures altogether and use dynamic SQL in their applications, a practice that we do not endorse. Because of these workarounds to address T-SQL s procedural limitations, CLR integration is a welcome new feature in SQL Server, and it has caught the market s attention. Meanwhile, T-SQL and, one might argue, SQL itself is vastly superior to procedural languages for querying and manipulating data. Its set-based syntax and implementation simply transcend the approach of procedurally iterating through rows of data. This is no less true in SQL Server 2005 than in previous versions of the product, and T-SQL has been greatly enhanced in this release (as detailed in Chapter 2), making it more valuable still. So this places database application developers at a crossroads. We must simultaneously learn how the SQL CLR features work and develop a sophisticated, judicious sense of when to use T-SQL instead of the SQL CLR feature set. We must resist the temptation to completely.netify our databases but learn to take advantage of the SQL CLR feature set where and when prudent. This chapter aims to help you learn to use SQL CLR features and to develop an instinct for their appropriate application. In this chapter, you will learn: How to enable (or disable) SQL CLR integration on your SQL Server How SQL Server accommodates CLR code, through the loading of.net assemblies How to use SQL Server 2005 and Visual Studio 2005 together to write SQL CLR code and deploy it, simply and quickly How to deploy SQL CLR code independently of Visual Studio, using T-SQL commands, with or without the help of SQL Server Management Studio How to create simple CLR stored procedures, triggers, functions, aggregates, and userdefined types, use them in your databases, and utilize them from T-SQL How both the standard SQL Server client provider and the new server-side library can be combined to implement SQL CLR functionality How SQL CLR security works and how to configure security permissions for your assemblies When to use SQL CLR functionality, and when to opt to use T-SQL instead Getting Started: Enabling CLR Integration Before you can learn how to use SQL CLR features, you need to know how to enable them. As with many new products in the Microsoft Windows Server System family, many advanced features of SQL Server 2005 are disabled by default. The reasoning behind this is sound: Each

16 Chapter 3: An Overview of SQL CLR 47 additional feature that is enabled provides extra surface area for attacks on security or integrity of the product, and the added exposure is inexcusable if the feature goes unused. The SQL CLR features of SQL Server 2005 are sophisticated and can be very useful, but they are also, technically, nonessential. It is possible to build high-performance databases and server-side programming logic without SQL CLR integration, so it is turned off by default. Don t be discouraged, though: Turning on the feature is easy. Microsoft provides both a user-friendly GUI tool (aptly named the SQL Server Surface Area Configuration tool) and a system stored procedure for enabling or disabling SQL CLR integration. We ll cover both approaches. To use the Surface Area Configuration tool, simply start it from the Configuration Tools subgroup in the Microsoft SQL Server 2005 programs group on the Windows start menu. Figure 3-1 shows the tool as it appears upon startup. Figure 3-1 The SQL Server 2005 Surface Area Configuration tool To configure CLR integration, click the Surface Area Configuration For Features link at the bottom of the form. After a short pause, the Surface Area Configuration For Features dialog box appears; a tree view-style list of features appears on the left, and the Ad Hoc Remote Queries feature is preselected. Click the CLR Integration node immediately below it, and you will see an Enable CLR Integration check box on the right of the form. (This is shown in Figure 3-2.) To enable SQL CLR features, make sure that the check box is checked, and click OK to close the Surface Area Configuration For Features window. (You can also clear the check box to disable SQL CLR integration.) Close the Surface Area Configuration tool by clicking its close box in the upper-right corner of the window.

17 48 Part I: Design Fundamentals and Core Technologies Figure 3-2 The Surface Area Configuration For Features window If you d prefer a command-line method for enabling or disabling SQL CLR functionality, open up SQL Server Management Studio and connect to the server you d like to configure. Then, from a query window, type the following commands, and click the Execute button on the Management Studio SQL Editor toolbar. sp_configure 'clr enabled', 1 GO RECONFIGURE GO That s all there is to it! To disable SQL CLR integration, just use a value of 0, instead of 1, as the second parameter value in the sp_configure call. Tip Don t forget that this will work from any tool that can connect to SQL Server, not just Management Studio. In fact, you could issue the previous command text from your own code using the ADO.NET SqlCommand object s ExecuteNonQuery method as long as your code can connect to your server and your sever can authenticate as a user in the sysadmin server role. With SQL CLR integration enabled, you re ready to get started writing SQL CLR code. Before we dive in, we need to discuss Visual Studio/SQL Server integration and when to use it. Visual Studio/SQL Server Integration Visual Studio 2005 and SQL Server 2005 integrate tightly in a number of ways. It s important to realize, however, that the use of Visual Studio integration is completely optional and the use of T-SQL is a sufficient substitute. T-SQL has been enhanced with new DDL commands for

18 Chapter 3: An Overview of SQL CLR 49 maintaining CLR assemblies, types, and aggregates, and its existing commands for stored procedures, triggers, and functions have been enhanced to recognize code within deployed assemblies. Visual Studio can execute those commands on your behalf. It can also make writing individual SQL CLR classes and functions easier. Ultimately, we think all developers should be aware of both Visual Studio assisted and more manual coding and deployment methods. You might decide to use one method most of the time, but in some situations you ll probably need the other, so we want to prepare you. As we cover each major area of SQL CLR programming, we will discuss deployment from both points of view. We ll cover some general points about Visual Studio integration now, and then we ll move on to cover SQL CLR development. SQL Server Projects in Visual Studio The combination of Visual Studio 2005 and SQL Server 2005 on the same development machine provides a special SQL Server Project type in Visual Studio and, within projects of that type, defined templates for the five basic SQL CLR entities. These templates inject specific code attributes and function stubs that allow you to create SQL CLR code easily. The attributes are used by Visual Studio to deploy your assembly and its stored procedures, triggers, and so on to your database. Some of them are also used by SQL Server to acknowledge and properly use your functions, user-defined types (UDTs), and aggregates. To test out the new project type and templates, start Visual Studio 2005 and create a new project by using the File/New/Project main menu option, the New Project toolbar button, the Ctrl+Shift+N keyboard accelerator, or the Create Project hyperlink on the Visual Studio Start Page. In the New Project dialog box (Figure 3-3), select Database from the Project types tree view on the left (the Database node appears under the parent node for your programming language of choice; in Figure 3-3, the language is C#), and click the SQL Server Project icon in the Templates list on the right. Enter your own project name if you d like, and click OK. Figure 3-3 The Visual Studio 2005 New Project dialog box with the SQL Server project type selected

19 50 Part I: Design Fundamentals and Core Technologies Next, the Add Database Reference dialog box appears (Figure 3-4). Figure 3-4 The Add Database Reference dialog box Because Visual Studio provides automated deployment of your SQL CLR code, it must associate your project with a specific server and database via a database reference (connection). Any database connections that have already been defined in the Server Explorer window appear in this window, as does an Add New Reference button that allows you to define a new connection, if necessary. Pick an existing connection or define a new one, and then click OK. The project opens. Note If no data connections have already been defined in the Server Explorer window, the New Database Reference dialog box will appear in place of the Add Database Reference dialog box. In the New Database Reference dialog box, you may specify server, login, and database details for a new database connection that will be used by your project as its database reference and added to the Server Explorer as a new data connection. You can easily add preconfigured classes for the five basic SQL CLR entities to your project. You can do this in a number of ways: directly from the Project menu or from the Add submenu on the Server Explorer s project node shortcut menu (Figure 3-5). You can also add the preconfigured classes from the Add New Item dialog box (Figure 3-6), which is available from the Project/Add New Item option on the main menu, or the Add/ New Item option on the Solution Explorer project node s shortcut menu.

20 Chapter 3: An Overview of SQL CLR 51 Figure 3-5 The Server Explorer project node shortcut menu and its Add submenu Figure 3-6 The Visual Studio SQL Server project Add New Item dialog box Automated Deployment Once opened, SQL Server projects add a Deploy option to the Visual Studio Build menu. In addition, the Play (Start Debugging) button and the Start Debugging, Start Without Debugging, and Step Over options on the Debug menu (and their keyboard shortcuts F5, Ctrl+F5, and F10, respectively) all deploy the project assembly in addition to performing their listed function.

21 52 Part I: Design Fundamentals and Core Technologies Visual Studio can do a lot of deployment work for you. But as you ll learn, you can do so on your own and, in certain circumstances, have more precise control over the deployment process when you do so. SQL CLR Code Attributes A number of.net code attributes are provided for SQL CLR developers; these are contained in the Microsoft.SqlServer.Server namespace. Many of them are inserted in your code when you use the various templates in the SQL Server project type, as is a using statement for the Microsoft.SqlServer.Server namespace itself. If you choose to develop code without these templates, you must add the appropriate attributes, and optionally the using statement, yourself. Although all these attributes are provided in the same namespace, some are used exclusively by Visual Studio and others are used by both Visual Studio and SQL Server. Covering all SQL CLR attributes and their parameters would itself require an entire chapter, so our coverage will be intentionally selective. Specifically, we will provide coverage of the SqlProcedure, SqlFunction, SqlTrigger, SqlUserDefinedAggregate, and SqlUserDefinedType attributes. We will not cover the SqlFacet and SqlMethod attributes. Just as certain attributes are not covered here, we cover only some of the parameters accepted by the attributes that we do cover. And in some cases, we cover only certain of the possible values that can be passed to these attributes. For example, SqlFunction accepts several parameters but the only ones we will cover are Name, FillRowMethodName, and TableDefinition. For SqlUserDefinedAggregate and SqlUserDefinedType, we will cover only a single value setting for the Format parameter, and will not cover the several other parameters those two attributes accept. The coverage we provide will be more than sufficient for you to implement basic, intermediate, and certain advanced functionality with all the basic five SQL CLR entities. The attributes and parameters that we won t cover are useful mostly for optimizing your SQL CLR code, and they are well documented in SQL Server Books Online and articles on MSDN. About the Sample Code The sample.net code for this chapter is provided in two versions. The primary material is supplied as a Visual Studio SQL Server project, accessible by opening the solution file Chapter03.sln in the Chapter03 subfolder of this chapter s VS sample code folder. We also supply the code as a standard Class Library project, accessible by opening the solution file Chapter03Manual.sln in the Chapter03Manual subfolder. The code in each project is virtually identical, although the Class Library project does not autodeploy when the various Build and Debug options are invoked in Visual Studio As we cover each SQL CLR feature, we ll discuss how automated deployment takes place from the SQL Server project and how command-driven deployment should be performed for the Class Library project.