The Data Access Handbook

The Data Access Handbook Achieving Optimal Database Application Performance and Scalability John Goodson and Robert A. Steward PRENTICE HALL Upper Saddle River, NJ Boston Indianapolis San Francisco New York Toronto Montreal London Munich Paris Madrid Cape Town Sydney Tokyo Singapore Mexico City

Contents Preface xv Acknowledgments xix About the Authors xxi Д Д Д PERFORMANCE ISN'T WHAT IT USED TO BE 1 Where Are We Today? 4 The Network 5 The Database Driver 5 The Environment 6 Your Database Application 7 Our Goal for This Book 8 Я1ЯН1М DESIGNING FOR PERFORMANCE: WHAT'S YOUR STRATEGY? 9 Your Applications 10 Database Connections 10 Transaction Management 21 SQL Statements 27 Data Retrieval 30 Extended Security 37 Static SQL Versus Dynamic SQL 43 The Network 44 The Database Driver 46 Know Your Database System 47 Using Object-Relational Mapping Tools 48 Summary 49 vii

viii Contents ИНДИИ DATABASE MIDDLEWARE: WHY ITS IMPORTANT 51 What Is Database Middleware? 52 How Database Middleware Affects Application Performance 52 Database Drivers 53 What Does a Database Driver Do? 54 Database Driver Architecture 55 Runtime Performance Tuning Options 62 Configuring Database Drivers/Data Providers 63 Summary 74 ННННИ THE ENVIRONMENT: TUNING FOR PERFORMANCE 75 Runtime Environment (Java and.net) 77 JVM 77.NET CLR 82 Operating System 83 Network 86 Database Protocol Packets 86 Network Packets 89 Configuring Packet Size 92 Analyzing the Network Path 94 Reducing Network Hops and Contention 96 Avoiding Network Packet Fragmentation 98 Increasing Network Bandwidth 106 Hardware 106 Memory 107 Disk 110 CPU (Processor) 112 Network Adapter 116 Virtualization 119 Summary 121

Contents ix ВИВИИЯ ODBC APPLICATIONS: WRITING GOOD CODE 123 Managing Connections 124 Connecting Efficiently 124 Using Connection Pooling 124 Establishing Connections One at a Time 125 Using One Connection for Multiple Statements 125 Obtaining Database and Driver Information Efficiently 126 Managing Transactions 127 Managing Commits in Transactions 127 Choosing the Right Transaction Model 135 Executing SQL Statements 136 Using Stored Procedures 136 Using Statements Versus Prepared Statements 138 Using Arrays of Parameters 139 Using the Cursor Library 141 Retrieving Data 142 Retrieving Long Data 142 Limiting the Amount of Data Retrieved 144 Using Bound Columns 145 Using SQLExtendedFetch Instead o/sqlfetch 147 Determining the Number of Rows in a Result Set 148 Choosing the Right Data Type 149 Updating Data 149 Using SQLSpeci al Col umns to Optimize Updates and Deletes 149 Using Catalog Functions 151 Minimizing the Use of Catalog Functions 151 Avoiding Search Patterns 152 Using a Dummy Query to Determine Table Characteristics 153 Summary 155 И Н Д И И JDBC APPLICATIONS: WRITING GOOD CODE 157 Managing Connections 158 Connecting Efficiently 158 Using Connection Pooling 158

X Contents Establishing Connections One at a Time 159 Using One Connection for Multiple Statements 159 Disconnecting Efficiently 160 Obtaining Database and Driver Information Efficiently 162 Managing Transactions 163 Managing Commits in Transactions 163 Choosing the Right Transaction Model 169 Executing SQL Statements 170 Using Stored Procedures 170 Using Statements Versus Prepared Statements 172 Using Batches Versus Prepared Statements 173 Using getxxx Methods to Fetch Data from a Result Set 175 Retrieving Auto-Generated Keys 176 Retrieving Data 177 Retrieving Long Data 177 Limiting the Amount of Data Retrieved 179 Determining the Number of Rows in a Result Set 181 Choosing the Right Data Type 182 Choosing the Right Cursor 182 Updating Data 186 Using Positioned Updates, Inserts, and Deletes fupdatexxx Methods) 186 UsinggetBestRowIdentifierC) to Optimize Updates and Deletes 186 Using Database Metadata Methods 188 Minimizing the Use of Database Metadata Methods 188 Avoiding Search Patterns 188 Using a Dummy Query to Determine Table Characteristics 189 Summary 191 КЯ1ВН1Я.NET APPLICATIONS: WRITING GOOD CODE 193 Managing Connections 194 Connecting Efficiently 194 Using Connection Pooling 194 Establishing Connections One at a Time 195 Disconnecting Efficiently 196 Obtaining Database and Data Provider Information Efficiently 198

Contents xi Managing Transactions 198 Managing Commits in Transactions 198 Choosing the Right Transaction Model 205 Executing SQL Statements 206 Executing SQL Statements that Retrieve Little or No Data 206 Using the Command. Prepare Method 208 Using Arrays of Parameters/Batches Versus Prepared Statements 209 Using Bulk Load 212 Using Pure Managed Providers 212 Selecting.NET Objects and Methods 213 Avoiding the CommandBui 1 der Object 213 Choosing Between a DataReader and DataSet Object 214 Using GetXXX Methods to Fetch Data from a DataReade г 215 Retrieving Data 216 Retrieving Long Data 216 Limiting the Amount of Data Retrieved 218 Choosing the Right Data Type 219 Updating Data 220 Summary 221 1Я8ЯИ1И CONNECTION POOLING AND STATEMENT POOLING 223 Connection Pool Model for JDBC 223 Configuring Connection Pools 225 Guidelines 226 Connection Pool Model for ODBC 228 Connection Pooling as Defined in the ODBC Specification 228 Configuring Connection Pools 229 Guidelines 230 Connection Pool Model for ADO.NET 230 Configuring Connection Pools 231 Guidelines 231 Using Reauthentication with Connection Pooling 232 Configuring Connection Pooling with Reauthentication in a JDBC Environment 235

xii Contents Using Statement Pooling 236 Using Statement Pooling with Connection Pooling 238 Guidelines 239 Summary: The Big Picture 240 Я1ИН1И DEVELOPING GOOD BENCHMARKS 243 Developing the Benchmark 244 Define Benchmark Goals 244 Reproduce the Production Environment 246 Isolate the Test Environment 251 Reproduce the Workload 252 Measure the Right Tasks 252 Measure over a Sufficient Duration of Time 254 Prepare the Database 257 Make Changes One at a Time 257 Assess Other Factors 258 Benchmark Example 258 Summary 264 И8ИНШ81 TROUBLESHOOTING PERFORMANCE ISSUES 265 Where to Start 266 Changes in Your Database Application Deployment 268 The Database Application 269 The Database Driver 270 Runtime Performance Tuning Options 271 Architecture 271 The Environment 272 Runtime Environment (Java and.net) 273 Operating System 273 Network 273 Hardware 274 Case Studies 277 Case Study 1 111 Case Study 2 281

Contents xiii Case Study 3 282 Case Study 4 285 Case Study 5 287 Case Study 6 291 Case Study 7 293 Case Study 8 295 Summary 297 ВЯ1ВЯ1Щ DATA ACCESS IN SERVICE-ORIENTED ARCHITECTURE (SOA) ENVIRONMENTS 299 What Is Service-Oriented Architecture (SOA)? 300 Data Access Guidelines for SOA Environments 302 Involve Data Experts in Addition to SOA Experts 302 Decouple Data Access from Business Logic 303 Design and Tune for Performance 305 Consider Data Integration 305 Summary 307 GLOSSARY 309 INDEX 319