DOWNLOAD THE CODE:
Download the Code 99148.zip

Executive Summary: To get the best performance out of Microsoft SQL Server, you need to avoid these 10 common pitfalls related to AWE memory usage, normalization, clustered indexes, search arguments (SARGs), wasted network bandwidth, and more.

One of the primary things that I do as a consultant is to help companies evaluate the gap between where their current SQL Server deployments are and where they should be in relation to industry best practices. By offering a number of different auditing services and options, I help clients validate code, optimize performance, and evaluate their needs for future growth. And because many of my audits are holistic in nature, I’ve gained great insights over the years into some common developer pitfalls and mistakes that can adversely affect performance. This article focuses on 10 common performance problems (presented in no particular order) that developers can avoid with a modicum of effort.

1. Failure to Properly Configure AWE Memory Usage
Although configuring server memory is obviously an IT or DBA responsibility, I’ve seen too many environments where “poorly performing code” was actually the result of a system with 8GB (or more) of RAM restricting SQL Server to only 2GB. In fact, from what I’ve seen, I’d wager that this problem is so widespread that more than half of production SQL Server deployments suffer from it. Of course, by helping clients remedy this situation, I end up being a hero; adding more RAM is typically one of the easiest ways to boost performance, as long as SQL Server can use the memory.

Happily, you, too, can be a hero by enabling SQL Server to use more than 2GB of RAM. It’s a relatively simple operation—although I wish that the SQL Server installer offered to configure Address Windowing Extensions (AWE) during installation. To start, verify that SQL Server is using AWE, which you can do by executing the code shown in the first part of Listing 1. (Note that some lines in the listings are wrapped to fit on the printed page.) If you have more than 2GB of physical RAM and AWE isn’t enabled, you can use the remaining code in Listing 1 to quickly configure SQL Server to use the additional memory.

The trick to enabling AWE is to ensure that the service account used by SQL Server has been granted the Lock pages in memory user right, as Figure 1 shows. Without this critical configuration change, SQL Server can’t use more than 2GB of RAM, no matter how much physical memory is available. Also note that if you need to change this configuration, you need to restart the SQL Server service before you can realize the benefits. (For more information, see the Microsoft article “How to configure SQL Server to use more than 2 GB of physical memory” at support.microsoft.com/?kbid=274750, which applies to SQL Server 2005 and SQL Server 2000.)

2. Failure to Normalize
Many performance-tuning articles suggest that you intentionally denormalize databases for the sake of performance (to what I jokingly call 2 ¾ normal form). Although internal denormalization is a valid approach that’s covered elsewhere, it’s also a sad fact that databases are occasionally deployed by developers who either don’t know anything about normalization or who fail to grasp its importance.

In these cases, the extra business logic required to extract useful information from jagged Frankencolumns of tangled data adds significant processing overhead. However, this additional overhead usually doesn’t become a performance problem until the database becomes heavily used. Of course, by that time there’s typically no easy or cost-effective way to remedy the situation. Therefore, although it sounds trite, one of the best things that developers can do to ensure good performance is start with a solid, normalized foundation. The cost and headache of doing otherwise can quickly become a true performance nightmare.

3. Failure to Use Clustered Indexes
When you’re looking to squeeze seconds or milliseconds of additional performance out of your code, the prospect of shutting down your production database for a few hours to correct a performance problem probably doesn’t make much sense. But doing so is the only way to remedy common performance problems stemming from the lack of a clustered index.

As with normalization problems, failure to use clustered indexes is a performance issue that typically won’t surface until a table starts to get fairly large (say, a few million rows). But by that time, the table has typically become heavily fragmented (both physically and logically, through the use of forwarding records), and queries involving bookmark operations can become slow enough to draw the attention of end users and management. Sadly, at this point the only available options are to either re-create the table or try to slap a clustered index on it. Both operations can take hours (though recreating the table is typically faster), and end users can’t use the table while the data is being restructured— which in most cases renders your database useless for lengthy periods.

Therefore, it’s a good rule of thumb to make sure that all tables have a clustered index. Likewise, given the fact that a well-placed clustered index can drastically improve ranged queries, it’s also a good practice to ensure that any table with more than 20,000 rows has a well-considered clustered index.

4. Failure to Optimize
If you’re interested in improving the performance of your code, your efforts will only go so far if your database is missing key indexes, has heavy table or index fragmentation, or doesn’t have accurate statistics. Furthermore, since some DBAs are either too swamped to optimize index placement (or consider it a developer responsibility), it behooves developers to ensure that foreign keys and other joined columns are indexed on both sides of the join (i.e., in both tables).

Likewise, for developers whose code is being fingered for performance problems, it’s always good policy to verify that indexes are being regularly rebuilt and defragmented and that statistics are being regularly updated. Doing so can help you avoid wasted effort. Because I find that “failure to DBA” is a common problem in many engagements where I’m called in to help tune “poorly running code,” I’ve provided two scripts (Web Listings 1 and Web Listings 2 at www.sqlmag.com, InstantDoc ID 99148) that you can use to help evaluate statistics and index fragmentation.

5. Failure to Create SARGable Queries
Highly optimized queries against large tables in SQL Server take advantage of index-seek operations instead of more expensive scanning operations involving tables and indexes. (The difference between these operations is analogous to seeking against the alphabetized listing of last names in your phone book to find every “Campbell” versus scanning the phone book for every “Michael”—the difference can be expressed in orders of magnitude.) Accordingly, queries that can take advantage of index-seek operations are commonly referred to as SARGable queries (where SARG is a word play on Search ARGument).

Creating SARGable queries might sound a bit spooky to the uninitiated, but once you learn a few simple syntax rules and ensure that a viable index is in place, creating SARGable queries actually becomes second nature. Happily, you can find a wealth of resources (both online and in print) that cover this topic in depth, but for a quick overview, see Listing 2.

Continue on Page 2

   Prev. page   [1] 2     next page



You must log on before posting a comment.

If you don't have a username & password, please register now.

Reader Comments

As a long-time SQL Server consultant, I like this article because it describes the same things I see every day. However, I have to take exception to some of the things said about AWE memory.

In that discussion there seems to be some confusion between physical memory and virtual address space. As I understand it and as the URLs below will confirm, AWE is irrelevant unless a machine has more that 4GB of RAM, not 2GB as implied here. I have seen the web page to which this article refers and it puzzles me. It contradicts everything else that Microsoft and others have published about AWE from day 1.

The default size of the user-mode portion of the 4GB virtual address space is 2GB and that may be the source of the confusion. The kernel-mode processes occupy the other 2GB. On a machine with 4GB of RAM you can expand the user-made address space to 3GB of the total 4GB (VAS, not RAM) with the /3GB switch in boot.ini, but that has nothing to do with AWE.

I know that there is a lot of disinformation, partial information, and outright wrong information on this topic. Most of it springs from a confusion between physical memory and virtual address space. However, I may have absorbed some of that disinformation too, but I have configured AWE on many,many systems and it worked fine. I have to admit I never tried enabling AWE on a system with less than 4GB because there seemed to be no point since any 32bit process can access 4GB of RAM without AWE. /#GB makes sense on a 4GB system, but not AWE.

The article also does not mention that AWE is only for the 32bit OS and does not mention the OS set up that needs to be done before any of this will work.

here are two of the hundreds of urls that will also affirm that AWE is only necessary in 32bit systems with more than 4GB of RAM. http://technet.microsoft.com/en-us/library/ms175581.aspx http://technet.microsoft.com/en-us/library/ms179301.aspx

Regards,

Kurt Survance

ksurvance

Article Rating 3 out of 5

Hi Michael, I'm not sure if your article specifically recommend to allocate to create 0.25 to 1 data file per filegroup per processor (point 9). If yes, I dont think i would agree with it as I feel this recommendation is only contained to tempdb.

PSS has specifically mention in their article stating this recommendation is not for user database as it does not usually have DDL contention, unless for whatever reason you're creating/dropping too many temp tables on your user database.

http://blogs.msdn.com/psssql/archive/2007/02/21/sql-server-urban-legends-discussed.aspx

Most people have this misconception of a worker allocated per data file in sql. Actually, this one worker per data file is i.e. when you create database different drive, hence, workers do the creation so the database may be created faster due to parallel work streams.

Please comment and reply to me on your thoughts.

Regards, Garry

garry.mortimer@btfinancialgroup.com

Article Rating 2 out of 5

Kurt,

Check out this article: http://technet.microsoft.com/en-us/library/ms190673.aspx

Where there are a couple of cases where MS points out that AWE can, indeed, address physical memory below the 4GB limit (and above the 2GB non-AWE window).

Here's a quote: Since AWE-mapped memory is supported below 3 GB, you can define the min server memory and max server memory values within the physical memory range, or use the default values for both options.

This article also points out that you can configure AWE on systems where the OS LIMITS you to only 4GB - which infers that you could (and would want to) set AWE on systems with as 'little' as 4GB of physical RAM.

That said, I agree - this is one of the most confusing aspects of SQL Server management and configuration. It's also worth pointing out that in the article I was really only talking about cases where I come in to an environment where there's some beefy hardware and can spot that AWE isn't configured correctly because task manager is only showing just under 2GB of RAM being used. In other words, I wasn't talking about setting AWE in those 2-4GB ranges either.

And um, cough, since I have to rate the article to submit comments... I'll um... rate it as a 5 /5 *grin*

overachiever

Article Rating 5 out of 5

Gary,

You are correct - that .25/processor ratio is specifically for the tempDB, otherwise the 1/processor applies to user DBs. Sadly, given the tiny bit of space I had to work with, I just had to say .25-1/processor to be 'all inclusive'. Interestingly enough, I THOUGHT I had a link to that EXACT PSS post in my article.

That said... I've also written another article focusing solely on improving IO performance that should be getting published... .sometime soon - where I deal with this just a bit more.

--Mike www.sqlservervideos.com

overachiever

Article Rating 5 out of 5