SideBar    Sqlcmd
DOWNLOAD THE CODE:
Download the Code 95385.zip

Suppose that you hire a new DBA and assign him or her to a large-scale SQL Server infrastructure that consists of more than 100 servers divided not only by purpose and criticality but also by geography. This new DBA would likely spend the entire first week getting to know the lay of the land—using SQL Server Management Studio (SSMS) to connect to each SQL Server machine, one by one, to gather essential information such as version, edition, server configuration, existing databases, and scheduled backup jobs. A daunting task indeed.

I learned early in my career that spending time up front to automate otherwise manual and time-consuming tasks can preserve your sanity. I therefore developed a fairly simple solution that connects to each available SQL Server machine, pulls information into a central repository database, and feeds the combined data to a report for DBAs and other IT staff to use. In this article I describe the solution I used. (Note that the solution I offer is intended to enhance, not replace, a preexisting full monitoring and alerting solution.)

Tool Time
Several tools are available to the intrepid DBA setting out to discover the uncharted server landscape. For example, you can use the Microsoft SQL Server Health and History Tool (SQLH2; available at http://www.microsoft.com/downloads/details.aspx?familyid=eedd10d6-75f7-4763-86de-d2347b8b5f89&di splaylang=en) to populate a repository database. However, this tool is outdated and not very flexible. (For more information about SQLH2, see "SQL Health and History Tool," November 2006, InstantDoc ID 93544.)

For my project, I employed the command-line tool Sqlcmd, SQL Server Integration Services (SSIS), a SQL Server database for the repository called DBA_ Rep, and SQL Server 2005 Reporting Services (SSRS). (For more information about Sqlcmd, see the Web-exclusive sidebar "Sqlcmd," http://www.sqlmag.com, InstantDoc ID 95387; for related articles about SQL Server 2005 and SSIS, see the Learning Path.) In this article, I explain how to use Sqlcmd and SSIS to construct and populate the DBA_Rep repository database. In a later article I'll explain how to build and deploy the three SSRS reports designed to deliver the data from this repository. (Note: This article makes several nonstandard recommendations, such as querying system tables directly, employing pseudo temp tables, and using xp_cmdshell to run commands. If you stringently adhere to best practices and use only supported techniques, you'll need to withhold your judgment temporarily—until you see that the nonstandard methods I use are efficient and aren't detrimental.)

Create the Repository Database
Now that you have a list of servers to use as input, you might wonder how you can use that input directly in an SSIS package. But don't get ahead of yourself—first, you must store the information somewhere. As most DBAs know, the best place to store a list of data is in a table. Before we examine the SSIS package, let's take a look at the database that will be the repository for the SSIS load.

The table that will store the list of servers on the network from the Sqlcmd command is called ServerList_SSIS. In addition to this table, six other base tables in the DBA_Rep database store data to give a DBA enough basic information at a glance to answer some fundamental questions about the state of each server. These tables are SQL_Servers, Database_Info, Databases, Backup_History, Jobs, and Jobs_Archive. Each of these six tables holds specific information about each SQL Server instance. Web Listing 1 (http://www.sqlmag.com, InstantDoc ID 95385) contains each table's schema; this listing also serves as the script to build the database for the SSIS package to populate the database. To run this script successfully, you need to create a blank database called DBA_Rep in your SQL Server 2005 instance. After you create the DBA_Rep database, the script that Web Listing 1 contains will create the necessary tables in the database.

The table ServerList_SSIS is empty, waiting to be filled with server names. Although I didn't automate this process, you can use SSIS techniques similar to the following techniques that I discuss to easily do so. I used the special stored procedure xp_ cmdshell to run the Sqlcmd shell to return the server list. Assuming that you've run the script to create the DBA_Rep database that contains the ServerList_SSIS table, you can easily use an Insert Into statement with the Sqlcmd /Lc command to insert the server data. In SSMS, open a new query window and enter the following command:

USE DBA_Rep 
   GO 
Insert Into ServerList_SSIS Exec xp_cmdshell ‘sqlcmd /Lc'

At this point you need to consider the following caveats. First, the result set returns NULL records. The table can accommodate NULL records, and the SSIS package's logic will in turn filter out these rows. You could build in logic to take care of the NULL values before the insertion, but I chose to do it as part of the SSIS package. Because this table has no defined indexes that require unique values, truncating the table also ensures that no duplicate rows occur each time the Sqlcmd /Lc command loads the table. You also need to ensure that xp_cmdshell is configured with the proper permissions to execute. By default, xp_cmdshell is disabled in SQL Server 2005; you can use the SQL Server Surface Area Configuration tool to enable xp_cmdshell.

Obtain the Data
The information that's most important to DBAs on a day-to-day basis is server information, such as version (e.g., SQL Server 2005 or 2000), service pack level, edition (i.e., Standard or Enterprise), and default collation. Fortunately, SQL Server 2005 and 2000 store this information in handy system tables. My solution queries each server and stores this information in the SQL_Servers table.

Database information is just as important or even more important than server information; crucial database information includes the two tables Databases and Database_Info. The Databases table simply holds the name of the server and the name of the database. Database_Info holds details, such as size, file location, and recovery model for each database. You can use the Server field to join the Database_Info table to the Databases table. (For ease and simplicity, I used the Server field for joins rather than a key field or a server ID field that I would need to manually update. Although this solution wasn't ideal, it was sufficient for my purposes.)

The final three tables store information about SQL Server Agent jobs and database backups, which I believe is the most important information for any DBA to have. For example, knowing which jobs are failing and need attention is imperative when you're working with hundreds or even tens of database servers and databases. Jobs often fail—and because most jobs perform routine full and transaction log backups if they fail, response must be swift. The Backup_History table holds detailed information about backups that occurred in the past n days. I've found that 5 is a good number of days of history to analyze.

   Prev. page   [1] 2 3     next page



You must log on before posting a comment.

If you don't have a username & password, please register now.

Reader Comments

I can't seem to get this to run. It complains about the MultiServer connection. I followd the article and everything _should_ work, but it isn't. Am I the only one who can't get it to work? I updated the reference to QASRV.DBA_REP to a valid server/db & table on one of my servers. I can populate the ADO recordset successfully, it errors on the ForEach loop...

Error at Populate_DBA_REP [Connection manager "MultiServer"]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.

Also, the ForEach tasks that populate the tables fail validation, I assume because the MultiServer obejct hasn't been populated with a value yet...so it may be acceptable?

Haywood

Article Rating 3 out of 5

Haywood, The connection to QASRV.DBA_REP will be the connection to the repository database itself created from a script provided in the code download. I assume you have created this DBA_Rep database and the QASRV.DBA_REP is pointing to that database. Also, make sure that you can connect to the servers via Management Studio that are in the ADO recordset populated via sqlcmd /Lc in the ServerList_SSIS table. Let me know if I can be of more help getting this working in your environment. (Also, was their a specific SQL error such as "Server does not exist or access denied" message?) Rodney

rlandrum

Article Rating 5 out of 5

*****Note from Author****

Another possible resolution to the issue described in the first reader comment

"Error at Populate_DBA_REP [Connection manager "MultiServer"]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005."

There is a property setting on each of the objects in the package that will delay validation until runtime. I suspect that what is happening is that the package is trying to establish the connection to the MultiServer data source server, which is actually a variable set at runtime. If you set the delay validation object property to TRUE in the object properties (Load Servers Task for example), you should get past the validation issues. Set this property on each object that shows invalid and the package should execute.

rlandrum

Article Rating 5 out of 5

I get the following error on some of my servers, is there a work around for this ? also is there a way to find out which server on the list is generating this error ?

Error 1 Validation error. Load Databases: OLE DB Source [1]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Invalid object name 'tempdb.dbo.HoldforEachDB'.". Populate_DBA_REP.dtsx 0 0

salagheh

Article Rating 4 out of 5

salagheh, Weblisting 3 contains the DDL to create the HoldForEachDB table in TempDB that is required for the validation to complete and for the package to execute locally. Also, after each restart of SQL Server, this table will be deleted because of it's reseidence in TempDB. This only needs to happen on machine where the SSIS package is executing. It will be dynamically created on each target server at runtime. To avoid this, you can also set each objects in the package to delay validation. I have found that this corrects the issue permanently with no need to recreate the HoldForEachDB table. Rodney Landrum (author)

rlandrum

Article Rating 5 out of 5

one issue I see with sqlcmd /Lc is it will only return sql servers listening in its immidiate subnet, which are likely to only be a small subset if all sql servers.

BikeBoy

Article Rating 4 out of 5

I get an error of .....An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x800004005 Description: "Login timeout expired"

sueboo

Article Rating 3 out of 5

I keep getting several errors, seemingly the largest is the problem with the Load Servers task. It always turns red with the OLE DB Source turning red too. I have tried the "delay validation" but it doesn't seem to work. What could be some possible problems and solutions?

sqlmast

Article Rating 3 out of 5

Note from author Make sure that you can connect to each of the server individually via SSMS with the same name as what is in the Serverlist_SSIS table that feeds the variable. Next, make sure you have permissions to access each of the servers in the list. Finally, check the Progress tab for the detailed error and also the Error List and Output windows. Rodney Landrum

rlandrum

Article Rating 5 out of 5

I know that I can connect to each server and I know that I have the rights to the servers. The two top errors that I get are 1) [OLE DB Source [1]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "'SERVERPROPERTY' is not a recognized function name.". 2) [Connection manager "MultiServer"] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E4D. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E4D Description: "Communication link failure". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E4D Description: "TCP Provider: An existing connection was forcibly closed by the remote host. ". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E4D Description: "Login failed for user '**********\***DBA'.".

-I think that the main problem may be the connection manager in the second part. Maybe the login is wrong. I have a system where I can connect to the servers via windows authentication, but when I remote desktop out to the server, I need to use a special account called ***DBA. I have used '*' to denote company information. Could this be the problem with multiserver? If not I am open to suggestions. Finally, if the author would be interested, I would love to have a discussion, aka learn from the author, about SSIS and the configuration database. I know that it would most likely be a treat for my team to be able to arrange this.

sqlmast

Article Rating 5 out of 5

sqlmast, I would be willing to discuss the article, SSIS, Reporting Services et al with you and your DBA team. My e-mail is posted in the bio if you like to contact me directly. Thank You, Rodney Landrum - author.

rlandrum

Article Rating 5 out of 5

Awesome! Have been wanting something like this for quite awhile since we have been adding SQL instances at an alarming rate. Seemed like such a huge task because I haven't used SSIS much, but your instructions were excellent. Added steps to audit logins, roles and database users. In order to do that I did have to add a default value to the SVR_Conn variable. On to the reports! Thanks again.

abates100

Article Rating 5 out of 5

Great Article...and appreciate the generosity for the code....:)

I did a similar solution and my SSIS package just gets the backup info....But when I connect to SQL2005 servers I get this message...I am executing the SSIS packages from a JOB in SQL2005, the agent account is a domain account and it has access to all servers...I am able to get information for all 2000 servers but 2005 servers have this issue: Any input will be appreciated:

Error: 2007-09-06 02:26:48.54 Code: 0xC0202009 Source: DatabaseBackupsForEachServer Connection manager "Source Conn" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections.". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Communication link failure". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Named Pipes Provider: No process is on the other end of the pipe. ". End Error Error: 2007-09-06 02:26:48.55 Code: 0xC020801C Source: Data Flow Task Source [1] Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Source Conn" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed. End Error

rsqlmag

Article Rating 5 out of 5

Great article, this is going to be immensly useful! One bit of info, not explicitly stated, that I was having trouble finding, is how the connection is dynamically set. You have the variable set, but never mention that. For those who might be having the same problem finding this vital bit of info(ssis beginners unite!), it is set in the connection manager as an expression: @[User::SRV_Conn].

sgreene

Article Rating 5 out of 5

rsqlmag, This could be caused by a number of things. Here are a couple to check out. Named Pipes is not enabled by default. If for whatever reason a Named Pipe connection is being tried for the 2005 servers you can enable it via the SQL Server Configuration Manager under SQL Server 2005 Network Configuration. Also, make sure that remote connections are allowed. This can be enabled with the Surface Area Configuration for Services and Connections. Set Remote Connections to use both, or if local connections only is checked select, allow local and remote connections. On SQL Server Express, Local connections only is the default if I am not mistaken. Hope this helps. Rodney Landrum - author.

rlandrum

Article Rating 5 out of 5

See More Comments  1   2