This month's script, GroupedShutdowns, is dedicated to those hard-working systems administrators who've had to manually reboot all their servers in the middle of the night. Manually accessing the console on numerous servers can take time and cause mistakes. Even writing a script that performs the shutdown and restart via your favorite command-line tool can fail if nodes don't respond to the Shutdown command. A properly designed script can come to the rescue by making this repetitive task easier and the results accurate and verifiable.
The GroupedShutdowns script can help you manage a sequenced shutdown and restart of many nodes. You can use any of several available command-line tools to perform remote shutdowns and restarts. However, issuing shutdown/restart commands to your entire server environment simultaneously can cause some obvious problems. I've found that a two-tiered approach works best. Such an approach lets you shut down and restart clustered or load-balanced servers sequentially, so that resources remain online on at least one node throughout the reboot process. This method lets you reboot file or database servers first, then boot Web servers, which are dependent on file-server virtual directories or databases.
How GroupedShutdowns Works
GroupedShutdowns' main tasks are to shut down and restart all the servers in an input list. The shutdowns occur in two groups; the first group of servers must all be back online before the second group reboots. After all the servers are online, the script verifies that the Shutdown command has worked.
The excerpt from Grouped-Shutdowns in Listing 1 shows the essential parts of the script. The script performs these basic steps:
- Divide the input list into two groups of reboot targets.
- Verify the input list reboot targets by checking whether each server in the list can be pinged, as the code at callout A in Listing 1 does.
- Shut down the first group of servers by issuing them the Shutdown command. Pause the script for 120 seconds to let users save their work. The code at callout D invokes the Shutdown command to perform the shutdown. (I discuss Shutdown and the other commands that GroupedShutdowns uses in the next section.)
- After a 4-minute timeout (this value is configurable), begin testing whether the first group's nodes are online again. If the nodes are online, proceed to the next group of servers. If any node remains offline after the initial 4-minute timeout, the script performs four additional 1-minute tests. After these tests, if any nodes still remain offline, the script notifies the operator and aborts the run.
- Shut down the second group of servers by issuing them the Shutdown command. After a 4-minute timeout (this value is configurable), begin testing whether the first group's nodes are online again. If the nodes are online, proceed to the next group of servers as in Step 4. If any node remains offline after the initial 4-minute timeout, the script performs four additional 1-minute tests and then a Ping test. If any nodes remain offline after these tests complete, the script notifies the operator and aborts the run.
- Perform a final verification that all nodes in the two groups responded to the Shutdown command by logging the uptime on each node, as the code at callout E does.
Notice that GroupedShutdowns performs a couple of repetitive tasks: It pings some or all the nodes at three different times and issues the Shutdown command once for each of the two server groups. The script also needs to abort to let the operator solve a problem, then restart where it left off. Say a pinged node hasn't returned online and the systems administrator has to stop the script and manually reboot the node. After the node is online again, you need a mechanism to restart the script from that point. The challenges of repeating tasks within a script and restarting a script at a particular point provide an opportunity to introduce two handy scripting techniques—multiple-usage routines and entry-point labels—which I explain in the sidebar "New Tricks for Your Scripting Tool Belt."
Tools That GroupedShutdowns Uses
GroupedShutdowns calls three utilities: Sleep (sleep.exe), Uptime (uptime.exe), and Shutdown (shutdown.exe). Sleep, which has been around since the Microsoft Windows NT Server 4.0 Resource Kit, lets you schedule a program to wait for a specified number of seconds. GroupedShutdowns calls sleep.exe to pause the script for 4 minutes. Sleep is available in the Microsoft Windows Server 2003 Resource Kit, which you can download at http://www.microsoft.com/downloads/details.aspx?familyid=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en.
Uptime also dates back to the NT 4.0 resource kit. Uptime isn't included in recent resource kits, but you can download it at http://www.microsoft.com/ntserver/nts/downloads/management/uptime. Uptime lets you determine when a server was last rebooted. GroupedShutdowns calls uptime.exe to determine whether the targeted servers actually rebooted from the remote shutdown/restart command. By examining the server uptime information in the uptime.txt log file, you can verify that the servers all rebooted correctly.
Shutdown, which lets you shut down and reboot a PC, is a built-in command in Windows 2003 and Windows 2000 Server. To view the command's switch options, at a command line enter
shutdown /?
Be careful not to omit the /? switch; if you do, the command will initiate a shutdown of your PC or server. If you initiate an unintended shutdown and want to stop it, you can abort the shutdown by running the command
shutdown -a
I tested the GroupedShutdowns script on a Windows XP Service Pack 2 (SP2) PC. Consequently, Grouped-Shutdowns uses the XP version of shutdown.exe, which differs slightly from the Windows 2003 version. Be sure you examine the nuances of the switches on the version of shutdown.exe you're using to determine the correct syntax for your situation. The major difference you'll find is that some versions precede switches with a dash (-), whereas others precede switches with a slash (/).
Prev. page
[1]
2
next page 
