SQL Server Magazine

Over the past several months, I've examined many aspects of LoadSim. In the first installment of this series, I presented basic LoadSim topics, such as the theory behind LoadSim and the approach you will use to test Exchange servers with it. In part 2, I discussed LoadSim installation and configuration. In part 3, I showed you how to customize LoadSim settings for your specific user profile requirements. This month, I'll show you how to collect and analyze data, and how to make sense of the work you're going to do with LoadSim.

In part 2, I proposed a sample scenario for walking through LoadSim installation and configuration, and I'll continue with that scenario in this article. I assumed two major requirements: First, I wanted to ensure that one server sufficiently handles 300 users that closely match LoadSim's Medium user profile. Second, I wanted response times of 1 second or less.

Saving Your Work and LoadSim Settings
When you're finished configuring LoadSim as I was at the end of part 3, be sure to save your work. From the File menu, select Save, and give the file a name. LoadSim has a few different support files, which I describe in Table 1, page 172. For this example, name the file lstest.sim, and save the file in your LoadSim home directory for later use.

Initializing the Exchange Database
Now that you have fully configured LoadSim and have names in the Exchange Server Directory, you need to initialize the Exchange database for the test. In part 2, when I created the LoadSim topology, the Exchange Directory was populated with LoadSim users in a recipient container called LoadSim. At this point, these imported users have nothing in their mailboxes; they need to have some data before you start a test run. Select Run, Initialize Test from the LoadSim main menu to begin populating Microsoft Exchange's Information Store with data. This action simulates having mail in each mailbox as though the LoadSim users had been using Exchange all along.

For the Medium user profile, shown in Table 2, page 174, the LoadSim users read existing mail 15 times a day. The Initialize Test process creates that existing mail. Also, having some data in the database doesn't give the Exchange database engine an unfair performance advantage of dealing with an empty database.

Running Initialize Test adds data to the private store (priv.edb). If you plan to run LoadSim on a live database, note that this approach adds data to your live database for the LoadSim users that are in the LoadSim recipients container. You can delete the data when you finish your test. If you are running multiple tests so you can compare the results from each test, I suggest you start with the database at the same state each time. In other words, don't let the messages pile up in the LoadSim users' mailboxes; always start your test with the same number of messages in the mailboxes.

When you run Initialize Test, the LoadSim client begins methodically stuffing messages into mailboxes on the server according to the parameters specified in the Initialization tab in the Customize Test property sheet, which Screen 1 shows. This property sheet appears when you select Configuration, Test Properties from the LoadSim main menu and click Customize.

You also use Initialize Test to populate the public store (pub.edb) with messages, just as you do the private store. When you run the Initialize Test, it will ask whether you also want to initialize public folders. This procedure worked differently in LoadSim 4.0, in which you had an explicit Public Folder Initialization process. With LoadSim 5.0, initializing public folders is part of Initialize Test.

Now you're ready to run Initialize Test. Open the Run menu, and click Initialize Test. LoadSim starts initializing the database. You can run this task on one LoadSim client, walk away, and come back when it's finished. You'll then be ready to run the real tests and start gathering data.

At the Medium user setting, Initialize Test increases the priv.edb database by about 2MB for each user. So for 300 users, expect to end up with a 600MB database. The Light user setting is about 500KB for each user. The amount of time Initialize Test takes to complete depends on the user profile you've chosen, the performance of your LoadSim client and Exchange Server machines, and the number of mailboxes LoadSim has to populate.

Running the Test
Now you're ready to run the test. Open the Run menu and select Run Simulation. LoadSim will start churning away as processes start up and LoadSim users log on. Don't worry if LoadSim appears to be doing nothing. Other than an increasing number of users, you won't see any activity on the screen unless LoadSim needs to create Exchange profiles for its users. After all the LoadSim users are logged on, LoadSim has created all the profiles, and LoadSim has started its work, you'll see a screen like Screen 2, page 175.

You'll want to let the test run for a fixed amount of time. If you want reliable scores, let LoadSim run for at least 4 to 5 hours: 1 hour to warm up (reach steady state) and 3 to 4 hours to collect data. Then you can use the LSLOG utility to parse the LoadSim logs.

Analyzing the LoadSim Score with LSLOG
When your LoadSim run finishes, you need to analyze the data that LoadSim gathered to see how the Exchange server performed. The metric you will be looking at is the score, or response time in milliseconds (ms).

LSLOG is a command-line utility to parse the LoadSim logs (.log files) and give you the performance scores for the run. The utility, lslog.exe, comes with LoadSim as one of the program executables. See the Sidebar, "Command Syntax for LSLOG 5.0," page 176, for details.

By default, when you run LSLOG on a LoadSim log, you'll see the 95th percentile score. So, if you are simulating 100 users and 1100 is the LSLOG score, 95 of the simulated users are experiencing an average response time of 1.1 seconds or less. You must use LSLOG to derive the score to use for any data analysis.

Obtaining a set of scores is useful if you want to see how fast Exchange Server is servicing client requests. Furthermore, if you impose performance goals of subsecond client response time, as I did in my example scenario, you expect the score to be 1000 or less during a LoadSim run. Screen 3 shows typical LSLOG output.

To get this output I performed two steps: First, I ran my LoadSim log file through LSLOG's truncate process. From the LoadSim home directory command prompt (assuming lslog.log is in the same directory), I typed

LSLOG truncate lslog.log > lslog1-4.log

LoadSim takes about an hour to reach steady state, so I want to remove the first hour of data and only analyze data from the second through the fourth hours. Conveniently, the truncate parameter defaults to the values I want to use (i.e., 1:00 and 4:00).

Second, I ran the resulting file, lslog1-4.log, through LSLOG's answer process, as shown with the command

LSLOG answer lslog1-4.log

Screen 3 represents the output for my test, which the LSLOG answer command provided. You can see from the Weighted Avg line at the bottom of the screen (identified by <--"score") that the 95th percentile score is 270. This score tells me that 95 percent of the users can expect to experience a response time of 0.27 seconds or less. Realistically, this result predicts that users will experience half-second or less response time for most operations.

This result is well below the subsecond requirement I imposed in the example criteria. I can conclude from the test that the Exchange server under test is providing adequate response time for the users. Assuming you don't find any weird NT Performance Monitor (Perfmon) data on the Exchange Server computer, you are ready to deploy 300 users on the server.

Screen 3 lists other scores, broken out for each activity type in LoadSim. In addition to the respective 95th percentile score, you can see the weight LoadSim gives each task and the number of times the task occurred (Hits). Note that the final average is a weighted average based on the Weight column in Screen 3, not a mean average.

   Prev. page   [1] 2     next page