• subscribe
August 21, 2008 12:00 AM

Can the Transaction Log Tell Us What Happened?

SQL Server Pro
InstantDoc ID #100076

In all my years of teaching SQL Server classes and answering questions on public forums, one of the most commonly asked questions has always been “How can I see the information in a database’s transaction log?” Most people asking this question seem to want this information to be able to recover data that was lost due to the infamous "user error." Some people seem to get upset, or even irate, when told that Microsoft doesn't provide a facility for viewing the contents of the log, as if it's their unalienable right to be able to see this information. In my opinion, the log contents are akin to the source code, and buying a license to use the product should in no way guarantee that you'll be able to see exactly how SQL Server does what it does. The log format and its contents are proprietary information; part of the reason for not making this information freely available is that it would expose too much information about the way SQL Server actually works.

If you, or your users, perform an action, you should know what you did. You can prevent unwarranted actions, such as running DELETE operations that remove too many rows, by using triggers or by allowing DELETEs only through carefully coded and tested stored procedures. If you let your users do things they shouldn’t, you’ll need to clean up your own mess. You can also set up custom auditing mechanisms to get the information you want get from the log by using SQL Trace. SQL Server 2008 provides an even richer array of auditing capabilities.

The transaction log is intended for SQL Server’s internal use, such as when performing a rollback operation or restoring data changes that you've captured in a log backup. In addition, the log lets SQL Server protect your database from being corrupted, which could happen if there were a system failure in the middle of a data modification operation. Think of what might happen if the data was updated, but SQL Server hadn't yet updated all the index pointers before the failure occurred. If there were no transaction log recording what changes were being made, SQL Server wouldn't realize upon system restart that the index structures were inconsistent and corrupt. The log lets SQL Server run recovery on system restarts and undo any incomplete operations.

Although Microsoft doesn’t provide you with the ability to read the transaction log, there are third-party products that provide this functionality. The vendors developing these tools based their work on an undocumented command, DBCC LOG, which was initially added to SQL Server to help support providers track down strange system behavior. Back when I worked for product support at Sybase, we used DBCC LOG regularly, and in fact the online Sybase documentation (at http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.39996_1250/html/svrtsg/svrtsg102.htm) still includes this command.

The first commercial log reader tool, from Lumigent, was actually acquired from a small consulting company that was run by a Sybase consultant. The consultant had written a log reader tool based on his knowledge of DBCC LOG and used this tool in his consulting business. Lumigent acquired the tool and made it commercially available. Other vendors began offering log reader tools, and SQL Server Magazine published a comparative review of the Lumigent and Red Gate Software log reader products in the article "When SQL Server Tool Vendors Compete, DBAs Win," at http://www.sqlmag.com/Articles/ArticleID/93174/93174.html.

Looking at the article now, it's not obvious which SQL Server version is being discussed. The structure of the log records and the exact details that are returned with the DBCC LOG command changed substantially between SQL Server 2000 and SQL Server 2005, so any vendor that wrote a tool and examined the results of DBCC LOG would have to completely rewrite their product for SQL Server 2005. Throughout the review, no SQL Server version is mentioned. Finally, at the end of the review, a third log reader tool from ApexSQL is mentioned with the comment that it supports SQL Server 2005, which leads us to assume that Lumigent and Red Gate Software's tools don't support SQL Server 2005. And none of the vendors have announced support for SQL Server 2008. 

So what can you do? You can search the Internet for references to using DBCC LOG and the function that wraps around it: fn_dblog. You can try using the output from one of those commands to get an idea of what has been done to your data. However, your best course of action is to plan ahead, either by preventing unwanted changes before they happen or by defining a trace to your own specifications that you can analyze when needed.



ARTICLE TOOLS

Comments
  • Dan
    4 years ago
    Aug 25, 2008

    FYI, the Red Gate 'SQL Log Rescue' product specifically tells you that it only works with SQL Server 2000 when you try to run it against a newer version.

  • David
    4 years ago
    Aug 23, 2008

    Most of us manage just fine without a log reader tool. I don't think it should be top of the agenda, but it would be a very welcome addition to SQL Server.

    I had a play about with the Lumigent tool a little while ago. I think I remember that it was able to roll back individual transactions in the log file - this could potentially be a lifesaver.

    DBAs benefit when 3rd party vendors write tools for SQL Server. We still want to see as many features as possible squeezed into the RTM though. Despite popular belief, I don't think MS is trying to take over the world - I'm sure they balance the demands of it's users without setting out to alienate 3rd party vendors. Adding a feature like this would be a major blow to a company like Lumigent and could possibly make other vendors wary about developing apps to enhance SQL Server.

    On a similar note - I wonder how the new intellisense in SQL 2008 will affect RedGate? They are probably pleased about the lack of intellisense support for SQL2000/2005 in SSMS 2008. Intellisense is a popular feature I think MS would need to include it in SQL Server at some point. I guess it probably didn't come as that much of a surprise. Also, they seem to have a number of other tools that I can't see been included in SQL Server any time soon...

  • KALEN
    4 years ago
    Aug 23, 2008

    Wow... I have never gotten so many comments so quickly about one of my articles. I must really have touched a nerve here!

    There is a difference between the actual data rows referenced by the log, and the log format. It's the log format, and giving people full details about what is in the log, that is propriatary information. There is nothing specifically bad about giving people that information. However, calling me names because I don't stand up on a soapbox and DEMAND that MS add this functionality seems a little extreme. There are plenty of other things MS could do with the product and providing a log reader tool is way down on the list.

    Yes I realize it is important to some people, but there are many other ways to get this information through tracing etc. If the developer resources are limited at MS, I would much prefer they spend their time on more important stuff. MS knows it's important that people have this information, that's why they added a great deal of additional tracing capabilities in SQL Server 2008.

    Also, keep in mind that a log reader tool wouldn't help you debug problems with logic, or with bad reports due to faulty SELECTs. If your WHERE clause was written badly, a log reader tool could tell you which rows were affected, but not WHY. You'd need a tracing tool for that. Vogelm's comment that a log reader tool would help troubleshoot bad queries from 3rd party apps is not true; you need to see the statements for that, not just the affected data.

    I do appreciate kbreneman's comment that the real problem is one of perception. MS should make clear that the transaction log is not an audit tool; if you want auditting, you need to set it up on your own, because you're the only one who knows what's important for you to capture.

    Thanks for all the feedback!
    Kalen

  • Ravindra
    4 years ago
    Aug 21, 2008

    It is disappointing to see a highly regarded professional like you being mediocre. You need to demand more from a tool like SQL Server and expect it to rise up and provide a log reader. Oracle has done this long ago and is a great diagnostic/audit tool. It is not rocket science. Microsoft has already opened up full .NET source code. So I don't see much substance in the claim that somebody will misuse the proprietary log format; well if they do, kudos to them - I hope their product will have a log reader ;)

  • Preethiviraj
    4 years ago
    Aug 21, 2008

    Dear Kalen,
    We expect more from you.
    We do not want to open the Log file using a binary editor and read everything. We want a tool to read the basic data and show us what has happened. Atleast they should disclose some information on LOG, may be from DBCC LOG or fn_dblog()

    If Lumigent and RedGate can provide the data, why can't Micorsoft. Is it MS way of supporting the clients. I see this as a move for earning revenue through Product Support.

You must log on before posting a comment.

Are you a new visitor? Register Here
  • SP1?
    I know there is a SP1 for SQL 2008 R2 available....and there is a "feature pack" as well... ...
  • SQL database mirroring
    I have SQL Server 2008 R2 Enterprise 64bit on Windows 2008 R2 Enterprise 64bit.  Each SQL Server has...
  • Dell Compellent Disk Drive
    Does anybody has experience with Dell Compellent Disk Drive? Basically, this system manages all disk...
  • Sql server performance tuning
    I need to find a tool that help me to optimize sql server,queries,improve the performance and solve ...