Thorough Package Logging - Some Lessons Learned and Recommendations

At some point you'll need to decide on a logging methodology, and this is my 2ps worth:

(1) As a starting point I'd say take a look at Jamie's excellent article on event-handler logging: http://blogs.conchango.com/jamiethomson/archive/2005/06/11/SSIS_3A00_-Custom-Logging-Using-Event-Handlers.aspx. Initially I used the same setup and we standardised all packages on it but with hindsight I'd now recommend to do things slightly differently. The main difference would be to not use property expressions to do the insert to the log table but instead for event-handling, call a stored procedure. Why? Because the property expression will become completely illegible when you try to escape single and double quotes - both of which might appear in a server-generated error message.
We currently just log the OnError and OnPostExecute events, but I'm looking again to see if any of the others would be particularly useful. Finally I'd say that this goes hand-in-hand with using a decent naming convention (Jamie has some standards in his blog). The reason being that the events will log the task name so if they are all called "Data Flow", "Data Flow 1", "Data Flow 2" etc it's just a complete pain when debugging!

(2) As well as event logging to a central table, do some 'simple workflow message logging' to the same table. In my package template I have 2 tasks which sandwich the rest. This makes reports nice and easy.

 

(3) Make sure each end task in the control flow is a logging message task. In this task use a message that is either constant, or which begins consistently. That way you know that a package always has a start and end message pair or a start and an error message pair. This makes reporting really simple.

(3) Add another parameter to the logging table - "PackageSessionGUID". Have an intial task which generates a guid (use newid() against a SQL OLEDB connection) and which then saves the value to a package variable. Use this variable in all subsequent logging. Why? Because this allows you to distinguish between parallel executes of a package.

(4) Also log to a text file. This can be really useful when the problem lies with the normal logging setup. EG what happens if the connection to the database which is used as a logging repository is not correct? Normally you'd not get any info because you rely on this to write the log entries, but the text file works as a kind of backup log.

(5) Also log the output of the job itself, assuming the package is scheduled.
This will help debug any other problems associated with the package and it's batch file.

 

What did you think of this article?




Trackbacks
  • No trackbacks exist for this entry.
Comments

  • 12/9/2007 5:33 PM Luke wrote:
    One problem with using event handlers for logging: my current project makes extensive use of nested packages, and I found it impossible to create a log4net-like email logger using only event handlers. I had to create a custom, stateful logger to filter out the garbage.
    Reply to this
Leave a comment

Submitted comments will be subject to moderation before being displayed.

 Enter the above security code (required)

 Name

 Email (will not be published)

 Website

Your comment is 0 characters limited to 3000 characters.