Monday, March 29, 2010

Using WindDbg and Sosex to Identify an ASP.NET Deadlock

This post will show how to use create an ASP.NET deadlock and then then use WinDbg and sosex to identify it the idea is to practice this before needing it when debugging a production server :)


The environment for my test is

  1. Create a new web application and put this in the code behind for Default.aspx:
    public partial class _Default : Page
    {
        /// <summary>
        /// Locks on page load.
        /// </summary>
        protected void Page_Load(object sender, EventArgs e)
        {
            object lockA = new object();

            object lockB = new object();

            ThreadStart firstJob = new ThreadStart(() => this.DualLockingCall(lockA, lockB));
            Thread firstThread = new Thread(firstJob);
            firstThread.Name = "firstThread";
            firstThread.Start();

            ThreadStart secondJob = new ThreadStart(() => this.DualLockingCall(lockB, lockA));
            Thread secondThread = new Thread(secondJob);
            secondThread.Name = "secondThread";
            secondThread.Start();

            Thread.Sleep(Timeout.Infinite);
        }

        /// <summary>
        /// lock on a then lock on b.
        /// </summary>
        /// <param name="onThis">The first instance to lock on.</param>
        /// <param name="onThat">The second instance to lock on.</param>
        private void DualLockingCall(object onThis, object onThat)
        {
            lock (onThis)
            {
                lock (onThat)
                {
                    Thread.Sleep(TimeSpan.FromSeconds(5));
                }
            }
        }
    }
  2. Navigate to the url like localhost/projectName/Default.aspx
  3. Open a command prompt and navigate to the debugging tools for windows directory.
  4. Run this command adplus -pn w3wp.exe -o c:\ -hang
    1. The hang command takes a memory dump immediately.  It doesn't wait for a hang.
  5. Note the output directory.  It should be the last line adplus printed out.
  6. Run windbg and press ctrl+d to open a memory dump.  Navigate to the dump adplus just created.
  7. type .loadby sos mscorwks - This will load the sos.dll from the same directory mscorwks was loaded from.
  8. type .load pathToSosex\sosex.dll
  9. type !dlk
  10. On my machine I get this output:
    *** WARNING: Unable to verify checksum for LockItUp.DLL
    *** ERROR: Module load completed but symbols could not be loaded for LockItUp.DLL
    Deadlock detected:
    CLR thread 8 holds sync block 0000000002583228 OBJ:00000001bf1f2be0[System.Object]
                 waits sync block 00000000025831e0 OBJ:00000001bf1f2bf8[System.Object]
    CLR thread 9 holds sync block 00000000025831e0 OBJ:00000001bf1f2bf8[System.Object]
                 waits sync block 0000000002583228 OBJ:00000001bf1f2be0[System.Object]
    CLR Thread 8 is waiting at LockItUp._Default.DualLockingCall(System.Object, System.Object)(+0x25 IL)(+0x74 Native)
    CLR Thread 9 is waiting at LockItUp._Default.DualLockingCall(System.Object, System.Object)(+0x25 IL)(+0x74 Native)

    1 deadlock detected.
Easy cheesy in this simulated test eh?



5 comments:

Anonymous said...

I consider, that you are not right. Let's discuss it.

Anonymous said...

Beneficial info and excellent design you got here! I want to thank you for sharing your ideas and putting the time into the stuff you publish! Great work!

Anonymous said...

It’s really a nice and helpful piece of information. I’m glad that you shared this helpful info with us. Please keep us informed like this. Thanks for sharing.

Anonymous said...

Great site. A lot of useful information here. I’m sending it to some friends!

Anonymous said...

I’ve recently started a blog, the information you provide on this site has helped me tremendously. Thank you for all of your time & work.