Pfthb

A few days ago (11/12) our CI builds for a .NET 4.7 Windows Forms app started "partially succeeding" unexpectedly. I traced the problem back to the Visual Studio Test steps. After breaking out the unit tests into separate steps for each DLL I've been further able to isolate it to the UI test DLL.

It is a plain vanilla MSTest project where the classes test portions of the UI that can be manipulated behind the scenes. Mostly manipulating a control's data model and then inspecting details of controls to ensure that data model changes have the expected effect. UI elements are instantiated in code, either through new declarations within a class or TestClasses that are children of the controls being tested. No message boxes are presented nor UI elements actually rendered for humans or external automation to review.

These tests are running in an Azure "Hosted VS2017" environment. The test runs that fail all have exactly the same error message:

The active test run was aborted. Reason: Unhandled Exception: System.AppDomainUnloadedException: The application domain in which the thread was running has been unloaded.

I have not been able to discern a pattern with the failures. The exception can occur after different individual tests. Running the tests in isolation, in parallel, or not parallel doesn't appear to impact the failures. The failures do appear to be increasing in frequency however. All attempts to recreate this locally fail; any Visual Studio 2017 installation we've used to run the unit tests has passed them all without issue.

All I can really find on this particular topic via Google is either entries to long-past versions of Visual Studio where this was a bug that's been corrected, or to other unanswered questions. Has anyone ran into this kind of trouble before?

Self-followup to document how I resolved this for my situation.

The cause: instantiating objects that derive from System.Windows.Forms.Control or System.Windows.Forms.Form without properly disposing of them.

My fix: wrapped all instantiations inside a using() block.

When a background thread is created in any test and throws an exception while other tests are still running this will crash the test run.

It's important that any code that starts threads correctly waits for these threads to finish and to handle any exceptions that may occur. Or to set up your test so that it can perform this error handling and/or clean-up.

Having background threads running while other tests execute can make your whole test suite flakey :). As you may have found out. And probably can cause problems with your production code as well.