xUnit.net - Unit testing framework for C# and .NET (a successor to NUnit)

Created Issue: TestRunner does not set ERRORLEVEL for file not found. [9775]

AlGonzalez — Tue, 21 Feb 2012 04:03:35 GMT

When I call the xunit.console.exe test runner with an assembly that does not exist, I get the following error:

error: file not found: {assemblyFileName}

The problem is that the exit code (ERRORLEVEL) is not set. Testing "IF ERRORLEVEL 1" does not work.

Commented Issue: unwanted deselection occurs in xUnit.net Test Runner [9773]

bartelink — Mon, 20 Feb 2012 08:44:13 GMT

to reproduce ...
(a) download the samples
(b) compile them
(c) run xunit.gui stack.dll
(d) 14 tests pass with [ Run All ]
(e) select a subset of the tests
(f) click [ Run Selected ]
(g) the subset test(s) pass

BUT
the selection is lost
and
the button returns to its [ Run All ] state.

imho, the subset state should remain until cleared by the programmer

edit: this applies to xunit.gui.exe and xunit.gui.clr4.exe.

Comments: Dup of http://xunit.codeplex.com/workitem/8799 ?

Commented Issue: unloading assembly followed by unhandled exception [9774]

gerrylowry — Sun, 19 Feb 2012 21:37:37 GMT

win 7 vs2010
-----------------------
See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain.
at System.AppDomain.get_SetupInformation()
at Xunit.ExecutorWrapper.get_ConfigFilename()
at Xunit.Gui.RunnerForm.UnloadAssemblies(IEnumerable assemblyListViewItems)
at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ToolStrip.WndProc(Message& m)
at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

************** Loaded Assemblies **************
mscorlib
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.488 (RTMLDR.030319-4800)
CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll
----------------------------------------
xunit.gui.clr4
Assembly Version: 1.9.0.1566
Win32 Version: 1.9.0.1566
CodeBase: file:///C:/xUnit19/xunit.gui.clr4.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.454 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System.Drawing
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.1 built by: RTMRel
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.523 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
xunit.runner.utility
Assembly Version: 1.9.0.1566
Win32 Version: 1.9.0.1566
CodeBase: file:///C:/xunit19/xunit.runner.utility.DLL
----------------------------------------
System.Xml
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.450 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
System.Web
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.547 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_64/System.Web/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Web.dll
----------------------------------------
System.Core
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.450 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
xunit.runner.utility.{Dynamic}
Assembly Version: 0.0.0.0
Win32 Version: 1.9.0.1566
CodeBase: file:///C:/xunit19/xunit.runner.utility.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
Comments: steps to reproduce load a recent assembly ... click the [ Run All ] button ... load the same recent assemble ... now there are two copies ... unload one of the copies ... unload the remaining copy

Commented Issue: unloading assembly followed by unhandled exception [9774]

gerrylowry — Sun, 19 Feb 2012 21:34:50 GMT

win 7 vs2010
-----------------------
See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain.
at System.AppDomain.get_SetupInformation()
at Xunit.ExecutorWrapper.get_ConfigFilename()
at Xunit.Gui.RunnerForm.UnloadAssemblies(IEnumerable assemblyListViewItems)
at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ToolStrip.WndProc(Message& m)
at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

************** Loaded Assemblies **************
mscorlib
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.488 (RTMLDR.030319-4800)
CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll
----------------------------------------
xunit.gui.clr4
Assembly Version: 1.9.0.1566
Win32 Version: 1.9.0.1566
CodeBase: file:///C:/xUnit19/xunit.gui.clr4.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.454 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System.Drawing
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.1 built by: RTMRel
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.523 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
xunit.runner.utility
Assembly Version: 1.9.0.1566
Win32 Version: 1.9.0.1566
CodeBase: file:///C:/xunit19/xunit.runner.utility.DLL
----------------------------------------
System.Xml
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.450 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
System.Web
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.547 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_64/System.Web/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Web.dll
----------------------------------------
System.Core
Assembly Version: 4.0.0.0
Win32 Version: 4.0.30319.450 built by: RTMLDR
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
xunit.runner.utility.{Dynamic}
Assembly Version: 0.0.0.0
Win32 Version: 1.9.0.1566
CodeBase: file:///C:/xunit19/xunit.runner.utility.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
Comments: steps to reproduce ... load a recent assembly ... immediately load it again.

Created Issue: unloading assembly followed by unhandled exception [9774]

gerrylowry — Sun, 19 Feb 2012 21:32:31 GMT

Edited Issue: unwanted deselection occurs in xUnit.net Test Runner [9773]

gerrylowry — Sun, 19 Feb 2012 21:28:07 GMT

Edited Issue: xUnit.net GUI Test Runner missing "Help", "Help About" [9772]

gerrylowry — Sun, 19 Feb 2012 21:24:18 GMT

generally, to identify the version of a GUI program under Windows, one checks
"Help", "About _____"

currently, to view the version, one needs to check the .exe properties via the containing folder.

edit: this applies to xunit.gui.exe and xunit.gui.clr4.exe.

Created Issue: unwanted deselection occurs in xUnit.net Test Runner [9773]

gerrylowry — Sun, 19 Feb 2012 21:16:47 GMT

Created Issue: xUnit.net GUI Test Runner missing "Help", "Help About" [9772]

gerrylowry — Sun, 19 Feb 2012 21:10:53 GMT

generally, to identify the version of a GUI program under Windows, one checks
"Help", "About _____"

currently, to view the version, one needs to check the .exe properties via the containing folder.

New Post: xUnit.net Home page Table of Contents browser layout

mhthomas5 — Fri, 17 Feb 2012 22:42:25 GMT

I am talking about the comparisons page under:

How does xUnit.net compare to other .NET testing frameworks?

New Post: xUnit.net Home page Table of Contents browser layout

mhthomas5 — Fri, 17 Feb 2012 22:36:52 GMT

Apparently using HTML tables to display xUnit.net Home page Table of Contents is FAIL in Firefox and Chrome, can someone PLEASE test the page on at least one browser other than IE? The last column of all the tables is cut off.

New Post: Getting test debugging to work

cadmann — Tue, 07 Feb 2012 19:47:16 GMT

Okay, a duh moment. We missed a step.

For future searches in the forums, we did:

Open project properties and go to debug tab.

Under "Start Action" set to "Start External Program" to the xunit runner executable of choice.

Under "Start Options" set "Command Line Arguments" to the name of your project's debug dll.

Then set "Working Directory" to the project's bin/Debug directory (which we missed before).

Then press F5 to run your test in debug mode. Breakpoints should work.

New Post: Getting test debugging to work

cadmann — Tue, 07 Feb 2012 19:14:00 GMT

Hi,

We're new to xUnit.net and are very interested in using it going forward for our test framework.

We have a test running under xUnit.net using the xunit.console runner from inside VS 2010. We configured the project's Debug property to run the xunit.console as an external tool. We did that by using cmd.exe, passing "/c <path-to-xunit.console.exe> test.dll" When we run the test in debug, the console pops up in an external window, but our breakpoint is not hit. At this point, google led me to this post: http://bradwilson.typepad.com/blog/2010/06/whats-new-in-xunitnet-16.html

At the moment, we've avoided TestRunner.net and Resharper since they are commercial. We tried Gallio, but didn't have any success.

Any pointers on how to get debugging of xunit.net tests going in VS 2010?

Thanks!

Commented Issue: Assert.Equal with IEnumerable fails after matching entire sequence [9771]

moswald — Tue, 07 Feb 2012 18:47:44 GMT

I have two IEnumerable<string>s that contain the same data. However, they are provided by two different functions so their type doesn't match, and I think that is confusing Assert.Equal. Here is a quick repro case:

(moved to http://pastebin.com/nBa5wYik due to codeplex formatting fail)

Results:

UnitTests.MyTests.FailingTest[FAIL]
Assert.Equal() Failure
Position: First difference is at position 15
Expected: UnitTests.MyTests+<Flatten>d__1e`1[System.String]
Actual: System.Linq.Enumerable+<SelectManyIterator>d__14`2[System.Collections.Generic.IEnumerable`1[System.String],System.String]

It is entirely possible that this is as designed, since the two types don't match, but I would argue that if I'm testing two IEnumerables, I care more about their contents and not where they came from.
Comments: Ah, sorry. I didn't realize this project hadn't updated to the 1.9 nuget package. All clear.

Commented Issue: Assert.Equal with IEnumerable fails after matching entire sequence [9771]

bartelink — Mon, 06 Feb 2012 23:51:03 GMT

I have two IEnumerable<string>s that contain the same data. However, they are provided by two different functions so their type doesn't match, and I think that is confusing Assert.Equal. Here is a quick repro case:

(moved to http://pastebin.com/nBa5wYik due to codeplex formatting fail)

Results:

UnitTests.MyTests.FailingTest[FAIL]
Assert.Equal() Failure
Position: First difference is at position 15
Expected: UnitTests.MyTests+<Flatten>d__1e`1[System.String]
Actual: System.Linq.Enumerable+<SelectManyIterator>d__14`2[System.Collections.Generic.IEnumerable`1[System.String],System.String]

It is entirely possible that this is as designed, since the two types don't match, but I would argue that if I'm testing two IEnumerables, I care more about their contents and not where they came from.
Comments: What version? (There have been recent changes going from 1.8 to 1.9)

Edited Issue: Assert.Equal with IEnumerable fails after matching entire sequence [9771]

moswald — Mon, 06 Feb 2012 15:06:18 GMT

I have two IEnumerable<string>s that contain the same data. However, they are provided by two different functions so their type doesn't match, and I think that is confusing Assert.Equal. Here is a quick repro case:

(moved to http://pastebin.com/nBa5wYik due to codeplex formatting fail)

Results:

UnitTests.MyTests.FailingTest[FAIL]
Assert.Equal() Failure
Position: First difference is at position 15
Expected: UnitTests.MyTests+<Flatten>d__1e`1[System.String]
Actual: System.Linq.Enumerable+<SelectManyIterator>d__14`2[System.Collections.Generic.IEnumerable`1[System.String],System.String]

It is entirely possible that this is as designed, since the two types don't match, but I would argue that if I'm testing two IEnumerables, I care more about their contents and not where they came from.

Created Issue: Assert.Equal with IEnumerable fails after matching entire sequence [9771]

moswald — Mon, 06 Feb 2012 15:03:58 GMT

I have two IEnumerable<string>s that contain the same data. However, they are provided by two different functions so their type doesn't match, and I think that is confusing Assert.Equal. Here is a quick repro case:

public static IEnumerable<T> Flatten<T>(this T[,] items)
{
for (int i = 0; i < items.GetLength(0); ++i)
{
for (int j = 0; j < items.GetLength(1); ++j)
{
yield return items[i, j];
}
}
}

public static IEnumerable<T> Flatten<T>(this IEnumerable<IEnumerable<T>> items)
{
return items.SelectMany(enumerable => enumerable);
}

[Fact]
public void FailingTest()
{
var origData = new[,]
{
{ "A", "B", "C" },
{ "D", "E", "F" },
{ "G", "H", "I" },
{ "J", "K", "L" },
{ "M", "N", "O" }
};

List<List<string>> lists = new List<List<string>>();

for (int i = 0; i != origData.GetLength(0); ++i)
{
var newList = new List<string>();

for (int j = 0; j != origData.GetLength(1); ++j)
{
newList.Add(origData[i, j]);
}

lists.Add(newList);
}

Assert.Equal(reelData.Flatten(), lists.Flatten());
}

// results:
UnitTests.MyTests.FailingTest[FAIL]
Assert.Equal() Failure
Position: First difference is at position 15
Expected: UnitTests.MyTests+<Flatten>d__1e`1[System.String]
Actual: System.Linq.Enumerable+<SelectManyIterator>d__14`2[System.Collections.Generic.IEnumerable`1[System.String],System.String]

It is entirely possible that this is as designed, since the two types don't match, but I would argue that if I'm testing two IEnumerables, I care more about their contents and not where they came from.

New Post: Test Runner NuGet Package

idavis — Mon, 06 Feb 2012 06:41:32 GMT

This was just released on 2/1/2012: http://nuget.org/packages/xunit.runners

Thanks guys!

Source code checked in, #90cf5f87a714

bradwilson — Wed, 01 Feb 2012 18:29:36 GMT

Added xunit.runners NuGet package

New Post: Asynchronous test runs

mtanneryd — Tue, 31 Jan 2012 22:14:07 GMT

That's great news! Looking forward to it.