3

Resolved

Exception when generic type contains no back tick

description

When an Assert.Equal assertion fails, the code reporting the failure throws an exception when an involved type is generic, but don't contain a back tick (`) in its type name.

This is the case for F# sequences, which are compiled into generic types with names such as mkSeq@543

Steps to reproduce

Write the following unit test in F#:
[<Fact>]
let Repro() =
    let expected = [ 3; 1; 2 ] |> Seq.sort
    let actual = [ 1; 3; 5 ] |> Seq.sort
    Assert.Equal<int>(expected, actual)

Expected result

Notice that the test is expected to fail, since the two sorted lists contain different elements. The expected result is:
Test 'FSharpSeqAssertRepro.Workaround' failed: Assert.Equal() Failure
Position: First difference is at position 1
Expected: Int32[] { 1, 2, 3 }
Actual: Int32[] { 1, 3, 5 }
C:\Users\mark\Desktop\FSharpSeqAssertRepro\FSharpSeqAssertRepro\Library1.fs(15,0): at FSharpSeqAssertRepro.Workaround()

Actual result

The test fails with the following exception:
Test 'FSharpSeqAssertRepro.Repro' failed: System.ArgumentOutOfRangeException : Length cannot be less than zero.
Parameter name: length
at System.String.InternalSubStringWithChecks(Int32 startIndex, Int32 length, Boolean fAlwaysCopy)
at Xunit.Sdk.AssertActualExpectedException.ConvertToSimpleTypeName(Type type)
at Xunit.Sdk.AssertActualExpectedException.ConvertToString(Object value)
at Xunit.Sdk.AssertActualExpectedException..ctor(Object expected, Object actual, String userMessage, Boolean skipPositionCheck)
at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer)
at Xunit.Assert.Equal[T](IEnumerable`1 expected, IEnumerable`1 actual)
C:\Users\mark\Desktop\FSharpSeqAssertRepro\FSharpSeqAssertRepro\Library1.fs(9,0): at FSharpSeqAssertRepro.Repro()

Workaround

It's possible to work around this bug by converting the lazy sequence into a well-known BCL generic type, such as an array:
[<Fact>]
let Workaround() =
    let expected = [ 3; 1; 2 ] |> Seq.sort |> Seq.toArray
    let actual = [ 1; 3; 5 ] |> Seq.sort |> Seq.toArray
    Assert.Equal<int>(expected, actual)
However, it would be nice to have this error fixed.

Resolution

It should be possible to resolve this issue by modifying Xunit.Sdk.AssertActualExpectedException.ConvertToSimpleTypeName(Type). Right now, this method implicitly assumes that backTickIdx is zero or positive, but an if statement should test that assumption before calling into baseTypeName.Substring.

The tricky part of doing this is to add a unit test reproducing the issue. This is easy to do if it's OK to add a new unit test project, written in F#, to the overall xunit solution. If not, I don't know how to reproduce this behaviour.

If you take pull requests, I'd be happy to provide a fix.

comments

BradWilson wrote Aug 9, 2013 at 3:53 PM

Sure, we would be happy to take a pull request for this. :)

ploeh wrote Aug 9, 2013 at 5:51 PM

OK. Just to be sure about this: is it acceptable that I add an F# unit test project to the solution, then?

BradWilson wrote Aug 24, 2013 at 6:28 PM

Fixed in d5e3f9f9d74f74a9d468f92abb178c44041f1ce4.