Secret unit testing tools no one ever told you about

The secret unit testing tools no one ever told you about
Dror Helper (@dhelper)
Demos: https://github.com/dhelper/SecretUnitTestingToolsDemos

About.ME
Consultant @CodeValue
Developing software (professionally) since 2002
Mocking code since 2008
Clean coder & Test Driven Developer
Pluralsight Author
Blogger: http://blog.drorhelper.com

Why should I care about tools?

Background: unit testing tools

Server
Dev Machine
Source ControlBuild Server
Test Runner
Code
Coverage
Build Agent
Unit Testing
Framework
Isolation
Framework
?

xUnit test framework
public class BeforeAndAfter {
[SetUp]
public void Initialize() {
}
[TearDown]
public void Cleanup() {
}
[Test]
public void test1() {
}
[Test]
public void test2() {
}
}
Test Suite
Fixture
Test Case
Test Case
Test Case
Test Case
Test Case
Fixture
Test Case
Test Case
Fixture
Test Case
Test Case
Test Case

Mocking Frameworks
Unit test
Code under test
DependencyFake object(s)

What Mocking framework do?
1. Create Fake objects
2. Set behavior on fake objects
3. Verify method was called
And more...

[Test]
public void Calculate_ReturnTwoValidNumbers_ServerCalled()
{
IDataAccess fakeDataAccess = A.Fake<IDataAccess>();
A.CallTo(() => fakeDataAccess.GetData(A<string>.Ignored))
.Returns(new Tuple<int, int>(2, 3));
var fakeCalculatorService = A.Fake<ICalculatorService>();
var cut = new DistrobutedCalculator(fakeDataAccess, fakeCalculatorService);
cut.Calculate();
A.CallTo(() => fakeCalculatorService.Add(2,3)).MustHaveHappened();
}

Definition: unit tests
Automated piece of code that invokes
a unit of work in the system and then
checks a single assumption about the
behavior of that unit of work
[Roy Osherove, The Art Of Unit Testing]

Unit test structure
[Test]
public void MyTest()
{
}

No guidance  Fragile tests  Stop unit testing

What about AAA?
[Test]
public async void GetUserFromUrl() {
var clientFake = A.Fake<IJsonClient>();
A.CallTo(() => clientFake.HttpGetUncompressedAsync(A<string>.Ignored))
.Returns(Task.FromResult(JsonResult));
var userRepository = new UserRepository(clientFake);
var user = await userRepository.GetUser(11361);
var expected = new User {
Id=11361, DisplayName = "Dror Helper", ImageUrl=DefaultAvatar, Reputation=13904
};
Assert.That(user, Is.EqualTo(expected));
}
 Act
 Assert
Arrange

Arrange issues
[TestMethod]
public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() {
var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 });
var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
await viewModel.LoadUser();
var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color);
Assert.AreEqual(Colors.White, result);
}

Solution: Setup/TearDown
[TestInitialize]
public async Task InitilizeUserViewModel() {
var user1 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 };
_viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
}
[TestMethod]
await _viewModel.LoadUser();
var result = InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color);
}

You shouldn’t use Setup in unit tests
private UserDetailsViewModel _viewModel;
[TestInitialize]
public async Task InitilizeUserViewModel() {
_viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
}
[TestMethod]
var result = InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color);
}

Solution: Extract to methods
[TestMethod]
var user1 = CreateUser(reputation: 10);
var user2 = CreateUser(reputation: 10);
var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color);
}

Over extraction
[TestMethod]
public async Task LoadUser_ReputationStaysTheSame() {
var viewModel = InitializeSystem(10, 10);
CheckColor(Colors.White, viewModel);
}

The problem with factories
private User CreateUser(int reputation) {
return new User {
ImageUrl = "http://dummy.jpg",
Reputation = reputation
};
}
private User CreateUser(int reputation, string imageUrl) {
return new User
{
ImageUrl = imageUrl,
Reputation = reputation
};
}

Builder pattern
class UserBuilder
{
private int _id, _reputation;
private string _name, _imageUrl;
public UserBuilder() {
_id = 1;
_name = "dummy";
_imageUrl = "http://dummy.jpg";
}
User Build() {
return new User {
Id = _id, Name = _name, ImageUrl = _imageUrl, Reputation = _reputation
};
}
}

Builder pattern (cont.)
class UserBuilder
{
...
public UserBuilder WithName(string displayName) {
_displayName = displayName;
return this;
}
public UserBuilder WithReputation(int reputation) {
_reputation = reputation;
return this;
}
...
}
var user1 = new UserBuilder()
.WithReputation(10)
.Build();

Tool: AutoMocking Containers
Test Container SUT
http://blog.ploeh.dk/2013/03/11/auto-mocking-container/
New()
Configure
CreateCreate SUT
Act

Automocking with AutoFixture
[Fact]
public void YellIfTouchHotIron() {
var fixture = new Fixture().Customize(new AutoFakeItEasyCustomization());
Fake<IMouth> fakeMouth = fixture.Freeze<Fake<IMouth>>();
Fake<IHand> fakeHand = fixture.Freeze<Fake<IHand>>();
A.CallTo(() => fakeHand.FakedObject.TouchIron(A<Iron>._)).Throws<BurnException>();
var brain = fixture.Create<Brain>();
brain.TouchIron(new Iron {IsHot = true});
A.CallTo(() => fakeMouth.FakedObject.Yell()).MustHaveHappened();
}
https://helpercode.com/2013/12/22/on-object-creation-and-unit-tests/

Problem: Complex inputs
• Big objects
• Deep objects
• Need for precision
• Lack of knowledge

Solution: use trivial inputs
Sometimes missing the point
Not always enough for “real test”

Solution: Serialization
Supported in most programming languages (XML, json).
Need development  testing delayed
Production code change  indefinitely delayed

Can you spot the problem?
[TestMethod]
public void PerformSomeActionReturns42()
{
var myClass = ...
bool initOk = myClass.Initialize();
var result = myClass.PerformSomeAction();
Assert.IsTrue(initOk);
Assert.AreEqual(42, result);
}
http://stackoverflow.com/q/26400537/11361

Can you spot the problem?
[TestMethod]
public void TestPasswordComplexity()
{
var result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "1!").Result; //Changes the password.
Assert.IsFalse(result.Succeeded);
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789").Result; //Changes the password.
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789!").Result; //Changes the password.
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijk").Result; //Changes the password.
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijK1!").Result; //Changes the password.
Assert.IsTrue(result.Succeeded);
}
http://stackoverflow.com/q/26400537/11361

How many Assert(s) per test?
One Assert Per Test!
Two Assert == Two Tests  Usually ???
”(…the Code is more)
what you'd call guidelines
than actual rules”

Sometimes multiple asserts make sense
[TestMethod]
public void CompareTwoAsserts()
{
var actual = GetNextMessage();
Assert.AreEqual(1, actual.Id);
Assert.AreEqual("str-1", actual.Content);
}

public class AssertAll
{
public static void Execute(params Action[] assertionsToRun)
{
var errorMessages = new List<exception>();
foreach (var action in assertionsToRun)
{
try
{
action.Invoke();
}
catch (Exception exc)
{
errorMessages.Add(exc);
}
}
if(errorMessages.Any())
{
var separator = string.Format("{0}{0}", Environment.NewLine);
string errorMessage = string.Join(separator, errorMessages);
Assert.Fail(string.Format("The following conditions failed:{0}{1}", Environment.NewLine, errorMessage));
}
}
}
http://blog.drorhelper.com/2011/02/multiple-asserts-done-right.html

Using AssertAll
[TestMethod]
public void CompareTwoAsserts()
{
var actual = CreateMessage();
AssertAll.Execute(
() => Assert.AreEqual(1, actual.Id),
() => Assert.AreEqual("str-1", actual.Content);
}

Some frameworks are catching up!
https://github.com/nunit/docs/wiki/Multiple-Asserts

Choosing the right Assert?
IsTrue vs. AreEqual
Parameter ordering confusion
StringAssert/CollectionAssert
It’s all about proper error messages

Tool: 3rd party assertion libraries
Better error messages
Readability
Multiple asserts*
×Additional dependency
×Limited UT framework support
×System.Object “SPAMED” by extension methods

Shouldly
[Fact]
public void AddTest()
{
var calculator = new Calculator();
var result = calculator.Add(2, 3);
Assert.Equal(6, result);
}
[Fact]
public void AddTest_Shouldly()
{
var calculator = new Calculator();
var result = calculator.Add(2, 3);
result.ShouldBe(6);
}
https://github.com/shouldly/shouldly
Shouldly.ShouldAssertException
result
should be
6
but was
5
Xunit.Sdk.EqualException
Assert.Equal() Failure
Expected: 6
Actual: 5

FluentAssertions
[Fact]
public void CompareTwoObjects()
{
var customer1 = new Customer("cust-1", "John Doe");
var customer2 = new Customer("cust-2", "John Doe");
customer1.ShouldBeEquivalentTo(customer2,
o => o.Excluding(customer => customer.Id));
}
http://www.fluentassertions.com/

AssertHelper
[Test]
public void CheckCompare()
{
var myClass = new MyClass();
Expect.That(() => myClass.ReturnFive() == 10);
}
[Test]
public void CheckTrue()
{
var myClass = new MyClass();
Expect.That(() => myClass.ReturnFalse() == true);
}
[Test]
public void StringStartsWith() {
var s1 = "1234567890";
Expect.That(() => s1.StartsWith("456"));
}
[Test]
public void CollectionContains()
{
var c1 = new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 };
Expect.That(() => c1.Contains(41));
}
https://github.com/dhelper/AssertHelper

Test structure issues
What to call the test?
AAA is not mandatory
What should I test?
How to avoid unreadable, complicated tests?
- Unit testing framework provide no structure

The BDD approach
Step
Definitions

Specifications == focused test
Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen

BDD Example: SpecFlow
http://www.specflow.org/

Tool: BDDfy
[TestClass]
public class CardHasBeenDisabled {
private Card _card;
private Atm _subject;
void GivenTheCardIsDisabled() {
_card = new Card(false, 100);
_subject = new Atm(100);
}
void WhenTheAccountHolderRequestsMoney() {
_subject.RequestMoney(_card, 20);
}
void ThenTheAtmShouldRetainTheCard() {
Assert.IsTrue(_subject.CardIsRetained);
}
void AndTheAtmShouldSayTheCardHasBeenRetained() {
Assert.AreEqual(DisplayMessage.CardIsRetained, _subject.Message);
}
[TestMethod]
public void Execute()
{
this.BDDfy();
}
}

Everybody needs a CI server
Unit tests without a CI server are a waste of time
- if you're running all of the tests all of the time
locally you're a better man then I am

Tool: Continuous testing
DotCover
Typemock Runner
nCrunch
VS2017 (Enterprise)

The right tools will help you write good tests
Arrange
Builder Pattern
AutoMocking
Containers
Export
Assert
Shouldly
FluentAssertions
AssertHelper
Test
Structure
BDDfy
Continuous
Testing
Typemock
Runner
DotCover
nCrunch
Live Unit Testing

Dror Helper | @ dhelper | http://helpercode.com
Thank you!
Demos: https://github.com/dhelper/SecretUnitTestingToolsDemos

Secret unit testing tools no one ever told you about

More Related Content

What's hot (20)

Similar to Secret unit testing tools no one ever told you about (20)

More from Dror Helper (20)

Recently uploaded (20)

Secret unit testing tools no one ever told you about

Editor's Notes