Testing multiple properties with single assertion

Every time I was trying to test an object's properties I was neither satisfied writing very verbose tests nor in using some of the out of the box hamcrest matchers. Although using the matchers was a big help, I never managed to make them read the way I wanted.

Another thing that was very important to me, I wanted to have a single assertion per method and a very descriptive description if the test did not pass.

I've decided to write my own matcher and hopefully it will be useful to other people. So, that's what I've done:

BeanMatcher

Hamcrest matcher to match multiple attributes of an object within a single assertion.

How to use it

NOTE: Make sure you are using org.hamcrest.MatcherAssert.assertThat instead of the JUnit one.

If you run this test, you will get a message like

Now, change the age check to

   property("age", greaterThan(60)) 

And you should get:

Testing object graphs

You can also do this

I use a combination of two matchers to do that:
- BeanMatcher: Provides the "has" method responsible to group all the property matchers.
- BeanPropertyMatcher: Provides the "property" method.

I expect to make more changes to them, so for the most up-to-date version, please check BeanMatcher on my github account.

Enjoy!!!

Extract, Inject, Kill: Breaking hierarchies (part 3)

In part one I explain the main idea behind this approach and in part two I start this example. Please read parts one and two before reading this post

Although the main ideas of Extract, Inject, Kill is already expressed, it's good to finish the exercise just for completion's sake. Here is where we stopped:

Let's have a look at the VoucherPricingService, that now is the only concrete class at the bottom of our hierarchy. 

Note that it uses the VoucherService class to calculate the voucher value.

Before anything, let's write some tests to VoucherPricingService.java

Once thing to notice is that the User parameter is not used for anything. So let's remove it.

Now it is time to user the Extract, Inject, Kill on the VoucherPricingService. Let's Extract the content of the VoucherPricingService.applyAdditionalDiscounts(double, String) method and add it to a class called VoucherDiscountCalculation. Let's call the method calculateVoucherDiscount(). Of course, let's do that writing our tests first. They need to test exactly the same things that are tested on VoucherPricingService.applyAdditionalDiscounts(double, String). We also take the opportunity to pass the VoucherService object into the constructor of VoucherDiscountCalculation.

If you noticed, when doing the extraction, we took the opportunity to give proper names to our new classes and methods and also to pass their essential dependencies to the constructor instead of using method injection.

 

Let's now change the code in the VoucherPricingService to use the new VoucherDiscountCalculation and see if all the tests still pass.

Cool. All the tests still pass, meaning that we have the same behaviour, but now in the VoucherDiscountCalculation class, and we are ready to move to the Inject stage.

Let's now inject VoucherDiscountCalculation into PricingService, that is the top class in the hierarchy. As always, let's add a test that will test this new collaboration.

And here is the changed PriningService.

Now it is time to kill the VoucherPricingService class and kill the PricingService.applyAdditionalDiscounts(double total, String voucher) template method, since it is not being used anymore. We can also kill the VoucherPricingServiceTest class and fix the PricingServiceTest removing the applyAdditionalDiscounts() method from the testable class.

So now, of course, we don't have a concrete class in our hierarchy anymore, since the VoucherPricingService was the only one. We can now safely promote UserDiscountPricingService to concrete.

That is now how our object graph looks like:

Our hierarchy is another level short. The only thing we need to do now is to apply  Extract, Inject, Kill once again, extracting the logic inside UserDiscountPricingService into another class (e.g. UserDiscountCalculation), inject UserDiscountCalculation into PricingService, finally kill UserDiscountPricingService and the calculateDiscount(User user) template method.  UserDiscountPricingService, 

Since the approach was described before, there is no need to go step by step anymore. Let's have a look at the final result.

Here is the diagram representing where we started:

  After the last Extract, Inject, Kill refactoring, this is what we've got:

The cool thing about the final model pictured above is that now we don't have any abstract classes anymore. All classes and methods are concrete and every single class is independently testable. 


That's how the final PricingService class looks like:


For a full implementation of the final code, please look at https://github.com/sandromancuso/breaking-hierarchies

Note: For this three part blog post I used three different approaches to drawing UML diagrams. By hand, using ArgoUML and Astah community edition. I'm very happy with the latter. 

Extract, Inject, Kill: Breaking hierarchies (part 2)

In part 1 of this post I explain the problems of using the template method in deep class hierarchies and how I went to solve it. Please read it before reading this post.

Here is a more concrete example in how to break deep hierarchies using the Extract, Inject, Kill approach. Imagine the following hierarchy.

Let's start with the StandardPricingService. First, let's write some tests:

Note that I used  a small trick here, extending the StandardPricingService class inside the test class so I could have access to the protected method. We should not use this trick in normal circumstances. Remember that if you feel the need to test protected or private methods, it is because your design is not quite right, that means, there is a domain concept missing in your design. In other words, there is a class crying to come out from the class you are trying to test.

Now, let's do the step one in our Extract, Inject, Kill strategy. Extract the content of the calculateProductPrice() method into another class called StandardPriceCalculation. This can be done automatically using IntelliJ or Eclipse. After a few minor adjusts, that's what we've got.

And the StandardPriceService now looks like this:

All your tests should still pass.

As we create a new class, let's add some tests to it. They should be the same tests we had for the StandardPricingService.

Great, one sibling done. Now let's do the same thing for the BoxingDayPricingService.

Now let's extract the behaviour into another class. Let's call it BoxingDayPricingCalculation.

The new BoxingDayPriceService is now

We now need to add the tests for the new class.

Now both StandardPricingService and BoxingDayPricingService have no implementation of their own. The only thing they do is to delegate the price calculation to StandardPriceCalculation and BoxingDayPriceCalculation respective. Both price calculation classes have the same public method, so now let's extract a PriceCalculation interface and make them both implement it.

Awesome. We are now ready for the Inject part of Extract, Inject, Kill approach. We just need to inject the desired behaviour into the parent (class that defines the template method). The calculateProductPrice() is defined in the PricingService, the class at the very top at the hierarchy. That's where we want to inject the PriceCalculation implementation. Here is the new version:

Note that the template method calculateProductPrice() was removed from the PricingService, since its behaviour is now being injected instead of implemented by sub-classes.

As we are here, let's write some tests for this last change, checking if the PricingService is invoking the PriceCalculation correctly. 

Great. Now we are ready for the last bit of the Extract, Inject, Kill refactoring. Let's kill both StandardPricingService and BoxingDayPricingService child classes. 

The VoucherPricingService, now the deepest class in the hierarchy, can be promoted to concrete class. Let's have another look at the hierarchy:

And that's it. Now it is just to repeat the same steps for VoucherPricingService and UserDiscountPricingService. Extract the implementation of their template methods into classes, inject them into PricingService, and kill the classes.

In doing so, every time you extract a class, try to give them proper names instead of calling them Service. Suggestions could be VoucherDiscountCalculation and PrimeUserDiscountCalculation.

There were a few un-safe steps in the re-factoring described above and I also struggled a little bit to describe exactly how I did it since I was playing quite a lot with the code. Suggestions and ideas are very welcome.

For the final solution, please check the last part of this blog post.

NOTE
If you are not used to use builders in your tests and is asking yourself where the hell aProduct() and aShoppingBasket() come from, check the code in here:

ProductBuilder.java
ShoppingBasketBuilder.java

For more information about the original problem that triggered all this, please read part 1 of this blog post.
In part 3 I finish the exercise, breaking the entire hierarchy. Please have a look at it for the final solution. 

Extract, Inject, Kill: Breaking hierarchies (part 1)

Years ago, before I caught the TDD bug, I used to love the template method pattern. I really thought that it was a great way to have an algorithm with polymorphic parts. The use of inheritance was something that I had no issues with. But yes, that was many years ago.

Over the years, I've been hurt by this "design style". That's the sort of design created by developers that do not TDD.

The situation

Very recently I was working on one part of our legacy code and found a six level deep hierarchy of classes. There were quite a few template methods defined in more than one of the classes. All classes were abstract with the exception of the bottom classes, that just implemented one or two of the template methods. There were just a single public method in the entire hierarchy, right at the very top.

We had to make a change in one of the classes at the bottom. One of the (protected) template method implementations had to be changed.

The problem

How do you test it? Goes without saying that there were zero tests for the hierarchy.

We know that we should never test private or protected methods. A class should "always" be tested from its public interface. We should always write tests that express and test "what" the method does and not "how". That's all well and good. However, in this case, the change needs to be done in a protected method (template method implementation) that is part of the implementation of a public method defined in a class six level up in the hierarchy. To test this method, invoking the public method of its grand grand grand grand parent we will need to understand the entire hierarchy, mock all dependencies, create the appropriate data, configure the mocks to have a well defined behaviour so that we can get this piece of code invoked and then tested.

Worse than that, imagine that this class at the bottom has siblings overriding the same template method. When the siblings need to be changed, the effort to write tests for them will be the same as it was for our original class. We will have loads of duplications and will also need to understand all the code inside all the classes in the hierarchy. The ice in the cake: There are hundreds of lines to be understood in all parent classes.

Breaking the rules

Testing via the public method defined at the very top of the hierarchy has proven not to be worth it. The main reason is that, besides painful, we already knew that the whole design was wrong. When we look at the classes in the hierarchy, they didn't even follow the IS-A rule of inheritance. They inherit from each other so some code could be re-used.

After some time I thought: Screw the rules and this design. I'm gonna just directly test the protected method and then start breaking the hierarchy.

The approach: Extract, Inject, Kill

The overall idea is:

1. Extract all the behaviour from the template method into a class.

2. Inject the new class into the parent class (where the template is defined), replacing the template method invocation with the invocation of the method in the new class.

3. Kill the child class (the one that had the template method implementation).

Repeat these steps until you get rid of the entire hierarchy.

This was done writing the tests first, making the protected template method implementation public.

NOTES

1. This may not be so simple if we have methods calling up the stack in the hierarchy.

2. If the class has siblings, we have to extract all the behaviour from the siblings before we can inject into the parent and kill the siblings.

This probably is too complicate to visualise, so in part 2 of this post I'll be giving a more concrete example. 

Code coverage is a side effect and not an end goal

In my previous life as a consultant, one of our clients hired us to "increase the code coverage" of their code base. We would not develop any new features or make any changes. They just wanted us to write tests. Basically they wanted us to bring the coverage up to 70%. Why 70% you may ask. Probably because someone pulled a finger out of his or her back orifice, sniffed it and thought: Hmm.. this definitely smells like 70%.

Regardless the stupid and random number, as consultants we actually did our best to actually write proper and meaningful tests. They did not want us to refactoring. Just tests. The developers themselves (client employees) would not be writing tests. Probably too busy doing proper work, breaking the tests after each commit. Thanks for them, they had the monkeys here fixing the tests. The reason for that was that the same person with the smelly finger decided that everyone would have a higher bonus if they reached 70% test coverage. 

More recently, I've seen a similar behaviour. Keeping a close eye on the metrics provided by Sonar, that does a great job by the way, managers and senior developers are constantly pushing the teams to improve the quality of the code. Time is allocated in every sprint and developers are asked to increase the code coverage. The intention behind it is good, right? Bit by bit the application get better. Awesome. 

The idea was to improve the quality and reliability of the application. Unfortunately, developers with the urge to finish the task, just focused on the "increase code coverage" part of the task. They then started writing "tests" that were not asserting anything. Basically what they were doing was writing some code inside a test method that would invoke public methods in classes but would not test the outcomes or side effects of those method invocations. 

What does code coverage measure?

It measures the percentage of the code exercised by code written inside test methods. It does not measure the percentage of code tested by testing code. Having a test method that just invokes a public method in a class and does not test (assert/verify) anything, will make the code coverage go up but will not test the system. This, of course, is totally pointless since it does not test if the public method invoked actually does what it is supposed to do or if it does anything at all.

Why do we invest time in writing tests? (In no particular order)

• Make sure we understand what the application does, according to the executable requirements (tests); 
• Make sure that the application still does the right thing after we make changes to it or indicates possible areas of conflicts (failing tests);
• Make sure we have regression test; 
• Make sure that in a click of a button we can know if the application is working correctly; 
• Make sure that we have a quick and small feedback loop in terms of quality and correctness of our application; 
• Make sure the application is simplified and easily maintainable, via testing and refactoring; 
• Make sure the complexity of the application is spread thin instead of having localised big balls of mud that we are scared to touch;
• Make sure that we can easily add new features quickly; 
• Make sure that we extend the ROI in our software, extending its lifespan; 
• Make sure that our software does not rot, impeding and/or slowing down business progress; 
• Make sure that our clients are happy with the software;

By the way, when I mention test, I mean all types/levels of tests, not just unit.

What should we focus on?

Focus on what the metrics are telling you and not on improving the numbers. Focusing on numbers may give you a false impression of quality.

My point here is that the focus should be on TESTING the application and not in increasing its code coverage metric. We need to make sure we capture the behaviour of our application via tests, expressing its intent, refactoring to make it better. Tests (at all levels) should tell you a story about the behaviour of your application. And guess what? In writing tests for your application, besides all the benefits mentioned above, your code coverage metric will still go up as a side effect. 

Mentorship in Software Craftsmanship - part 3

In previous posts I covered:
Part 1: Mentor and mentee roles, gains and sacrifices, mutual respect
Part 2: How to choose mentors and mentees and mentorship misconceptions



Walking the long road together (or at least part of it)

Once the relationship between mentor and mentee is established, it is fair to say that they will be in a journey together. Every software craftsman is on a personal journey towards mastery. They are all walking the long road. Both mentor and mentees will be learning while they share part of their journey with each other.

What mentors and mentees should do during the mentorship?

Well, this may vary a lot depending of the type of mentorship. In general, they are expected to write code, loads of code. Ideally they should build something together, where the mentee should be the most active in terms of writing the code. The mentor is expected to pair-program with the mentee whenever possible and to be reviewing his code.

Agreeing on creating an application would probably be the best option since that would involve not just writing the code but also thinking about requirements, being able to prioritize, defining a development process, deployment strategies, production environment and everything else that is related to a software project in the real life.

Working on katas is also a valid. It's a simpler and quicker approach for the mentee to learn basic skills. This could be used when the mentees are interested in the basics of TDD, naming, refactoring, programming languages paradigms, etc. However, after learning a few basic skills using katas, they should aim to do something larger that really can mimic the sort of applications they would be working on in their professional environments.

Establishing goals and tracking progress

Establishing goals is definitely a good practice in a mentorship. It keeps mentor and mentees focused in whatever they want to achieve. Working towards an objective is always the best way to study and learn something. Goals should be achievable and used to motivate and direct the menteed and not to be treated as a hard deadline. However, they should be as concrete as they can be, for example, writing an application with features X, Yand Z and have it deployed into production or doing a number of katas using TDD, write blog posts, read books, submit patches to open source projects, or whatever the pair agrees on.

It's important that progress is tracked so both mentor and mentees can see how they are getting on and how much the mentee is evolving. Tracking progress is all about feedback. The quicker and shorter the feedback loop is, the better. It's also a good tool to trigger conversation about improvements and refinements.

How long a mentorship should last?

Unfortunately that's another question that does not have an exact answer. This will depend a lot on the type of mentorship, how much the mentee wants to learn from the mentor and also how much the mentor has to offer.

Some say that this should be a lifetime commitment, some say that it should last between 2 to 5 years and some say that it could be as short as a few months. Some mentorships start with very technical and specific things like learning the basics of a language or test disciplines. However they can evolve to an entire project lifecycle or even to a longer term career advice, networking, help with books and conferences, etc.

I, personally, would never try to define that upfront. Just enjoy the journey and let time tell when the mentorship should terminate.

How and when does it terminate?

For the majority of the relationships, both mentor and mentees at some point need to continue their journey in separate ways. This does not mean that they will never talk to each other again or that they don't like each other. This just means that they need to move on, teach or learn from other people. Also, we all change our minds in terms of what our next steps would be and we need to react to that.

One important thing is that regardless who wants to terminate the relationship, this termination should be explicit. It should be clear to both parts that the mentorship is over.

Professionalism, Reputation and Public recognition

Software craftsmanship is all about professionalism and it is almost impossible to talk about professionalism without talking about reputation.

Throughout the mentorship, it is important that the mentor advertises the progress of her mentee. Mentors should public recognise all the skills acquired by the mentee, what would help the mentee to start building her own reputation. However, mentors also need to be aware that every time they vouch for someone, they are also putting their reputations on the line. Mentees are also expected to be recognising their mentors for all the things they've been learning. This mutual recognition is one of the things that may help both to build their reputations.

Raising the bar

Mentorship is at the heart of software craftsmanship and this is probably one of the most efficient ways for us, developers, to help raising the bar of our industry. Sharing our knowledge and experiences with each other is what will help us learn from our mistakes and successes.

We probably can teach many things to someone in a fraction of the time that took us to learn them. With this, mentees could absorb a lot of knowledge in a much shorter space of time, making them, overtime, a much more complete professional than any of the mentors she had in her career.

Regardless of our level of seniority or if we are being mentored by someone else, if we all take the responsibility to mentor someone, we will all be helping to raise the bar of our industry.


For previous parts of this blog post, please check:

Part 1: Mentor and mentee roles, gains and sacrifices, mutual respect
Part 2: How to choose mentors and mentees and mentorship misconceptions

Installing Io language on Ubuntu

Since I'm new to Ubuntu and I had a hard time installing Io language, I've decided to record my steps. Hopefully I'll remember all of them and other people won't struggle as much as I did. I'm running Ubuntu 11.10.

The main problem I had was that I did not have all the dependencies needed to install Io and the installation was failing. The dependencies are yajl and libevent

If you've got then installed, you can skip the next steps.

NOTE: You will need to have cmake and make. You can install them running the following commands from the terminal:
sudo apt-get install cmake
sudo apt-get install make

Installing yajl

1. Download yajl from here: https://github.com/lloyd/yajl/downloads
1. In my case it was: lloyd-yajl-2.0.1-0-gf4b2b1a.zip
2. Extract it into a folder.
3. Open the terminal and go to the folder you extracted yajl
4. Run the following commands
1. mkdir build
2. cd build
3. cmake ..
4. sudo make install

For more information, please check:
https://github.com/lloyd/yajl
https://github.com/lloyd/yajl/blob/master/BUILDING

Installing libevent

1. Download libevent from here: http://libevent.org/
1. In my case it was: libevent-2.0.15-stable.tar.gz
2. Extract it into a folder
3. Open the terminal and go to the folder you extracted libevent
4. Run the following commands
1. ./configure
2. make
3. sudo make install

For more information please check:
http://libevent.org/
Also check the README file inside the .tar.gz for your version

Installing Io language

You can go to the Io language website and download the language or click here to go start downloading it. As there is no Ubuntu package for that, you will be downloading the Io's source code. This will point to Steve Dekorte's version. I ended up downloading Jeremy Tregunna's version. It should work the same. Check each one is the most up-to-date.

1. Download the Io version here: https://github.com/jeremytregunna/io/zipball/master
2. Extract it into a folder.
3. Open the terminal and go to the folder you extracted Io
4. Run the following commands:
1. mkdir build
2. cd build
3. cmake ..
4. sudo make install

IMPORTANT: When running 'cmake ..', you may get a few errors. Even then, try to run 'sudo make install'. Some libraries may fail to compile because they are OS specific.

For mode information please check:
http://iolanguage.com/
https://github.com/stevedekorte/io
https://github.com/jeremytregunna/io


Updating ld.so.conf

Now we just need to update ld.so.conf so Io can be accessed from anywhere in your computer.

1. From the terminal, type the following command: sudo gedit /etc/ld.so.conf
2. Add the following line to the file: include /usr/local/lib
3. Save and close the file
4. From the terminal run the following command: sudo ldconfig

Running Io

That's it. Hopefully now you will be able to open a terminal window and type: io
You should see the Io runtime environment: Io> _