Crafted Software

Code coverage is a side effect and not an end goal

2012-02-06T22:11:00.000Z

In my previous life as a consultant, one of our clients hired us to "increase the code coverage" of their code base. We would not develop any new features or make any changes. They just wanted us to write tests. Basically they wanted us to bring the coverage up to 70%. Why 70% you may ask. Probably because someone pulled a finger out of his or her back orifice, sniffed it and thought: Hmm.. this definitely smells like 70%.

Regardless the stupid and random number, as consultants we actually did our best to actually write proper and meaningful tests. They did not want us to refactoring. Just tests. The developers themselves (client employees) would not be writing tests. Probably too busy doing proper work, breaking the tests after each commit. Thanks for them, they had the monkeys here fixing the tests. The reason for that was that the same person with the smelly finger decided that everyone would have a higher bonus if they reached 70% test coverage.

More recently, I've seen a similar behaviour. Keeping a close eye on the metrics provided by Sonar, that does a great job by the way, managers and senior developers are constantly pushing the teams to improve the quality of the code. Time is allocated in every sprint and developers are asked to increase the code coverage. The intention behind it is good, right? Bit by bit the application get better. Awesome.

The idea was to improve the quality and reliability of the application. Unfortunately, developers with the urge to finish the task, just focused on the "increase code coverage" part of the task. They then started writing "tests" that were not asserting anything. Basically what they were doing was writing some code inside a test method that would invoke public methods in classes but would not test the outcomes or side effects of those method invocations.

What does code coverage measure?

It measures the percentage of the code exercised by code written inside test methods. It does not measure the percentage of code tested by testing code. Having a test method that just invokes a public method in a class and does not test (assert/verify) anything, will make the code coverage go up but will not test the system. This, of course, is totally pointless since it does not test if the public method invoked actually does what it is supposed to do or if it does anything at all.

Why do we invest time in writing tests? (In no particular order)

Make sure we understand what the application does, according to the executable requirements (tests);
Make sure that the application still does the right thing after we make changes to it or indicates possible areas of conflicts (failing tests);
Make sure we have regression test;
Make sure that in a click of a button we can know if the application is working correctly;
Make sure that we have a quick and small feedback loop in terms of quality and correctness of our application;
Make sure the application is simplified and easily maintainable, via testing and refactoring;
Make sure the complexity of the application is spread thin instead of having localised big balls of mud that we are scared to touch;
Make sure that we can easily add new features quickly;
Make sure that we extend the ROI in our software, extending its lifespan;
Make sure that our software does not rot, impeding and/or slowing down business progress;
Make sure that our clients are happy with the software;

By the way, when I mention test, I mean all types/levels of tests, not just unit.

What should we focus on?

Focus on what the metrics are telling you and not on improving the numbers. Focusing on numbers may give you a false impression of quality.

My point here is that the focus should be on TESTING the application and not in increasing its code coverage metric. We need to make sure we capture the behaviour of our application via tests, expressing its intent, refactoring to make it better. Tests (at all levels) should tell you a story about the behaviour of your application. And guess what? In writing tests for your application, besides all the benefits mentioned above, your code coverage metric will still go up as a side effect.

Mentorship in Software Craftsmanship - part 3

2011-10-28T00:19:00.000+01:00

In previous posts I covered:
Part 1: Mentor and mentee roles, gains and sacrifices, mutual respect
Part 2: How to choose mentors and mentees and mentorship misconceptions

Walking the long road together (or at least part of it)

Once the relationship between mentor and mentee is established, it is fair to say that they will be in a journey together. Every software craftsman is on a personal journey towards mastery. They are all walking the long road. Both mentor and mentees will be learning while they share part of their journey with each other.

What mentors and mentees should do during the mentorship?

Well, this may vary a lot depending of the type of mentorship. In general, they are expected to write code, loads of code. Ideally they should build something together, where the mentee should be the most active in terms of writing the code. The mentor is expected to pair-program with the mentee whenever possible and to be reviewing his code.

Agreeing on creating an application would probably be the best option since that would involve not just writing the code but also thinking about requirements, being able to prioritize, defining a development process, deployment strategies, production environment and everything else that is related to a software project in the real life.

Working on katas is also a valid. It's a simpler and quicker approach for the mentee to learn basic skills. This could be used when the mentees are interested in the basics of TDD, naming, refactoring, programming languages paradigms, etc. However, after learning a few basic skills using katas, they should aim to do something larger that really can mimic the sort of applications they would be working on in their professional environments.

Establishing goals and tracking progress

Establishing goals is definitely a good practice in a mentorship. It keeps mentor and mentees focused in whatever they want to achieve. Working towards an objective is always the best way to study and learn something. Goals should be achievable and used to motivate and direct the menteed and not to be treated as a hard deadline. However, they should be as concrete as they can be, for example, writing an application with features X, Yand Z and have it deployed into production or doing a number of katas using TDD, write blog posts, read books, submit patches to open source projects, or whatever the pair agrees on.

It's important that progress is tracked so both mentor and mentees can see how they are getting on and how much the mentee is evolving. Tracking progress is all about feedback. The quicker and shorter the feedback loop is, the better. It's also a good tool to trigger conversation about improvements and refinements.

How long a mentorship should last?

Unfortunately that's another question that does not have an exact answer. This will depend a lot on the type of mentorship, how much the mentee wants to learn from the mentor and also how much the mentor has to offer.

Some say that this should be a lifetime commitment, some say that it should last between 2 to 5 years and some say that it could be as short as a few months. Some mentorships start with very technical and specific things like learning the basics of a language or test disciplines. However they can evolve to an entire project lifecycle or even to a longer term career advice, networking, help with books and conferences, etc.

I, personally, would never try to define that upfront. Just enjoy the journey and let time tell when the mentorship should terminate.

How and when does it terminate?

For the majority of the relationships, both mentor and mentees at some point need to continue their journey in separate ways. This does not mean that they will never talk to each other again or that they don't like each other. This just means that they need to move on, teach or learn from other people. Also, we all change our minds in terms of what our next steps would be and we need to react to that.

One important thing is that regardless who wants to terminate the relationship, this termination should be explicit. It should be clear to both parts that the mentorship is over.

Professionalism, Reputation and Public recognition

Software craftsmanship is all about professionalism and it is almost impossible to talk about professionalism without talking about reputation.

Throughout the mentorship, it is important that the mentor advertises the progress of her mentee. Mentors should public recognise all the skills acquired by the mentee, what would help the mentee to start building her own reputation. However, mentors also need to be aware that every time they vouch for someone, they are also putting their reputations on the line. Mentees are also expected to be recognising their mentors for all the things they've been learning. This mutual recognition is one of the things that may help both to build their reputations.

Raising the bar

Mentorship is at the heart of software craftsmanship and this is probably one of the most efficient ways for us, developers, to help raising the bar of our industry. Sharing our knowledge and experiences with each other is what will help us learn from our mistakes and successes.

We probably can teach many things to someone in a fraction of the time that took us to learn them. With this, mentees could absorb a lot of knowledge in a much shorter space of time, making them, overtime, a much more complete professional than any of the mentors she had in her career.

Regardless of our level of seniority or if we are being mentored by someone else, if we all take the responsibility to mentor someone, we will all be helping to raise the bar of our industry.

For previous parts of this blog post, please check:

Part 1: Mentor and mentee roles, gains and sacrifices, mutual respect
Part 2: How to choose mentors and mentees and mentorship misconceptions

Installing Io language on Ubuntu

2011-10-23T02:15:00.000+01:00

Since I'm new to Ubuntu and I had a hard time installing Io language, I've decided to record my steps. Hopefully I'll remember all of them and other people won't struggle as much as I did. I'm running Ubuntu 11.10.

The main problem I had was that I did not have all the dependencies needed to install Io and the installation was failing. The dependencies are yajl and libevent

If you've got then installed, you can skip the next steps.

NOTE: You will need to have cmake and make. You can install them running the following commands from the terminal:
sudo apt-get install cmake
sudo apt-get install make

Installing yajl

Download yajl from here: https://github.com/lloyd/yajl/downloads

In my case it was: lloyd-yajl-2.0.1-0-gf4b2b1a.zip

Extract it into a folder.
Open the terminal and go to the folder you extracted yajl
Run the following commands

mkdir build
cd build
cmake ..
sudo make install

For more information, please check:
https://github.com/lloyd/yajl
https://github.com/lloyd/yajl/blob/master/BUILDING

Installing libevent

Download libevent from here: http://libevent.org/

In my case it was: libevent-2.0.15-stable.tar.gz

Extract it into a folder
Open the terminal and go to the folder you extracted libevent
Run the following commands

./configure
make
sudo make install

For more information please check:
http://libevent.org/
Also check the README file inside the .tar.gz for your version

Installing Io language

You can go to the Io language website and download the language or click here to go start downloading it. As there is no Ubuntu package for that, you will be downloading the Io's source code. This will point to Steve Dekorte's version. I ended up downloading Jeremy Tregunna's version. It should work the same. Check each one is the most up-to-date.

Download the Io version here: https://github.com/jeremytregunna/io/zipball/master
Extract it into a folder.
Open the terminal and go to the folder you extracted Io
Run the following commands:

mkdir build
cd build
cmake ..
sudo make install

IMPORTANT: When running 'cmake ..', you may get a few errors. Even then, try to run 'sudo make install'. Some libraries may fail to compile because they are OS specific.

For mode information please check:
http://iolanguage.com/
https://github.com/stevedekorte/io
https://github.com/jeremytregunna/io

Updating ld.so.conf

Now we just need to update ld.so.conf so Io can be accessed from anywhere in your computer.

From the terminal, type the following command: sudo gedit /etc/ld.so.conf
Add the following line to the file: include /usr/local/lib
Save and close the file
From the terminal run the following command: sudo ldconfig

Running Io

That's it. Hopefully now you will be able to open a terminal window and type: io
You should see the Io runtime environment: Io> _

Mentorship in Software Craftsmanship - part 2

2011-10-10T02:14:00.000+01:00

In part one I gave a bit of background history and also described the roles and responsibilities of mentors and mentees according to the Software Craftsmanship principles. In this second post I'll be convering a few other areas related to the relationship itself. Remember that the focus here is on mentorships outside work. Let's start from the beginning.

How do we find mentors or mentees?

That's probably the most difficult question to answer. I can think in three possibilities (in no particular order):

Try to find someone that works with you in a daily basis. It will make the approach easier.
Via user groups and communities. If you are an active member of a user group or community you will have the opportunity to meet a lot of people and maybe find the mentor or mentee you are looking for.
Via indication: A friend could recommend/introduce you to someone.

Choosing a mentor

Although not a rule, normally, in Software Craftsmanship, it is the mentee that chooses the mentor or, at least, start the conversation. The mentee needs to have an idea of the path he wants to take. For example, he may want to learn more about mobile development, work as a consultant, work in high-performance low-latency systems, learn about gaming development, improve his automated testing techniques or, in the best case, just learn how to become a better developer. Whatever the mentee's ambitions are, the mentee needs to make sure that his future mentor is a person that can provide the knowledge he or she is looking for.

Unfortunately, this may be easier said than done. Depending where the mentee is in his career, he may not know exactly what he wants. He may not even know what options he has. Although good developers are good developers and the foundation and priciples of software development always apply, we all know that the skills set used in different types of systems can vary a lot. A well experienced developer that spent the majority of his career doing web-development and became an expert in user experience may take a while to perform well in a completely server-side, asynchronous application with no GUI. Working on an application that heavily rely on algorithms and that don't use any database can be totally different from developing a mobile client application. The same way that working for a consultancy company can be very different from working for a bank or a startup.

Those are some of the things that developers at early stages of their careers may not even have a clue. If you are at this stage in your career, the best suggestion is that you focus purely on being a great developer and learn the basics. Things like Test-Driven Development, clean code, refactoring, Object-Oriented principles, different languages and paradigms. Once you have that, it will be easier for you to choose your next move.

Choosing a mentor involves is not an easy task. Maybe a single mentor won't be able to teach everything a mentee wants to know. Mentees should focus on their very next step(s) instead of focusing in everything they want to learn throughout their entire career. Mentees may change their minds quite often as soon as new opportunities and options are presented to them. They should keep their options open and choose a different mentors as time goes by.

Choosing a mentee

Mentors, before accepting a mentee, should ask themselves: Would I be able to dedicate time to this person? Can I really help his or her career? If the answer is yes, fantastic.

The first thing to look for in a mentee is passion. Since a mentor will be allocating time to help someone, they should make sure that this person deserves their help. Make sure that the mentee is committed to be better.

Another thing mentors need to decide is what sort of seniority level their future mentee should have. Some mentors will prefer to take graduates, others prefer juniors with one or two years of experience and others prefer to get seniors or mentees on the verge of becoming seniors.

Choosing a mentee is a very personal thing and different mentors will have completely different criteria. If the mentor already knows the mentee, than it is easier. When the mentor doesn't know the mentee I could suggest a few options that could either be combined or used independently:

Introduction letter: The mentor could ask the mentee for a summary of his career so far (not a CV). In this summary, the mentee would describe what he has done, where he thinks he are in his career, the things he learnt, provide (if any) links to his public profile like github account, twitter, blog, web/mobile applications and most importantly, what he expect from the mentor.
Coding challenge: The mentor can set up a challenge before considering to speak and accept a mentee. E.g. the mentee, using his preferred language, needs to write a simple blog or wiki application, have it deployed in somewhere (heroku, cloudbees, Google App Engine, Amazon Beanstalk, etc) and code should be public (github, for example). Or it could be simpler like solving a few katas using two different languages or anything along these lines.
Blog: Mentee should create a blog, if he or she does not have one, and publish a few posts related to things he has been learning on his own.

The mentor, if going down this route, should set the challenges according to the level of seniority he is expecting from the mentee. Once the mentor is satisfied with the initial effort by the potential mentee, he could decide if he is going to accept or not the mentee.

Mentorship common misconceptions

The mentor is not better than the mentee. In general the mentor will have more knowledge and experience in the areas where the mentee has chosen to be mentored. However, the mentee can easily have more knowledge than the mentor in other areas. Mentees should not expect the mentor to have all the answers to all the problems and mentors should not be naive to think that the mentee doesn't know anything. Both mentors and mentees have skills and limitations and this needs to be understood by both parts.

Mentors should not be looking for themselves a few years younger. This can be frustrating for the mentor. People are different and the mentor will be missing out when not being more open to the type of mentee he or she is prepared to take on board. There is no reason to be so restrictive. Mentoring someone with different personality, possibly slightly different ambitions can be extremelly enriching for both mentors and mentees. However it is important that both have same values and principles.

Mentees should not expect that mentors will change their lives. The mentors will be there for the mentees, giving them advices and teaching what they know but it is up to the mentee to decide what to do with it. Mentees should do their part to improve and not think they are going to be spoon-fed by mentors. It's up to the mentee to look after his or her own career.

In the next posts I'll be covering things like different types of mentorship, activities that could be performed by mentors and mentees, public recognition, professional reputation, graduation, mentorship duration, how this could change our industry and a few other points. Watch this space.

Part 3: What mentors and mentees should do, how long it should last, how does it end public recognition and reputation

Part 1: A bit of history, roles of the mentors and mentees, gains and sacrifices and mutual respect

Mentorship in Software Craftsmanship - part 1

2011-10-05T00:45:00.000+01:00

How's mentorship seen in Software Craftsmanship?

First, a little bit of background and metaphor

In the medieval times, apprentices would work in workshops an would be mentored by senior craftsmen (journeymen) or by the master craftsman himself. The apprentice had the responsibility to learn, observing the master's and everyone else's work, questioning everything and practising as much as he could. This was different from the teacher/student relationship where the teacher had the responsibility to teach. Here it was the apprentice that had to push his own development.

The apprentice, mentored by the master, would learn and refine his skills on a daily basis. Over time, the master would reach his limit as in what he could teach the apprentice and the knowledge and skills gap between both would not be that big any more. The master would then publicly recognise the apprentice as a professional that could take on work on his own and would deliver it with the same quality that the he would deliver himself. The master, at this point, would be putting his own reputation on the line since he was the one that trained the apprentice and was now vouching for his ability.

This would be the graduation point. Now the apprentice was ready to start his own journey, as a journeyman. As a journeyman, he would then go from town to town, work for and learn from different masters up to a point that he would be recognised by all of these masters and other craftsmen as a master himself. He would then be ready to have his own shop and start mentoring other journeymen and apprentices.

Back to the present

From now on, instead of master/apprentice, I'll be using mentor/mentee. The main reason is that you don't need to be a master craftsman to mentor someone. You also don't need to be an apprentice to have a mentor. Besides that, each developer has different areas of expertise. They can be very senior in certain areas and completely ignorant in other areas. As we all know, software development is not as limited as a blacksmith's workshop in the medieval times.

The role of the mentor

Deciding to mentor someone is a big responsibility. The role of a mentor is more than just be available on the phone and answer a few questions here and there. Although this can be very helpful, the role of a mentor goes way beyond than that. The mentor has the responsibility to make the mentee a good professional and that includes the technical and the personal side. More than just teaching the mentee a specific framework or technique, the mentor will also be teaching the mentee how to become a professional software developer, including ethics and code of conduct. From the technical perspective, the mentor will do his best to teach everything he knows. If they don't work together, the mentor is expected to reserve formal time to work with the mentee. From the personal perspective, a mentor should help the mentee in his career (journey), giving some guidance, advices, opening doors or showing the doors he has already opened himself, introducing the mentee to his own professional circle and whatever else the mentor judges that can be important for the mentees.

The role of the mentee

The mentee is expected to do whatever he or she can to learn from the mentor. The mentee must be open-minded, be able to take criticism on board, be able to listen and also be very proactive in terms of perpetuating the knowledge. Practice is key. The mentee is expected to practice as much as possible, making him or herself better everyday. Mentees are also expected to produce something. Something where the mentor can measure their progress and possibly identify areas of improvement. Besides the direct benefits to the mentee, this is also the best way to keep mentors excited to be helping. Mentees must show passion and a clear desire to learn otherwise mentors will probably loose interest and find it a waste of time.

Gains and sacrifices

Mentors have the opportunity to perpetuate their knowledge since they need to organise their ideas to teach someone. Mentors will also need to be studying and practising hard to keep feeding his mentee, what obviously is a good thing. They will get the satisfaction of helping developers to progress in their careers, with good foundation, ethics and professionalism. They will be doing their part in rasing the bar of our industry, training the next generation, and this, on its own, is a very noble cause. But there are sacrifices and the main one is time. Mentors are expected to dedicate time to their mentees. If mentor and mentee work together in a daily basis, they won't need much time outside work. However, if they don't, mentors need to be clear they will need to find and reserve time for their mentees, ideally in a regular basis.

Mentees have the opportunity to speed up their development as professional software developers. They benefit from the mentor's experience acquired over the years, shortening the time they would take to learn something and avoiding to commit the same mistakes. They also have someone they trust that could offer a different perspective on the things they are going through (technical issue, problem with a manager, process, bureaucracy, deadlines, estimation, etc). Without any scientific evidence to back me up, I would dare to say that with the right attitude from both parts and a good amount of time together, the mentee could learn in two years what the mentor learned in ten. That gives the mentees a massive head-start in their careers. Time should never be called a sacrifice from the mentees perspective. When someone is getting a lot of knowledge and help for free, possibly saving years of their careers, complaining about time (or lack of it) would be very short-sighted, to say the least.

Mutual Respect

Both mentors and mentees should respect each other. In order to have a healthy and prosperous relationship, there must be a commitment from both parts. This commitment is the mutual respect. The mentor shows respect with regular interactions with the mentee and time commitment. The mentee shows respect with his commitment to excel and demonstration of progress. That's the minimum a mentee could do for someone that is sacrificing time with family and friends in order to help him.

In following posts I'll be exploring things like different types of mentorship, activities that could be performed by mentors and mentees, criteria for choosing a mentor and a mentee, public recognition, professional reputation, graduation, mentorship duration, how this could change our industry and a few other points. Watch this space.

Part 2: Choosing a mentor and a mentee and also common mentorship misconceptions
Part 3: What mentors and mentees should do, how long it should last, how does it end public recognition and reputation

LSCC's First Code Retreat

2011-09-20T09:59:00.000+01:00

On September 10th we had the London Software Craftsmanship Community's First Code Retreat.

We had 22 passionate and talented developers working in Java, C#, Ruby, Python and JavaScript. Some of them travelled two hours, waking up as early as 5am, to come to the code retreat and many others travelled at least one hour to be there.
We started at 8am, with breakfast and informal discussions. Then we had a quick introduction, where I explained the purpose of the day and used some of Corey Haines wise words. Here's a quick summary:

"In our day-to-day job, we need to get things done. We need to achieve something and deliver something. And we want and are committed to it. In order to do so, we end up cutting corners. So that's how we code. We sacrifice quality in order to deliver "faster".

But now imagine you have all the time in the world and all the knowledge in the world. Imagine what would be your idea of perfect code. Now compare it to what you do and how you code in your day-to-day job.

This gap is the measure of how much you suck. The bigger the gap is, the more you suck. The smaller the gap is, the less you suck. :)

But today, during the code retreat, is the day that we will practice perfect code. Today we don't need to deliver. Today we have no pressure, besides writing perfect code. That means the objective of a hands-on session like that, is not to finish the exercise, but to practice new approaches, expand our horizons and learn from others. Today we will reduce the gap from what we do in our day-to-day job and the perfect code, providing better value for our customers and being better professionals. "

It's not practice that leads to perfection, it's perfect practice that leads to it.

I also explained the four rules of simple design:
        - All tests need to pass;
        - No duplication;
        - Reveals intention (good naming)
        - Small

I have made this letter longer than usual, only because I have no time to make it shorter.
Blaise Pascal (sometimes also attributed to Mark Twain)

Format of the day

After a good breakfast, developers formed pairs and worked on the Conway's Game of Life. We had three sessions in the morning, then lunch and another three sessions in the afternoon. Each session lasted 45 minutes and had a retrospective afterwards where developers discussed their approaches and problems they faced. Then developers deleted the code from their machines. In each session new pairs were formed and worked on the same problem.

First session: It was free. Basically the pairs worked with no constraints just to make themselves familiar with the problem.
Second session: As a challenge, they were asked to use different data structure other than arrays and we introduced the concept of "primitive obsession".
Third session: Developers were asked not to use flags and use polymorphism instead. This was to promote the exploration of abstractions.

Lunch: We had nice sandwiches and deserts from a good delicatesse. We had 1.5 hours for lunch. During lunch time, some more senior developers said they haven't been challenged enough so I changed the strategy.

Fourth session: In order to satisfy developers from different levels, I asked them to come up with a list of challenges and each pair would then decide which challenge they would take on, according to their level of expertise. The list they came up with was;
- Maximum 3 lines of code per method;
- Object-Oriented Programming to extreme.
- No getters / setters / properties
- Every line must start with "return" or "final" (Java/C# - functional style)
Fifth session: TDD as if you meant it, that means, all code needs to be written inside the test method and then refactored out.
Sixth session: Developers were free to do whatever they wanted again but many decided to use some of the challenges from sesson four.
Pub

During the sessions, I also asked them to try different testing approaches: outside-in and inside-out. The reason was to compare how different the design would turn up. They were also encouraged to pair-program with developers from other languages, what is an amazing experience and broadens our mind.

The experiment with each pair choosing the challenge on session four had a mixed feedback. Some developers enjoyed it because they could push themselves. Others said that the retrospective was not so good since pairs were working with different challenges and could not relate to the problems and solutions exposed by other pairs. This is something I'll need to think about for future code retreats. How can I balance the challenges so everyone feels, well, challenged.

Overall, we all had a fantastic day. There were many very interesting discussions during the retrospectives and, according to the feedback, everyone learnt something. I definitely learnt a lot myself and facilitating my first code retreat was an amazing experience that I want to repeat.

I would like to thank Valtech for providing the venue and for the full sponsorship. They provided us everything we needed for this event and were key for its success.

For the full list of attendees, details about the event and more photos, please check http://www.meetup.com/london-software-craftsmanship/events/27600561/

One year of London Software Craftsmanship Community (LSCC)

2011-08-20T01:03:00.000+01:00

Without a sense of caring, there can be no sense of community. - Anthony Burgess

Happy Birthday, LSCC!!!!

On 18th of August 2011, the London Software Craftsmanship Community completed one year. And what a great year we had!

How did it all start?

David Green and I had worked together in the past and we kept in touch since. Over one year ago, we were talking to each other about our jobs, pet projects, technologies, etc and we both realised that we couldn't discuss everything we wanted just with the people we knew. We were also aware of our own limitations. So we thought that it would be awesome if there was a group of people that we could meet regularly and share ideas about software development in general.

By then, I was very into the whole Software Craftsmanship thing. David was also very aware of everything that was going on, but we had slightly different views of craftsmanship as a movement. I remember having quite a few interesting conversations with him about that. However, we both always believed in the same principles and values regardless of the labels people were using.

I had been involved with the London Java Community (LJC) even before its existence. At some point I managed to bring David along to one of the LJC's social events. On that evening, David and I had a chat with Barry Cranford, founder of the LJC, and with Martijn Verburg and John Stevenson, LJC organisers and also involved with other user groups and open source communities. We mentioned to them our idea of having a regular meeting where any topic related to software development could be discussed. Maybe show and write some code, discuss design, compare approaches. Completely language agnostic. We also mentioned to them one thing or two about the Software Craftsmanship movement. They all said that we should go ahead and found our own community instead and offered us LJC's support and blessings. I don't think that any of us was really thinking about founding a community. We just wanted to attend this sort of meetings.

David was always a bit more organised than me. But me, I could barely organise a bbq for my own birthday, let alone run a community. We left the pub that night and while walking to the station we though that actually it could be a good idea. Maybe we should give it a go. Why not?

When I got home that night, I went to the meetup.com website and created the London Software Craftsmanship Community.

Summary of our first year

We had our first meeting on 6th of October, 2010 (video). David and I gave a introductory talk on software craftsmanship and for our surprise we had over 100 people subscribed. That was the first sign that our community would take off. Right after we decided to have a monthly meeting called Software Craftsmanship Round-table

The Round-table meetings

This is the meeting that we always wanted to have and that triggered the whole thing. In general they are limited to 25 people. We get together and attendees write the topics they want to discuss on the whiteboard. Any thing related to software development is valid. Then we take a vote. The most voted topics are discussed. Sometimes we discuss just one or two topics, sometimes we discuss up to five topics. No one needs to be an expert on the topic. It's not a presentation. It's just a friendly group discussion.

During the round-table meetings over the year, we had people showing code they were working on, pieces of design and architecture, testing frameworks, and had discussions about a huge amount of other topics. We had discussions about hiring good developers, distributed agile teams, mentors and apprentices, TDD, legacy code, patterns, PaaS comparisons, programming languages, android testing frameworks, the future of web applications, literate programming, DSLs, BDD frameworks, DDD, anti-patterns, specific technologies like Hamcrest, JUnit theories and Spring Roo and many many others that it would be impossible to describe all.

I absolutely love the roundtables and learned a lot from them. Our latest addition to it is a series of lightning talks at the beginning of the night, when people fancy giving one. Another cool thing is that every meeting is different, depending of who is attending, the topics proposed and the ones people voted on.

Other meetings

During the year we also had a few other great meetings. One was the Software Craftsmanship Panel Discussion - How can craftsmanship move the industry forwards? | video - when we had the pleasure to have Ade Oshineye, Chris Parsons, Dan North and Dave Hoover in the panel. It was great to see them giving their views about many topics relevant to the future of our industry. We had Jason Huggins, creator of Selenium, giving a talk to us about "how to test 'untestable' applications" | video.

We also cross-promoted the Code Retreat Winchester, organised by Despo and Aimee, and the first Cambridge Dojo Day, organised by Alastair Smith, organiser of the Cambridge User Group - CAMDUG.

Future meetings

We will be keeping the monthly Software Craftsmanship Round-table meetings on every second week of the month. From this month onwards, we will have a monthly hands-on session on the last week of the month. Our first one is the Crafting Object-Oriented Code, on the 30th of August. We will be opening the space to our members to submit proposals for any hands-on session they want run in the following months.

We are also very happy to be having LSCC's First Code Retreat on 10th of September.

Our meetings tend to be fully subscribed, on average, in 24 hours after they are announced. We had round-table meetings being booked up in just 5 hours in the past. So, if you want to attend our meetings, make sure you register as soon as you get the email notification.

Community

Throughout the year, we met many extremely passionate and talented people. I learned things that I was not even aware they existed. And that's what this is all about. That's exactly what we always wanted. We met people from all levels of expertise, different backgrounds, working in different industries and with different technologies. All kind enough to share what they know and humble enough to want to learn from others.

LSCC had a lot of support from LJC and we are committed to support other communities as well. With that in mind, we would like to offer our support to any person or group of people thinking to found a software craftsmanship community in the UK or even abroad. We are more than happy to cross-promote any event to our members - if we judge they are relevant to our community and to the software craftsmanship moment as a whole.

The future of LSCC

We have quite a few things in our heads but I don't want to spoil the surprise and neither make promises I may not live up to. But there are a few things I can say. Besides all the regular meetings (round-tables and hands-on sessions), we want to focus more on the Software Craftsmanship movement itself. There are quite a few principles and values that we need to start focusing on. We are already having discussions about some of them so watch this space.

Another thing I personally would like to see is more software craftsmanship communities around the UK and Europe. We will do the best we can to help and support new communities.

As we grow, we will need to do more but one thing is for sure. We will not compromise the quality of our meetings or things we want to do just to bring more people in.

A big thanks

I would like to thank our sponsors - YouDevise, SkillsMatter and Valtech - for their support, for offering their premisses so we can run our events for free and for helping us get fat with all the pizzas and drinks.

A big thanks to all the kindred spirits that support and promote LSCC and, last but not least, a massive thanks to all LSCC members for their dedication to their craft and for making LSCC a great community. You guys rock!

I'm really proud to be a software craftsman and I'm happy to be able to give something back to this amazing community of developers that gave me so much and that helped me to become the professional I am today. Thank you all.

SRP: Simplicity and Complexity

2011-07-26T02:47:00.000+01:00

Simplicity does not precede complexity, but follows it. - Alan Perlis

The Single Responsibility Principle (SRP) - one of the SOLID principles - is definitely one of my favourites principles in Object-Oriented Design. I find its simplicity and, at the same time, its complexity fascinating. I have known this principle for years and wrote about it for the first time over one year ago. Still, I can't stop thinking about how hard it is to apply it. Speaking to and pairing with many developers over the years, I noticed that every single one of them understands what the principle is about but when it comes to write code, either they forget it or they simply can't find a way to adhere to it. I'll give you my view about the problem and what I've seen happening in many occasions.

The outside view

Every class and method should have just one responsibility

This is what I call outside view and naming plays a big role here.

This view is generally used when we need to add some totally new code. We think what the new code is about, create a class and/or a method, give it an appropriate name that represents what the new code does and we are done. We also use this outside view when reading code. When looking at a class or method name, it should be clear to us what they do.

I don't think anyone ever had a problem understanding this principle.

However, if it is so simple to understand, why don't we all follow it? Why do we still find classes and methods with hundreds, if not thousands of lines? A few things come to my mind to explain that. Maybe we are just bad at naming classes and methods. Maybe the names we give are too generic or too broad, allowing us to write loads of code in there. Maybe some of us just don't care.

The inside view

Classes and methods should have one, and only one, reason to change

This is what I call inside view and it is the other (often forgotten) side of the Single Responsibility Principle.

This view is generally used when we need to change some existing code or add more code to it. That's when we look inside each class and method to see exactly what they do. More often than not, we get frustrated because they do much more than they should do and we need to go through many lines of frustration and WTFs to understand the bloody thing.

However, it is much easier to adhere to the SRP when we keep the inside view in mind instead of the outside view. That means, always think about the number of reasons to change a class or method instead of thinking purely in their responsibilities.

So, when and where do things go wrong?

NOTE: In general I find more SRP violations in systems where developers are not doing TDD and refactoring.

Here is some food for thought. I'm risking to say that the majority of SRP violations are found in the classes and methods that are closer to the interface of the system. For example, in a web application, it's very common to find large controllers, actions or backing bean classes. For swing applications, we tend to find large event handlers. For systems where the entry point is a queue, we tend to find large methods that are responsible for processing the messages.

In general, this happens because the classes and methods closer to the interface of the system have a more generic and broader responsibility. Quite often, these methods are responsible to trigger loads of business rules and/or complex workflows.

Imaging that the input to our system is a huge XML file representing a trade. Also imagine that the first method invoked to handle that is called "processTrade(tradeXML)", located in a TradeService class. What is the responsibility of this method? It is to process a trade, right? Should it have a different name? Our system is expected to "process" all trades received so it is fair to say that the first method handling the request (or reading from a queue) should be called processTrade.

In another example, imagine that an user added a few items to her shopping basket (web application), provided some payment details and clicked on the "place order" button. In our back end, we will have a method handling this request that probably will be called something like placeOrder(order). Fair, right?

Developing an idea

In general, the closer the code is to the system's interface, the broader and more generic its responsibility will be. The further away the code is from the system's interface, the narrower and more specific its responsibility will be.

In both examples above, by their names, you can argue that the methods processTrade and placeOrder have a single responsibility. One processes the incoming trade and the other places a customer order. So, when developers take into account just the outside view of the SRP, they feel comfortable to add as much code as they need to satisfy these responsibilities.

The problem is that processing a trade or placing an order may be extremely complicated tasks, where a huge workflow will be triggered, many business rules and requirements need to be satisfied and we will need to write hundreds, if not thousands, of lines for that. So clearly, adding all these lines to a single method doesn't just simply violates the SRP. It's also plain stupid.

So, in order to make our code compliant to the SRP, we need to have just a single reason to change it. This leads to a complementary idea.

In general, the closer the class is to the system's interface, the more delegation the class will do. The further way the class is from the system's interface, less delegation the class will do.

A general example would be, in a traditional Java web application, the controllers, that are closer to the UI, have a broad responsibility and tend to delegate all the business logic to other objects. They just control the flow. At the other end, we have the DAOs that have a very specific and narrow responsibility, almost never delegating anything to another class. In the middle, we have services, that do some business logic of their own but also delegate work to other collaborator classes. Services tend to have a narrower responsibility when compared to a controller but a broader responsibility when compared to DAOs. Still, each class and method have a single responsibility.

Asking a different question

When mentoring and/or pair-programming with other developers, quite often we end up discussing about the amount of code in a method or methods in a class. Just using the SRP's outside view as an argument sometimes is not enough to convince some developers that the code is doing too much. That's why I find the inside view more useful. Instead of asking how many responsibilities a method or class have, I now ask how many reasons we would have to change them.

Testing legacy: Hard-wired dependencies (part 2)

2011-07-18T02:12:00.000+01:00

In part one, I showed how to unit test a method that uses a Singleton and makes static calls. So now, let's have a look at common code problems we find in legacy code, using the same example:

How many problems can you see? Take your time before reading the ones I found.. :-)

Refactoring
NOTE: When I've done it, I've done it step by step running the tests after every step. Here I'll just summarise my decisions.

The first thing I noticed is that the tripList variable does not need to be created when the logged user is null, since an exception is thrown and nothing else happens. I've decided to invert the outer if and extract the guard clause.

Feature Envy

When a class gets data from another class in order to do some calculation or comparison on that data, quite often it means that the client class envies the other class. This is called Feature Envy (code smell) and it is a very common occurrence in long methods and is everywhere in legacy code. In OO, data and the operations on that data should be on the same object.

So, looking at the code above, clearly the whole thing about determining if an user is friends with another doesn't belong to the TripService class. Let's move it to the User class. First the unit test:

Now, let's move the code to the User class. Here we can use the Java collections API a bit better and remove the whole for loop and the isFriend flag all together.

After a few refactoring steps, here is the new code in the TripService

Right. This is already much better but it is still not good enough.

Layers and dependencies

Some of you may still be annoyed by the protected methods we created in part one in order to isolate dependencies and test the class. Changes like that are meant to be temporary, that means, they are done so we can unit test the whole method. Once we have tests covering the method, we can start doing our refactoring and thinking about the dependencies we could inject.

Many times we would think that we should just inject the dependency into the class. That sounds obvious. TripService should receive an instance of UserSession. Really?

TripService is a service. That means, it dwells in the service layer. UserSession knows about logged users and sessions. It probably talks to the MVC framework and/or HttpSession, etc. Should the TripService be dependant on this class (even if it was an interface instead of being a Singleton)? Probably the whole check if the user is logged in should be done by the controller or whatever the client class may be. In order NOT to change that much (for now) I'll make the TripService receive the logged user as a parameter and remove the dependency on the UserSession completely. I'll need to do some minor changes and clean up in the tests as well.

Naming

No, unfortunately we are not done yet. What does this code do anyway? Return trips from a friend. Looking at the name of the method and parameters, or even the class name, there is no way to know that. The word "friend" is no where to be seen in the TripService's public interface. We need to change that as well.

So here is the final code:

Better, isn't it? We still have the issue with the other protected method, with the TripDAO static call, etc. But I'll leave this last bit for another post on how to remove dependencies on static methods. I'll park my refactoring for now. We can't refactoring the entire system in one day, right? We still need to deliver some features. :-)

Conclusion

This was just a toy example and may not even make sense. However, it represents many of the problems we find when working with legacy (existing) code. It's amazing how many problems we can find in such a tiny piece of code. Now imagine all those classes and methods with hundreds, if not thousands of lines.

We need to keep refactoring our code mercilessly so we never get to a position where we don't understand it any more and the whole business starts slowing down because we cannot adjust the software quick enough.

Refactoring is not just about extracting methods or making a few tweaks in the logic. We need to think about the dependencies, the responsibilities that each class and method should have, the architectural layers, the design of our application and also the names we give to every class, method, parameter and variable. We should try to have the business domain expressed in the code.

We should treat our code base as if it was a big garden. If we want it to be pleasant and maintainable, we need to be constantly looking after it .

If you haven't read it yet, check the part one of this post. If you want to give this code a go or find more details about the implementation, check: https://github.com/sandromancuso/testing_legacy_code

Testing legacy: Hard-wired dependencies (part 1)

2011-07-17T00:42:00.000+01:00

When pairing with some developers, I've noticed that one of the reasons they are not unit testing existing code is because, quite often, they don't know how to overcome certain problems. The most common one is related to hard-wired dependencies - Singletons and static calls.

Let's look at this piece of code:

public List<Trip> getTripsByUser(User user) throws UserNotLoggedInException {
    List<Trip> tripList = new ArrayList<Trip>();
    User loggedUser = UserSession.getInstance().getLoggedUser();
    boolean isFriend = false;
    if (loggedUser != null) {
        for (User friend : user.getFriends()) {
            if (friend.equals(loggedUser)) {
                isFriend = true;
                break;
            }
        }
        if (isFriend) {
            tripList = TripDAO.findTripsByUser(user);
        }
        return tripList;
    } else {
        throw new UserNotLoggedInException();
    }
}

Horrendous, isn't it? The code above has loads of problems, but before we change it, we need to have it covered by tests.

There are two challenges when unit testing the method above. They are:

User loggedUser = UserSession.getInstance().getLoggedUser(); // Line 3  
   
tripList = TripDAO.findTripsByUser(user);                    // Line 13

As we know, unit tests should test just one class and not its dependencies. That means that we need to find a way to mock the Singleton and the static call. In general we do that injecting the dependencies, but we have a rule, remember?

We can't change any existing code if not covered by tests. The only exception is if we need to change the code to add unit tests, but in this case, just automated refactorings (via IDE) are allowed.

Besides that, many of the mocking frameworks are not be able to mock static methods anyway, so injecting the TripDAO would not solve the problem.

Overcoming the hard-dependencies problem

NOTE: In real life I would be writing tests first and making the change just when I needed but in order to keep the post short and focused I will not go step by step here .

First of all, let's isolate the Singleton dependency on it's own method. Let's make it protected as well. But wait, this need to be done via automated "extract method" refactoring. Select just the following piece of code on TripService.java:

UserSession.getInstance().getLoggedUser()

Go to your IDE's refactoring menu, choose extract method and give it a name. After this step, the code will look like that:

public class TripService {

    public List<Trip> getTripsByUser(User user) throws UserNotLoggedInException {
        ...
        User loggedUser = loggedUser();
        ...
    }

    protected User loggedUser() {
        return UserSession.getInstance().getLoggedUser();
    }
}

Doing the same thing for TripDAO.findTripsByUser(user), we will have:

public List<Trip> getTripsByUser(User user) throws UserNotLoggedInException {
    ...
    User loggedUser = loggedUser();
    ...
        if (isFriend) {
            tripList = findTripsByUser(user);
        }
    ...
}  
 
protected List<Trip> findTripsByUser(User user) {
    return TripDAO.findTripsByUser(user);
} 
 
protected User loggedUser() {
    return UserSession.getInstance().getLoggedUser();
}

In our test class, we can now extend the TripService class and override the protected methods we created, making them return whatever we need for our unit tests:

private TripService createTripService() {
    return new TripService() {
        @Override protected User loggedUser() {
            return loggedUser;
        }
        @Override protected List<Trip> findTripsByUser(User user) {
            return user.trips();
        }
    };
}

And this is it. Our TripService is now testable.

First we write all the tests we need to make sure the class/method is fully tested and all code branches are exercised. I use Eclipse's eclEmma plugin for that and I strongly recommend it. If you are not using Java and/or Eclipse, try to use a code coverage tool specific to your language/IDE while writing tests for existing code. It helps a lot.

So here is the my final test class:

public class TripServiceTest {
        
    private static final User UNUSED_USER = null;
    private static final User NON_LOGGED_USER = null;
    private User loggedUser = new User();
    private User targetUser = new User();
    private TripService tripService;

    @Before
    public void initialise() {
        tripService  = createTripService();
    } 
        
    @Test(expected=UserNotLoggedInException.class) public void 
    shouldThrowExceptionWhenUserIsNotLoggedIn() throws Exception {
        loggedUser = NON_LOGGED_USER;
                 
        tripService.getTripsByUser(UNUSED_USER);
    }
        
    @Test public void 
    shouldNotReturnTripsWhenLoggedUserIsNotAFriend() throws Exception {             
        List<Trip> trips = tripService.getTripsByUser(targetUser);
                 
        assertThat(trips.size(), is(equalTo(0)));
    }
        
    @Test public void 
    shouldReturnTripsWhenLoggedUserIsAFriend() throws Exception {
        User john = anUser().friendsWith(loggedUser)
                            .withTrips(new Trip(), new Trip())
                            .build();
                 
        List<Trip> trips = tripService.getTripsByUser(john);
                 
        assertThat(trips, is(equalTo(john.trips())));
    }

    private TripService createTripService() {
        return new TripService() {
            @Override protected User loggedUser() {
                return loggedUser;
            }
            @Override protected List<Trip> findTripsByUser(User user) {
                return user.trips();
            }
        };
    }        
}

Are we done?

Of course not. We still need to refactor the TripService class. Check the part two of this post.

If you want to give it a go, here is the full code: https://github.com/sandromancuso/testing_legacy_code

Working with legacy code

2011-07-03T23:37:00.000+01:00

Context

Large organisations' systems may have from tens of thousands to a few million lines of code and a good part of those lines is legacy code. By legacy code I mean code without tests. Many of these systems started being written many years ago, before the existence of cool things and frameworks we take for granted today. Due to how some systems are configured (database, properties file, proprietary xml) , we cannot simply change a class constructor or method signature to pass in the dependencies without undertaking a much larger refactoring. Changing one piece of code can break completely unknown parts of the system. Developers are not comfortable in making changes in certain areas of the system. Test-first and unit testing is not widely used by developers.

The Rule

In our commitment to make the existing systems better, more reliable, easier to deal with (changing and adding more features), we established the following rule: We can not change existing code if it is not covered by tests. In case we need to change the existing code to be able to write the tests, we should do it only using the automated refactoring tools provided by our IDEs. Eclipse and IntelliJ are great for that, if you are a Java developer like me.

That means, before we make any change, we need to have the current code under test, guaranteeing that we don't break its current behaviour. Once the existing code is under test, we then write a new test for the new feature (or change an existing test if it's a change in existing behaviour) and finally we are can change the code.

Problem

There is an eternal discussion on forums, mailing lists and user groups about TDD being responsible for the increase or decrease of the team's velocity. I don't want to start this discussion here because we are not doing TDD for majority of the time. With legacy code, we spend the majority of our time trying to test existing code before we can do TDD to satisfy the new requirement. And this is slow. Very slow. It can be, in our experience, somewhere between 5 to 20 times slower than just making the change we need and manually test it.

You may be asking, why does it take so much longer? My answer is: If you haven't, try unit testing all the possible execution paths in a 2000+ line class? Have you tried to do it with a 1000 line method, full of multiple return statements, hard-wired dependencies, loads of duplication, a big chain of nested IFs and loops, etc? Believe me, it will take you a bloody long time. Remember, you can't change the existing code before writing tests. Just automated re-factorings. Safety first. :-)

Why are we "under-performing"?

Quite often, after start following this rule, management will enquire why the team is under-performing. Although it is a fact that the team is taking longer to implement a feature, this is also a very naive view of the problem. When management and development teams talk about team's velocity, they just take into consideration the amount of time taken to write the code related to a few specific requirements.

Over the time, management and teams just forget how much time they spend in things that are caused by the lack of quality and tests in their code. They never take into consideration that there is absolutely no way that developers would be able to manually test the entire system and guarantee they didn't break anything. That would take days, for every piece of code they write or change. So they leave that to QA.

By not increasing the test coverage, the team will never have a regression test suite. Manually testing the system, as it grows in size and complexity, will start taking longer and longer by the QA team. It will also be more error prone, since testing scripts need to be kept in sync with the current behaviour of the system. This is also waste of time.

Also, just hacking code all the time will make the system more fragile and way more complicated to understand, which will make developers to take longer to hack more code in.

Because we can't press a button and just unit test a specific piece of code, the team is forced to spend a lot of time debugging the application, meaning that the team needs to spend time compiling and deploying it. In certain systems, in order to test a small piece of code, you need to rely on other systems to trigger something so your system can respond. Or you need to manually send messages to a queue so your piece of code is invoked. Sometimes, depending of the complexity of your system and it's dependencies, you can't even run it locally, meaning that you need to copy your application to another environment (smoke, UAT, or whatever name you use in your company) in order to test the small piece of code you wrote or changed. And when you finally manage to do all that to execute that line of code you just added, you realise that the code is wrong. Sigh. Let's start over.

Since I started practising TDD, I realised that I very rarely use my debugger and very rarely have to deploy and run my application in order to test it.

So basically, when people think we are spending too much time to write a feature because we were writing tests for the existing code first, they are rarely considering the time spend elsewhere. I've seen it many times: "We don't have time to do it now. Let's just do a quick fix.". And then, contradicting the laws of physics, more time is created when bugs are found and the QA phase needs to be extended. Panic takes over: "Oh, we need to abort the release. We can't go live like that.". Exactly.

The graphic above is not based in any formal research. It is purely based on my own experience during my career. I also believe that the decrease in the team's velocity is related to the state of the untested code. There are untested code out there that is very well written and writing tests to it is quite straightforward. However, in my experience, this would be an exception.

In summary, if you are constantly applying the "Boy Scout Rule" and always keep improving the code you need to touch, over the time you will be making your code better and you will be going faster and easier. If you don't do that, you will have a false impression of good velocity at start but gradually and quite often without noticing, you will start to slow down. Things that used to take a few days, now take a few weeks. Because this lost of velocity happened slowly (over months or even years), no one really noticed until it was too late and everything now take months.

Drifting away and losing focus

One interesting side effect of constantly making legacy code better, writing tests and refactoring, is that it is too easy to be carried away. More than once I've seen pairs, while trying to clean a piece of code, drifting away from the task/story they were working on. That also happened to me quite a few times. We get carried away when making progress and making things better.

Very recently I was asked by one of the developers: "When do we stop?". My answer was: "When we finish the task.". In that context, that meant that the focus is always on task at hand. We need to keep delivering what was agreed with our product owners (or whoever the person in charge of the requirements are). So, make the system better but always keep the focus on the task. Don't go crazy with the refactoring trying to re-write the whole system in a few days. Do enough to finish the task and try to isolate the parts that are still not good enough. Baby steps.

A more positive side effect

Another very common problem in legacy systems is the lack of understanding about the system and also about the business. Quite often we find situations where the developers and business people are long gone and no one really knows how the system behave.

Writing tests for the existing code force us to understand what it does. We try to mine some hidden business concepts in the middle of the procedural code and try to make them very explicit when naming our tests. That is a great way to drive our refactoring later own, using the business concepts captured by the tests. Quite often, we also get the business people involved, asking them questions and checking if certain assumptions make sense.

In writing the tests and refactoring the code, the previously completely unknown behaviour of the system is now well documented by the tests and code.

Quality is a long term investment

In a recent conversation, we were discussing how we could measure our "investment" in quality. A quick and simple answer would be that we could measure that by the number of bugs and the velocity that the teams are delivering new requirements. However, this is never too simple. The question is, why do you think you need quality? Which problems do you have today that makes you think they are quality related? What does quality mean anyway?

When working with legacy, after many years, we tend to forget how things were in the past. We just "accept" that things take a long time to be done because they have been taking a long time to be done for a long time. We just accept a (long) QA phase. It's part of the process, isn't it? And this is just wrong.

Constantly increasing the quality level in a legacy system can make a massive difference in the amount of time and money spend on that project. That's what we are pursuing. We want to deliver more, faster and with no bugs.

It's up to us, professional software developers, to revert this situation showing a great attitude and respect for our customers. We should never allow ourselves to get into a situation where our velocity and quality of our work decreases because of our own incompetence.

A change in attitude - Legacy code

2011-06-21T03:31:00.000+01:00

Attitude is a little thing that makes a big difference. ~Winston Churchill

Not long ago, I gave a talk about Software Craftsmanship where I asked who liked to work on greenfield projects. Almost everyone raised their hands. Then I asked who liked to work with legacy code. Besides one or two friends that were there, almost everyone kept their hands down.

It is always nice to start a project from scratch, be able to choose the technology, use the latest frameworks and have a lot of fun writing code without worrying about breaking existing features or having to understand existing code. Working on greenfield is great, mainly with an experienced and disciplined team using TDD since day one. Progress flows naturally and quickly.
But as soon as we are working with code written by people that are long gone, no tests, no documentation, we go mental. We notice we are going mental by the number of WTF we say during the day. We may get moody and start hating to work on a specific system or in parts of it. Frustration becomes a constant.

A change in attitude

If you don't like something change it; if you can't change it, change the way you think about it. ~Mary Engelbreit

Although I wrote "we" and "us", that was really how I used to feel when working with legacy code. However, in the past few years, I learned many things. An obvious one is that moaning, complaining and cursing won't make my life easier or better. If I want things to be better I need to do something about it.

Today, when I look at legacy code, instead of moaning and getting frustrated, my attitude is to try to understand it and make it better, constantly applying the Boy Scout Rule.
As my friend David Green said in a twitter conversation, what I agree 100%, improving and understanding legacy code can be massively rewarding. The process of trying to make sense of a big ball of mud seems daunting but if we just focus in small parts of it, one at a time, and start improving them (writing tests, extracting methods and classes, renaming variables, etc), bit by bit, things become much easier and enjoyable.

Working with legacy code is almost like solving a big jigsaw puzzle. We don't do that all at once. We start by separating the pieces into groups, often starting with the edges and corners and then separating other small pieces by colour, pattern, etc. We now have a few (logical) groups and we start to form a higher level model in our head. What before was a bunch of random pieces (or a big ball of mud), now is a bunch of smaller groups of random pieces. Still not very helpful or encouraging, but nonetheless some progress. Bit by bit, we start working on one of these groups (parts of the code) and we start putting some pieces together (writing tests for the existing code, which will help with our understanding of the code, and refactoring it).

Once we start putting some pieces together, we start seeing a small part of our jigsaw puzzle picture. We get excited because now it's getting real. It's starting to make sense and we are happy we are making progress. The more pieces we put together the more excited we are about finishing the jigsaw puzzle. The more pieces we put together, the easier it gets to put more pieces together. And that's exactly the feeling I've got when working with legacy code now. For every piece of code I make better, more I want to make the whole code better. The feeling of achievement is really rewarding. What before was something I could barely read and took ages to understand, now reads like a story and every developer can understand it with no effort at all. Maybe when a good part of the code is stable (covered by tests, loosely coupled, well defined responsibilities, etc), I could even introduce the cool and new frameworks that I always wanted to use. I could upgrade some library versions. I could even throw a lot of code away and replace it with a framework because now my code is clean and modularised and replacing one part of the system will not break another. I don't even need to envy my friends working on greenfield projects any more.

Different challenges

Another cool thing about legacy code is that it forces you to think in a different way. In greenfield projects, when developing a new feature, we write a test and start implementing stuff. In legacy, sometimes you can't even instantiate a class in order to test it due to all its dependencies. A change in one place can cause an impact in obscure parts of the system. We have an option here. We can see these challenges as a pain in the ass or can be see them as very interesting problems to solve. I prefer the latter.

The professional side

Although it is OK to have fun and enjoy what we are doing, we always need to remember that we are professionals and being paid to write software. Software is an asset and a lot of money and time is invested on it. As every investment, clients want to maximise the return of their investment. The more we improve and keep the software clean, the longer the client will be able to benefit from it. Stretching the lifespan of a software also maximises the client's return of investment.

Conclusion

At the end of the day, it does not matter if you are working on a green or brownfield project. It's your attitude towards your job that will make it more or less enjoyable.

Re-drawing my own map: A new milestone

2011-05-17T04:38:00.000+01:00

For every step you take towards mastery, your destination moves two steps further away. Embrace mastery as a lifelong endeavour. Learn to love the journey.
--George Leonard, Mastery

It is with a mixture of sadness and excitement that I would like to announce that, after over five years, I'm leaving Valtech on the 18th of May. The decision was not an easy one and took me months to figure out what the next step in my long road would be. As I said in a recent interview, I love working for consultancy companies and that's the main reason I spent over ten years (two-thirds of my career) working as a consultant.

Working for Valtech

Valtech is a fantastic company to work for and had a huge impact in my personal and professional life. During my time there I had the opportunity to work on a great variety of projects, different companies, different industries and different technologies. Most importantly, I had the opportunity to meet and work with a lot of great people that helped me to become a much better professional.

If there is one thing that I will never complain about Valtech is that I did not have recognition for the work I've done. I have started at a Valtech in a relatively junior role, according to Valtech's grade scheme, and bit by bit, with a lot of support and trust from my colleagues I was given more and more responsibility and gradually climbed my way up to one of the most senior positions.

There is no such a thing as a perfect company but if I had to point out the best thing about Valtech, I would say that, beyond the shadow of a doubt, it is its people. People that I learnt a lot from, that helped me to feel at home in the UK, that helped me with personal issues, people that challenged me, that pushed me to my limits, that gave me constructive criticism, that trusted me and empowered me to do my job, people that helped me to be better.

I may not be in the office or in a client site full time any more but I will always be around. As it happened before (I took a year off to work on a startup idea and came back) I'll always be around for drinks and events.

Special thanks

I would like to thank you everyone at Valtech that had to put up with me for all this time. I know I can be a pain in the neck ~~quite often~~ sometimes. :-)

It's always tricky to mention names since there is always the risk of people left out feeling a bit upset but I feel that I need to thank some people in a more personal level, so apologies for the ones I did not mention. In no particular order: Toby Mckenzie for caring about me and every single consultant during tough times, all his herculean effort to keep every one happy, finding us good projects; David Draper for challenging many of my beliefs and opinions, for the many advices, for supporting my involvement with user groups and events and for the effort in making Valtech a place of excellence; Mashooq Badar for the fantastic time we had together in many projects, for all the things I learnt from him and for making me open my mind about so many things. Ah, and for all the Blazing hot! moments; Phil Hall for the support and keeping the doors open to me and LSCC; Kevin Harkin (I can't believe he does not use twitter) for the fun we had together, for his friendship, advices, ranting sessions and memorable nights at the pub; Andrew Rendell for his professionalism, trust and for being a great role model for every Valtech consultant; And last but not least, Akbar Zamir, for pushing me and challenging me to be better, for all the advice, trust, knowledge and help, for being a great career manager and most important of all, for being a great friend.

Thanks for the great five years that I spent there. It was an absolute pleasure to work with all of you and be part of this great company.

The future

It’s not just a question of conquering a summit previously unknown, but of tracing, step by step, a new pathway to it.
--Gustav Mahler, musician and composer

I'm joining UBS as a senior developer at a director level, starting on the 23rd of May. Due to my involvement with the software craftsmanship movement, this came as a surprise to many people, including myself, mainly because investment banks tend to be almost a hostile environment for agile and software craftsmanship initiatives. When I started re-drawing my own map, investment banks were an avenue that I was not considering to explore.

As I said before, choosing my next step was not easy. I had a few things in mind that were non-negotiable: I wanted different challenges, that means, things that I haven't done before, keep having fun and loving my job, a potential long term commitment where I would have time enough to put into practice many ideas and beliefs and most importantly, have a long term career as a software developer but with a lot of space to keep growing as a professional.

I was fortunate to have had many opportunities during this time but the majority of them could not satisfy all the items above. I was determined to keep doing what I had been doing throughout my entire professional life that is just to work for companies that I really want to work for, I mean, companies that would be able to offer me what I was looking for at that point in time. For me, that's the best way to keep fuelling the passion that I have for what I do. Joining a company just because of money is and always has been totally out of question.

UBS came along with a very interesting proposition. They want to improve the quality of their software and recognise that agile and software craftsmanship are a great way to get there. They were interested in people with no previous investment bank experience, what is very unusual for an investment bank. They want people that can think different, that are passionate and can help them drive this transformation. I had five interviews and was very pleased to see so many people striving to be and do better.

As far as I understand, my main role will be to work as a hands-on developer, embedded in a team, helping to improve quality, leading by example and mentoring other developers. They also expect me to give internal talks, training, promote events, disseminate passion and promote the craftsmanship values and techniques. In the future I'll be working with other teams in the UK and in other offices around the world. But make no mistake. I'll have a hell of interesting and tough challenges ahead of me and I hope to live up to all their expectations.

Besides that, I'll keep running the London Software Craftsmanship Community (LSCC) alongside my friend David Green and try my best to give something back to the wider and great community of software developers out there that some many times I benefited from.

Thanks everyone for being part of long road journey.

Open-source developers deserve respect

2011-05-05T01:52:00.000+01:00

I was recently reading Gojko Adzic's blog post called How is it even possible for code to be this bad? I must admit that I was very sad to see tremendous lack of respect towards the Hudson/Jenkins community and towards open source software developers in general.

Firstly I would like to say that any person out there that is working on an open source project deserves a lot of respect, mainly the ones working on projects that bring so many benefits to so many companies and developers around the world. The velocity that our industry moves forward and evolves is, in general, because of many open source initiatives. It's because of thousands of developers that work on the their spare time, for free, to create software that will make the lives of many other developers, companies and users much easier. These people are kind enough to offer their work to all of us and are humble enough to ask and accept help from many other developers around the globe.

One thing we need to understand is that every project has a history and many things, throughout the lifespan of a project, can be responsible for the success or failure of a project. It's always easier to criticise something instead of trying to help and make it better. As far as I'm aware, I don't think that anyone in the Hudson/Jenkins community ever claimed that their code base is the best example of quality software ever written and we all should learn from them. If that was the case, probably it would justify such harsh words written in the original blog post.

The only acceptable form of criticism towards any open source project or developer is constructive criticism.

If we think that a open source project is below standard and/or not good enough, we should either not use it or we should contribute to make it better.

However, I do understand the point Gojko is trying to make. Yes, I agree that good code is not the only thing that makes a project be successful. I think we all could name a few projects that we delivered where clients and/or users were relatively happy but the code base was a bit messy. There are always exceptions to the rules. Trying to get our code as perfect and as clean as possible does not guarantee that our projects will be successful and having a messy code does not always mean that our project will fail either. We all know that. Even my 5 month-old baby girl knows that.

However, I totally disagree with Gojko's statement that "[Hudson's success with "bad" code] is close to the final proof that God doesn't exist for the whole craftsmanship debate". It is almost like saying that Agile is rubbish and everyone should forget about it just because some waterfall projects succeeded. It is like saying that we all should forget about good principles of software development just because some projects succeeded with messy code. In summary, is like trying to get some exceptions and transform them into rules. There are many ways and many things that can contribute to the success of a project. Software craftsmanship is one of them and a very important one. In a software project, the most important thing is the software itself and looking after its quality is an obligation of every developer involved in it.

Although software craftsmanship on its own may not be enough to guarantee the success of a project, the lack of it can be reason for its failure. Single-handed.

Another important point we should ask is the definition of a "successful project" and in which context or environment but I'll leave that for another post.

If exceptions invalidate rules and that is the way we should look at things, does God exist for anything in software development?

Working for consultancy companies

2011-04-17T23:32:00.000+01:00

Recently I was approached by the guys from the Graduate Developer Community (GDC) where they asked me for an interview about life in a consultancy company. Their idea, that came out from an email I sent to the London Java Community (LJC) ages ago about different career paths, was to interview professionals that followed different career paths and publish these interviews on a website called GDC Careers so that graduates and people starting in our industry could have an idea about what is available out there in terms of jobs and career paths.

The interview was originally published on the GDC Careers but I'll also be publishing it here in full. I hope it helps any professional that would like to know more about life on consultancy companies.

NOTE: Throughout my career I've worked for three consultancy companies and this is just my general view of all of them. Each project is a different project and different consultants, working for the same consultancy company but in different projects, can have a completely different view of their employers.

Sandro Mancuso has been working as a software developer since 1996 but started writing code for pure enjoyment way before that. Although he has worked for software houses and startups, he spent the majority of his career working for international consultancy companies where he had the opportunity to work on a great variety of projects and across many different industries. He has a BSc in Computer Science and a MSc in Distributed Objects. He is a Co-founder of the London Software Craftsmanship Community (LSCC).

Title – What is your job title?

Senior Consultant.

What is your role about?

My role is to work with clients, helping them to identify what they really need, advise them, provide options, and help them to achieve whatever they want to achieve in the most efficient way.

OK, I know. That was not very specific or helpful. This is because the role of a consultant can vary quite a lot. Different consultants may choose different career paths. Some follow a more business oriented career, advising clients on strategy, marketing, investment, finance, etc. Some follow a more process oriented career, emphasising project management, Agile Coaching, training, business analysis, requirements, etc. Others prefer a more technical career path, working more often as developers, technical architects and team leaders, mentoring, etc. In my case, I took the more technical path so normally I do what I like doing, that is writing code, regardless of my position in the project. Although I was a hands-on developer on all my assignments, many of them had elements of mentoring. Some times I also got involved in customised training courses for clients, like TDD, Agile, etc.

I work mainly in Agile Java projects. Technologies vary quite a lot since every now and again we are working for different clients, different industries and completely different systems. Sometimes clients already have something in place and the technology stack is already defined. In this case, we often use whatever they are using to start with but we always try to make suggestions if we see that other technologies or frameworks should be used instead. Some other projects, mainly greenfield ones, we generally have freedom to choose the technology stack we judge is the best for the problem.

What are the best/most positive parts of the job/industry?

In my view, the best thing about being a consultant is the amount of technologies and business domains that you are exposed to during your career. Because of the frequency that we change projects, we end up being exposed to a lot of interesting problems quite early in our careers, giving us a great understanding of many different software projects.

Networking is definitely another very positive aspect of being a consultant. As we move from project to project, client to client, we end up meeting a lot of people, in different organisations, what can be very useful as you progress in your career. The more people you work with or for, the more options you will have in the future.

Another important point is that, as a consultant, you need to be sellable. This may sound a bit negative, but if fact, what it means is that you need to be at the top of your game, knowing the latest technologies, methodologies and trends. The more you know, the easier it will be for a consultancy to place you in a project (sell you to a client). As the consultancy companies recognise that, in general they have different mechanisms (training, conferences, internal talks, events, etc) that you can take advantage of to improve your skills.

Because your peers are also moving from project to project and using different technologies in each project, it is amazing what you can learn from them. The variety of information shared among consultants about projects, businesses, methodologies, technologies, etc, is absolutely priceless. There is always someone doing something different somewhere.

What are the negative parts to the job/industry?

The main complain that consultants have is that in general you don’t have much saying on where you are going to be sent to. Once you decide you want to work for a consultancy company, you need to be aware that travelling is always a possibility. A few lucky consultants get to stay for a long time on projects close to their homes but in general this is not the case. If travelling or long commute is not an option for you, you really need to think hard before becoming a consultant. For those living in big cities, like London for example, in general this is not a big problem since the majority of the clients will be there anyway.

According to your level of seniority, years in the company and history of projects you delivered, you can refuse to go to some projects, in case you don’t like the project (less often) or if the commute is bad (more often). However, this is a card that needs to be played very wisely since you don’t get to play it more than once in a short period of time.

In some projects, you may be seen as an outsider by a few permanent employees. Some feel threatened by the presence of a “consultant”, since you may be “exposing” problems in the current process and software. It’s part of our job to deal with that and do whatever we can to be seen as another team member, a person that shares the same goals, and not an outsider.

Career Path

What is the standard career path/qualifications?

This varies quite a lot since consultancies may offer different type of services and they will need people that are more specialised in some areas than in others. Services offered may include software development, project management, training, coaching, business advice (strategy, finance, investment, etc), auditing, etc. Some consultancies tend to work more on the “consulting” side of things and tend to engage with clients at a more senior level (CEOs, CTOs, Directors, etc). Other consultancies are more focused on software delivery and tend to engage with clients at lower levels (directors, project managers, head of departments, etc). And, of course, some consultancies do a bit of both.

I’ll just talk about consultancies that focus more on software delivery since that’s my background.

My title is Senior Consultant but this is a reasonably vague title. Many consultancies, internally, distinguish their consultants by grades. Each grade has different ranges of salaries and responsibilities and we take on responsibilities on projects, in general, according to our grade.

For people aspiring a more technical career, in the entry levels you are expected to have a good knowledge of a programming language (in our case Java or .Net), most common frameworks, good understanding of Object-Oriented Programming and you would be working as a team member (developer). As you move up through the grades, you may take roles where you will be leading teams, be the technical architect, enterprise architect, Agile coach, etc. At the higher grades, besides all the technical knowledge you will also have some extra responsibilities like writing technical proposals, pre-sales, project inceptions, project management, manage clients expectation, etc. However, some consultants manage to reach the higher grades and remain totally hands on. That’s the path I’m following.

To be successful as a consultant, you need to be a well-rounded professional. Just knowing a programming language well is far from being enough to be a good consultant. You are expected to have knowledge in many different areas since you never know in which project you will end up and which position in the project you will have. And regardless what the project is, you need to be ready. Of course that we all know that is simply impossible to know everything that is out there but at least you need to show that you have the motivation and drive to learn whatever is thrown at you.

Besides the technical skills, another thing that is very important for a good consultant is soft skills, due to the amount of iteration with clients. I’m still working to improve mine. :)

What are the prospects?

Over the years, after many different projects, a consultant will have a good understanding of different types of software projects, technologies and also will have met a lot of people working for many different companies. This gives the consultant a good variety of options for the future.

In your experience are you aware of any differences your role has between industries/sectors?

In terms of clients we work for, yes, there are huge differences. Working for clients where software is not their core business (a big supermarket chain, a media company, etc) is totally different from working for clients where software is their core business or is an extremely important part of it (software products, internet companies, telcos, etc). There’s also the investment banks, financial companies, government, gambling, etc. Each client will demand a different type of service and a different type of expertise. Some clients will ask a consultancy to help creating the process, roles and responsibilities, requirements, definite the technology stack, etc. Full freedom. Other clients, which have a more mature software capability, may ask a consultancy to provide help with a very specific problem. Sometimes they just need some good developers for a limited period of time. They may want someone in there to coach and mentor their own developers. That means, the role can vary quite a lot, depending on the client.

Reflection and The Future

What was it like coming into the industry?

Before joining my first consultancy company, I had worked for two small software houses and I didn’t know much about it. Job was announced in one of the main Brazilian newspapers. 800+ developers applied and just 34 were hired after a month long selection process, 4 phases, including a two week training and group dynamics. My two previous companies combined had in total 10 people. This consultancy company had more than one thousand employees just in Sao Paulo. Many thousands around the world. I was completely lost and a bit scared by the size of the projects. Before I was writing software for local shops and businesses and there I was working in projects for multinational companies and governments. Complete shock. New technologies, new people, massive projects, multiple teams involved, clients speaking different languages, etc. After a few months, I was quite comfortable there and knew that I had made the right choice. I guess that I was very lucky there, firstly because I was assigned to work on a very technical and capable team and secondly because for two and a half years I had the best mentor (my boss) that I could ever have asked for.

However, joining a modern and more agile consultancy company today is totally different, much easier and less traumatic, even when working on big projects for big clients. In general you will be working in smaller teams (even on long term projects) and following agile methodologies. The technologies used are accessible to every one. When I started, back in the 90ies, we couldn’t just go to a website, download certain technologies and install on our PCs. Many of the technologies were proprietary. There’s absolutely nothing to fear about joining a consultancy company today and the world is a much better place now.

Do you have any thoughts on the future of your role/industry?

I think that while there is a need for software development, consultancies and consultants will always have a place. However, the time where consultants were the only or the best specialists in the market are over. Due to the internet, open source projects, social networking, user groups, conferences, shared videos, podcasts, etc, any one, anywhere in the world, has access to the latest technology at the minute it is released. Consultants now, more than ever, need to be at the top of their game. Consultancy companies will need to do their best to get the best professionals in the market in order to survive. They will need, more than ever, to show to clients that they have the best people to work on their projects.

What advice would you give someone entering your industry?

The following advices are valid for any developer starting his or her career, regardless if he wants to be a consultant, the industry or career path he will choose:

- Firstly and most importantly, enjoy and be proud of what you do. Software development is a very cool profession. During your career, there will be many people out there that will be using and benefiting from stuff you created. And this is definitely really cool.

- If you think you studied hard during university, think again. You haven’t even started studying hard yet. Software development IS NOT a 9 to 5 profession. Be prepared to study hard, outside working hours, for the rest of your life.

- Always aim for jobs where you are going to learn more and not earn more. Preferably a place where you could have a good mentor. Be patient because money will come. If you are good and keep learning, you will have a very decent life throughout your career. If you just aim for the money instead of learning, you may get a good money now and be unemployable and out of the market in a few years time.

- Put a lot of effort on learning the fundamentals of software development instead of just learning the latest framework. If you have a good foundation, you will be able to learn any new technology much faster.

- Contribute to open source projects, learn from their code base and take the opportunity to communicate with other contributors.

- Always have a pet project (you may be excused if you are actively contributing to an open source project). Pet projects are great for you to try different technologies, techniques and methodologies. The cool thing about a pet project is that you have the power to do whatever you want, whenever you want, making it quite fun to work on. You don’t need to finish a pet project, in fact, you never will.

- Join you local user groups. The amount of stuff you learn and people you meet is just priceless.

Have you come across anything or anyone that has helped you move forward in the industry?

Definitely the mentor I had early on in my career. He was an exceptional developer and like a father for all of us in the team. He could hit us as hard as he wanted but no one else was allowed to do it. He shielded his team (us) from all the external heat and pressure. Quality was non-negotiable. He was a true believer that “how it’s done is as important as having it done“, phrase that I use as my blog’s subtitle. He believed in the apprenticeship model, although we’ve never used the term. It was there and then that I learned the true values of what today is called software craftsmanship. I worked for him and on that team for two and a half years and by far it was, if not the best, one of the best teams that I've been part of. My mentor gave me everything I needed to become the professional I am today.

Twitter: @sandromancuso
blog: http://craftedsw.blogspot.com
LSCC: http://www.londonswcraft.com

Frustrations and aspirations of a software craftsman

2011-04-02T20:27:00.000+01:00

For a while I've been thinking about what makes me like or dislike a project. Having spent a very big part of my career working for consultancy companies, I was exposed to many different environments, industries, team sizes, processes and technologies. There were projects that I absolutely loved, some projects were OK and some were a real pain.

There were even a couple of times in my career where I questioned myself if the choice of being a software craftsman and keep walking the long road would be the best thing for my sanity.

What makes me dislike a project?

Well, there are many factors. Here are just a few but is far from being a complete list:

Bureaucracy is something that can be really frustrating. That includes process for the sake of process, innumerable cycles of approvals, tortuous and long test and deployment cycles, pointless documentation, and all that anti-agile stuff.
Old technologies or the "wrong" technology for the job is always demotivating. We love new toys. There is nothing more annoying when the technology stack is imposed on the team. "You must use these tools from Oracle or IBM. But, hey, don't look like that. You have support if you need it."
Lack of autonomy and credibility. "You are just the developers here. You don't make decisions. You do what you are told to do. There are much smarter people here to worry about the _real_ problems. And by the way, you don't have admin rights to your PC and you can't access a few websites either."
Uninteresting domain. It's always difficult to find motivation to build a great software if you don't like what the software does or don't really believe in the business idea.
Demotivated people. How can we find motivation and have team spirit when your colleagues attitude is: "Oh, I just turn up to work, keep my head down and do what I'm told. If something goes wrong, it's not my fault."
Finger pointing and highly competitive environment, where no one plays as a team. This is an environment where everyone wants to be the boss, they are always looking for a scapegoat and the less work they do, the more they delegate, the better. If something goes wrong, it would never be their fault. If something goes well, they take all the credit.
Arrogant and unskilled people. Arrogance many times is used as a self-defence mechanism in order to hide the lack of skills a person may have. "I don't need to read any books. I think all these new technologies and methodologies are crap. I've been doing this for years. I know what it is best."
Software factory concept. "We need to go faster. Let's throw more developers here. Which ones? Doesn't matter. Just hire some monkeys."
Mortgage-Driven Development
Project managers that think they are the most important member of the team
Very deep hierarchy
You can't help those who don't want to be helped.
I really could go on forever here....

So what is really the problem here?

When I first mentally thought about all these items, I realised that almost all the things that make me dislike a project are related to people. Yes, people. One of the few exceptions are uninteresting domain and old technologies. So even if I make sure that I don't work on projects related to subjects I have no interest and that use the latest technologies, the people involved may make it very frustrating. Have you ever been on a project where you thought that the project had everything to be a great project but for some reason it was a pain?

After this analysis, things were not looking good, so I decided to look at all the projects I really enjoyed. It was when I realised that in some of them we didn't use the latest technologies and in a few of them I was not exactly passionate about the domain either. One or two were even quite bureaucratic. So, why did I enjoy them? What was in there that made me put them among the projects I liked the most? Once again, the answer was "people".

The good projects and what I always would like to find

My favourite projects had quite a few things in common but the most important ones were passion, craftsmanship, friendship and trust.

Passion

It's not because you like something that you are going to be good at it. However, to be really good at something, you must have a passion for it.

The best people for a job are the ones that love the job. This in the essential quality that drives people to be successful in whatever they decide to do. A passionate person will do whatever is in his or her power to keep acquiring skills and do the best they can. Passionate people bring innovation, they question what they are doing, they want to contribute, they want to get involved, they want to learn. They want to succeed. Passionate people CARE.

Craftsmanship

The only way to go fast is to go well - Uncle Bob Martin

In all my favourite projects, the focus on quality and the willingness to see users satisfied and using the system was always a big thing. The whole team was focused in delivering the best project we could, taking into consideration all the constraints we were under. It was clear to all of us that to be successful we had to be pragmatic. We also always believed that how it is done is as important as having it done.
Software craftsmen use the right tools for the job, are skilful, are pragmatic, care about the quality of their work, care about their reputation and want to delight every single one of their customers and users. Software craftsmen CARE.

Friendship

So far I've been talking about passionate people and care. However we know that people have different opinions and there are many ways to do the same thing. Now imagine a room full of very passionate people that don't really get along. In the best scenario, there will be some memorable arguments. In the worst, they will kill each other.

Friendship is the answer to that. The importance of social events for a team is enormous, even if it is just a few drinks once or twice a month. Like it or not, we spend more time with our colleagues than with our own family, so it is important that we have the friendliest environment possible. Having lunch together at least a few times per week is another thing that can help improve this friendship. Team members need some time together where they are not always just talking about work. Team members need time to know each other.

Among friends people feel comfortable to speak their minds. Working with friends helps to improve the quality of the discussions and no one is worried to expose ignorance or give suggestions. Friends help each other, friends learn from each other, friends CARE for each other.

Trust

Once people can demonstrate their commitment and passion, can demonstrate competence, willingness to learn and contribute and are able to deliver the best software possible, trust is easily established. With trust you gain autonomy and are free to decide what it is best for the project and to do your job well. With trust you can be more effective when delegating or sharing tasks. With trust we can remove all the bureaucracy that impedes that we do our job efficiently.

Aspirations

I just wish I could find the things I described above in all projects. Companies should aim to hire passionate and skilful people. People that can contribute to their projects and organisation. People that are willing to share and learn.

Unfortunately not always we will find all that. In those cases, the only thing we can do is to try our best to change the environment around us. We can try motivate people and share our passion. We can be nice to everyone, respect our colleagues and promote an environment where everyone feels comfortable to ask for help, to help each other and share knowledge.

With great people we can overcome any obstacle and have an environment where every morning, when we wake up, we would think: "Yay! Today I'll have another great day at work."

My first code retreat

2011-03-14T00:46:00.000Z

Yesterday I went to my first code retreat, in Winchester UK. In the past I had been sceptical about code retreats since I had doubts if I would really learn something. However, after speaking to a few developers that had been to one before, I was totally convinced that I should give it a go. I thought, even if I don't learn much, at least I'll have a whole day of fun writing code, catching up with some friends and meeting other developers. That was more than enough for me to be convinced that it would be a great day.

Summary of the day

There were around 20+ developers. After introductions and explanation of the problem (Conway's game of life), we started our first session. Throughout the day, we had 6 or 7 sessions. I can't even remember exactly because it was so enjoyable, intense and fun that even if it had been 20 I wouldn't have noticed. I've paired with some people that I knew before (although never worked with) and some people that I had just met. After each session, we shared what we have learned, approaches we took, problems we faced, etc. If I recollect well, different languages were used and tried including Java, C#, JavaScript, Ruby, Scala and I think Python as well (I'm sure I'm forgetting something).

The first two sessions we just sat down and tried to solve it, without any constraints, besides TDD, that was mandatory for all sessions. This was good so everyone could get familiar with the problem. From the third session onwards, we were asked to try different approaches, like not using if statements, TDD as if you meant it, OOP to the extreme and the best code we could possibly write. At the end of each session, the code is deleted.

We had a good break for lunch and went to the local pub in the evening.

Things I learned and experienced

No matter how much you think you know, it is still not much and not enough.
Pair-programming with other developers exposes you to different ways of thinking and opens your mind for new ideas.
Seeing and using different languages, makes you see software development with different eyes, giving you a much broader understanding and helps clearing up misconceptions you may have.
You are not alone. Like yourself, there more talented and passionate developers out there willing to share ideas and learn from each other.
I definitely learned a lot and feel I got back from there a better developer.
Besides all the technical learning, it was really great to meet so many talented developers.

Thank you

Firstly I would like to thank Despo and Aimee for organising it and Enrique for running it (and the great chat in the pub). I also would like to thank all the developers that I paired with for helping to make me a better developer. Finally, I would like to thank all the developers that were there for reinforcing my belief that software development is a great profession, full of talented, bright and passionate people and that we can really make a difference in moving our industry forward.

I wish you all have a great long road ahead of you and I'm looking forward to meeting you again any time soon.

Links

Official site for the Winchester Code Retreat
London Software Craftsmanship Community - LSCC

Readable Tests: Separating intent from implementation

2010-12-10T00:41:00.000Z

Very recently, I was working on a test class like this:

public class AnalyticsExpirationDateManagerTest extends TestCase {

 private static final long ONE_HOUR_TIMEOUT = 1000 * 60 * 60;
 private static final long TWO_HOUR_TIMEOUT = ONE_HOUR_TIMEOUT * 2;
 
 private Map<Parameter, Long> analyticsToTimeout;
 private long defaultTimeout;
 
 private Parameter minTimeoutParam;
 @Mock private CacheKeyImpl<Parameter> cacheKey;

    @Override
    protected void setUp() throws Exception {
     MockitoAnnotations.initMocks(this);
     
     this.minTimeoutParam = new Parameter("minTimeout", "type");
     
     when(cacheKey.getFirstKey()).thenReturn(minTimeoutParam);
     
     this.analyticsToTimeout = new HashMap<Parameter, Long>();
     this.defaultTimeout = 0;
    }
 
 public void
 testGetExpirationDateWhenAnalyticsToTimeoutsAndCacheKeyAreEmpty() {
  AnalyticsExpirationDateManager<Long> manager = 
    new AnalyticsExpirationDateManager<Long>(analyticsToTimeout, defaultTimeout);
  Date date = manager.getExpirationDate(cacheKey, 0L);
  assertNotNull(date);
 }
 
 public void 
 testGetExpirationDateWithMinimunTimeoutOfOneHour() {
  this.analyticsToTimeout.put(this.minTimeoutParam, ONE_HOUR_TIMEOUT);
  Collection<Parameter> cacheKeysWithMinTimeoutParam = new ArrayList<Parameter>();
  cacheKeysWithMinTimeoutParam.add(this.minTimeoutParam);
  when(this.cacheKey.getKeys()).thenReturn(cacheKeysWithMinTimeoutParam);
  
  AnalyticsExpirationDateManager<Long> manager = 
   new AnalyticsExpirationDateManager<Long>(analyticsToTimeout, defaultTimeout);
  Date date = manager.getExpirationDate(cacheKey, 0L);

  assertNotNull(date);
  Calendar expirationDate = Calendar.getInstance();
  expirationDate.setTime(date);
  
  Calendar currentDate = Calendar.getInstance();
  
  // Check if expiration date is one hour ahead current date. 
  int expirationDateHour = expirationDate.get(Calendar.HOUR_OF_DAY);
  int currentDateHour = currentDate.get(Calendar.HOUR_OF_DAY);
  assertTrue(expirationDateHour - currentDateHour == 1);
 }
 
 public void 
 testGetExpirationDateWhenCacheKeyIsNullAndDefaultTimeoutIsOneHour() {
  CacheKeyImpl<Parameter> NULL_CACHEKEY = null;
  AnalyticsExpirationDateManager<Long> manager = 
   new AnalyticsExpirationDateManager<Long>(analyticsToTimeout, ONE_HOUR_TIMEOUT);
  Date date = manager.getExpirationDate(NULL_CACHEKEY, 0L);
  
  assertNotNull(date);
  Calendar expirationDate = Calendar.getInstance();
  expirationDate.setTime(date);
  
  Calendar currentDate = Calendar.getInstance();
  
  // Check if expiration date hour is the same of current date hour.
  // When cache key is null, system date and time is returned and default timeout is not used.
  int expirationDateHour = expirationDate.get(Calendar.HOUR_OF_DAY);
  int currentDateHour = currentDate.get(Calendar.HOUR_OF_DAY);
  assertTrue(expirationDateHour - currentDateHour == 0);
 }
 
 public void 
 testGetExpirationDateWithDefaultTimeout() {
  // Default timeout is used when no time out is specified.
  Collection<Parameter> cacheKeysWithoutTimeoutParam = new ArrayList<Parameter>();
  cacheKeysWithoutTimeoutParam.add(new Parameter("name", "type"));
  when(this.cacheKey.getKeys()).thenReturn(cacheKeysWithoutTimeoutParam);

  AnalyticsExpirationDateManager<Long> manager = 
   new AnalyticsExpirationDateManager<Long>(analyticsToTimeout, ONE_HOUR_TIMEOUT);
  Date date = manager.getExpirationDate(cacheKey, 0L);
  
  assertNotNull(date);
  Calendar expirationDate = Calendar.getInstance();
  expirationDate.setTime(date);
  
  Calendar currentDate = Calendar.getInstance();
  
  // Check if expiration date is one hour ahead current date. 
  int expirationDateHour = expirationDate.get(Calendar.HOUR_OF_DAY);
  int currentDateHour = currentDate.get(Calendar.HOUR_OF_DAY);
  assertTrue(expirationDateHour - currentDateHour == 1);
 }
 
 public void 
 testGetExpirationDateWhenMinTimeoutIsSetAfterCreation() {
  AnalyticsExpirationDateManager<Long> manager = 
   new AnalyticsExpirationDateManager<Long>(analyticsToTimeout, ONE_HOUR_TIMEOUT);
  manager.setExpirationTimeout(this.minTimeoutParam.getName(), TWO_HOUR_TIMEOUT);
  
  Date date = manager.getExpirationDate(cacheKey, 0L);
  
  assertNotNull(date);
  Calendar expirationDate = Calendar.getInstance();
  expirationDate.setTime(date);
  
  Calendar currentDate = Calendar.getInstance();
  
  // Check if expiration date is two hour ahead current date. 
  int expirationDateHour = expirationDate.get(Calendar.HOUR_OF_DAY);
  int currentDateHour = currentDate.get(Calendar.HOUR_OF_DAY);
  assertTrue("Error", expirationDateHour - currentDateHour == 2);
 }
 
}

Quite frightening, isn't it? Very difficult to understand what's going on there.

The class above covers 100% of the class under test and all the tests are valid tests, in terms of what is being tested.

Problems

There are quite a few problems here:
- The intent (what) and implementation (how) are mixed, making the tests very hard to read;
- There is quite a lot of duplication among the test methods;
- There is also a bug in the test methods when comparing dates, trying to figure out how many hours one date is ahead of the other. When running these tests in the middle of the day, they work fine. If running them between 22:00hs and 00:00hs, they break. The reason is that the hour calculation does not take into consideration the day.

Making the tests more readable

Besides testing the software, tests should also be seen as documentation, where business rules are clearly specified. Since the tests here are quite messy, understanding the intention and detecting bugs can be quite difficult.

I've done quite a few refactorings to this code in order to make it more readable, always working in small steps and constantly re-running the tests after each change. I'll try to summarise my steps for clarity and brevity.

1. Fixing the hour calculation bug

One of the first things that I had to do was to fix the hour calculation bug. In order to fix the bug across all test methods, I decided to extract the hour calculation into a separate class, removing all the duplication from the test methods. Using small steps, I took the opportunity to construct this new class called DateComparator (yes, I know I suck naming classes) using some internal Domain Specific Language (DSL) techniques.

public class DateComparator {
 
 private Date origin;
 private Date target;
 private long milliseconds;
 private long unitsOfTime;
 
 private DateComparator(Date origin) {
  this.origin = origin;
 }
 
 public static DateComparator date(Date origin) {
  return new DateComparator(origin);
 }
 
 public DateComparator is(long unitsOfTime) {
  this.unitsOfTime = unitsOfTime;
  return this;
 }
 
 public DateComparator hoursAhead() {
  this.milliseconds = unitsOfTime * 60 * 60 * 1000;
  return this;
 }
 
 public static long hours(int hours) {
  return hoursInMillis(hours);
 }
 
 private static long hoursInMillis(int hours) {
  return hours * 60 * 60 * 1000;
 }
 
 public boolean from(Date date) {
  this.target = date;
  return this.checkDifference();
 }
 
 private boolean checkDifference() {
  return (origin.getTime() - target.getTime() >= this.milliseconds);
 }
}

So now, I can use it to replace the test logic in the test methods.

2. Extracting details into a super class

This step may seem a bit controversial at first, but can be an interesting approach for separating the what from how. The idea is to move tests set up, field declarations, initialisation logic, everything that is related to the test implementation (how) to a super class, leaving the test class just with the test methods (what).

Although this many not be a good OO application of the IS-A rule, I think this is a good compromise in order to achieve better readability in the test class.

NOTE: Logic can be moved to a super class, external classes (helpers, builders, etc) or both.

Here is the super class code:

public abstract class BaseTestForAnalyticsExperationDateManager extends TestCase {

 protected Parameter minTimeoutParam;
 @Mock protected CacheKeyImpl<Parameter> cacheKey;
 protected Date systemDate;
 protected CacheKeyImpl<Parameter> NULL_CACHEKEY = null;
 protected AnalyticsExpirationDateManager<Long> manager;

 @Override
 protected void setUp() throws Exception {
  MockitoAnnotations.initMocks(this);
  this.minTimeoutParam = new Parameter("minTimeout", "type");
  when(cacheKey.getFirstKey()).thenReturn(minTimeoutParam);
  this.systemDate = new Date();
 }

 protected void assertThat(boolean condition) {
  assertTrue(condition);
 }
 
 protected void addMinimunTimeoutToCache() {
  this.configureCacheResponse(this.minTimeoutParam);
 }
 
 protected void doNotIncludeMinimunTimeoutInCache() {
  this.configureCacheResponse(new Parameter("name", "type"));
 }
 
 private void configureCacheResponse(Parameter parameter) {
  Collection<Parameter> cacheKeysWithMinTimeoutParam = new ArrayList<Parameter>();
  cacheKeysWithMinTimeoutParam.add(parameter);
  when(this.cacheKey.getKeys()).thenReturn(cacheKeysWithMinTimeoutParam);
 }
}

3. Move creation and configuration of the object under test to a builder class

The construction and configuration of the AnalyticsExpirationDateManager is quite verbose and adds a lot of noise to the test. Once again I'll be using a builder class in order to make the code more readable and segregate responsibilities. Here is the builder class:

public class AnalyticsExpirationDateManagerBuilder {
 
 protected static final long ONE_HOUR = 1000 * 60 * 60;

 protected Parameter minTimeoutParam;
 private AnalyticsExpirationDateManager<Long> manager;
 private Map<Parameter, Long> analyticsToTimeouts = new HashMap<Parameter, Long>();
 protected long defaultTimeout = 0;
 private Long expirationTimeout;
 private Long minimunTimeout;

 private AnalyticsExpirationDateManagerBuilder() {
  this.minTimeoutParam = new Parameter("minTimeout", "type");
 }
 
 public static AnalyticsExpirationDateManagerBuilder aExpirationDateManager() {
  return new AnalyticsExpirationDateManagerBuilder();
 }
 
 public static long hours(int quantity) {
  return quantity * ONE_HOUR;
 }
 
 public AnalyticsExpirationDateManagerBuilder withDefaultTimeout(long milliseconds) {
  this.defaultTimeout = milliseconds;
  return this;
 }
 
 public AnalyticsExpirationDateManagerBuilder withExpirationTimeout(long milliseconds) {
  this.expirationTimeout = new Long(milliseconds);
  return this;
 }
 
 public AnalyticsExpirationDateManagerBuilder withMinimunTimeout(long milliseconds) {
  this.minimunTimeout = new Long(milliseconds);
  return this;
 }
 
 public AnalyticsExpirationDateManager<Long> build() {
  if (this.minimunTimeout != null) {
   analyticsToTimeouts.put(minTimeoutParam, minimunTimeout);
  }
  this.manager = new AnalyticsExpirationDateManager(analyticsToTimeouts, defaultTimeout);
  if (this.expirationTimeout != null) {
   this.manager.setExpirationTimeout(minTimeoutParam.getName(), expirationTimeout);
  }
  return this.manager;
 }

}

The final version of the test class

After many small steps, that's how the test class looks like. I took the opportunity to rename the test methods as well.

import static com.mycompany.AnalyticsExpirationDateManagerBuilder.*;
import static com.mycompany.DateComparator.*;

public class AnalyticsExpirationDateManagerTest extends BaseTestForAnalyticsExperationDateManager {

 public void
 testExpirationTimeWithJustDefaultValues() {
  manager = aExpirationDateManager().build();
  Date cacheExpiration = manager.getExpirationDate(cacheKey, 0L);
  assertThat(dateOf(cacheExpiration).is(0).hoursAhead().from(systemDate));
 }
 
 public void 
 testExpirationTimeWithMinimunTimeoutOfOneHour() {
     addMinimunTimeoutToCache();  
  manager = aExpirationDateManager()
      .withMinimunTimeout(hours(1))
      .build();
  Date cacheExpiration = manager.getExpirationDate(cacheKey, 0L);
  assertThat(dateOf(cacheExpiration).is(1).hoursAhead().from(systemDate));
 }
 
 public void 
 testExpirationTimeWhenCacheKeyIsNullAndDefaultTimeoutIsOneHour() {
  manager = aExpirationDateManager()
      .withDefaultTimeout(hours(1))
      .build();
  Date cacheExpiration = manager.getExpirationDate(NULL_CACHEKEY, 0L);
  // When cache key is null, system date and time is returned and default timeout is not used.
  assertThat(dateOf(cacheExpiration).is(0).hoursAhead().from(systemDate));
 }
 
 public void 
 testExpirationTimeWithDefaultTimeout() {
  doNotIncludeMinimunTimeoutInCache();
  manager = aExpirationDateManager()
      .withDefaultTimeout(hours(1))
      .build();
  Date cacheExpiration = manager.getExpirationDate(cacheKey, 0L);
  assertThat(dateOf(cacheExpiration).is(1).hoursAhead().from(systemDate));
 }
 
 public void 
 testExpirationTimeWhenExpirationTimeoutIsSet() {
  manager = aExpirationDateManager()
      .withDefaultTimeout(hours(1))
      .withExpirationTimeout(hours(2))
      .build();
  Date cacheExpiration = manager.getExpirationDate(cacheKey, 0L);
  // Expiration timeout has precedence over default timeout.
  assertThat(dateOf(cacheExpiration).is(2).hoursAhead().from(systemDate));
 }
 
}

Conclusion

Test classes should be easy to read. They should express intention, system behaviour, business rules. Test classes should express how the system works. They are executable requirements and specifications and should be a great source of information for any developer joining the project.

In order to achieve that, we need to try to keep our test methods divided in just 3 simple instructions.

1. Context: The state of the object being tested. Here is where we set all the attributes and mock dependencies. Using variations of the Builder pattern can greatly enhance readability.

manager = aExpirationDateManager()
                .withDefaultTimeout(hours(1))
                .withExpirationTimeout(hours(2))
                .build();

2. Operation: The operation being tested. Here is where the operation is invoked.

Date cacheExpiration = manager.getExpirationDate(cacheKey, 0L);

3. Assertion: Here is where you specify the behaviour expected. The more readable this part is, the better. Using DSL-style code is probably the best way to express the intent of the test.

assertThat(dateOf(cacheExpiration).is(2).hoursAhead().from(systemDate));

In this post I went backwards. I've started from a messy test class and refactored it to a more readable implementation. As many people now are doing TDD, I wanted to show how we can improve an existing test. For new tests, I would suggest that you start writing the tests following the Context >> Operation >> Assertion approach. Try writing the test code in plain English. Once the test intent is clear, start replacing the plain English text with Java internal DSL code, keeping the implementation out of the test class.

PS: The ideas for this blog post came from a few discussions I had during the Software Craftsmanship Round-table meetings promoted by the London Software Craftsmanship Community (LSCC).

A basic ActiveRecord implementation in Java

2010-12-07T21:25:00.000Z

Recently I was looking for different ways to develop applications in Java and thought that it would be interesting trying to use ActiveRecord in my persistence layer instead of implementing the traditional approach with a DAO. My idea is not to create my own ActiveRecord framework but use the existing ones as a support for this approach.

The scope: Write functional tests that could prove that the methods save and delete on an entity work. I'll use an entity called Traveller for this example.

The technology: I chose to use the following frameworks: Spring 3.0.5, JPA 2.0, Hibernate 3.5.3, AspectJ 1.6.9, JUnit 4.8.2, Maven 2.2.1, Eclipse Helios and MySQL 5.x

I'll be omitting things that are not too important. For all the details, please have a look at the whole source code at:

https://github.com/sandromancuso/cqrs-activerecord

Let's start with the test class:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(
  locations={
    "file:src/test/resources/applicationContext-test.xml",
    "file:src/main/resources/applicationContext-services.xml"    
  })
@TransactionConfiguration(transactionManager = "myTransactionManager", defaultRollback = true)
@Transactional
public class TravellerActiveRecordIntegrationTest extends BaseTravellerIntegration {
 
 @Test public void 
 testTravellerSelfCreation() {
  assertThereAreNoTravellers(named("John"), from("England"));
  
  Traveller traveller = aTraveller().named("John").from("England").build();
  traveller.save();
  
  assertThereIsASingleTraveller(named("John"), from("England"));
 }
 
 @Test public void
 testTravellerEdition() {
  Traveller traveller = aTraveller().named("John").from("England").build();
  traveller.save();
  
  traveller.setName("Sandro");
  traveller.setCountry("Brazil");
  traveller.save();
  
  assertThereAreNoTravellers(named("John"), from("England"));
  assertThereIsASingleTraveller(named("Sandro"), from("Brazil"));  
 }
 
 @Test public void 
 testDeleteTraveller() {
  Traveller traveller = aTraveller().named("John").from("England").build();
  traveller.save();

  traveller.delete();
  assertThereAreNoTravellers(named("John"), from("England"));
 }
 
}

A few things to notice about this test class:
- As this test is meant to insert and delete from the database, I set it to always rollback the transaction after each test, meaning that nothing will be committed to the database permanently. This is important in order to execute the tests multiple times and get the same results.
- The test class extends a base class, that has the methods assertThereAreNoTravellers and assertThereIsASingleTraveller. The advantage of doing that is that you separate what you want to test from how you want to test, making the test class clean and focused on the intention.
- Note that I also used the Builder pattern to build the traveller instance instead of using the setter methods. This is a nice approach in order to make your tests more readable.

The Traveller entity implementation

So now, let's have a look at the Traveller entity implementation.

@Entity
@Table(name="traveller")
@EqualsAndHashCode(callSuper=false)
@Configurable
public @Data class Traveller extends BaseEntity {

 @Id @GeneratedValue(strategy=GenerationType.AUTO)
 private long id;
 private String name;
 private String country;
 
}

The Traveller class is a normal JPA entity as you can see by the @Entity, @Table, @Id and @GenereratedValue annotations. To reduce the boiler place code like getters, setters, toString() and hashCode(), I'm using Lombok, that is a small framework that generate all that for us. Just add a @Data annotation.

The real deal here is Traveller's super class BaseEntity (for a lack of inspiration to find better name). Let's have a look:

@Configurable
public abstract class BaseEntity {

 @PersistenceContext
 protected EntityManager em;

 public abstract long getId();
    
 public void save() {
  if (getId() == 0) {
   this.em.persist(this);
  } else {
   this.em.merge(this);
  }
  this.em.flush();
 } 
 
 public void delete() {
  this.em.remove(this);
  this.em.flush();
 }
 
 public void setEntityManager(EntityManager em) {
  this.em = em;
 }
 
}

The BaseEntity class has quite a few things that can be discussed.

The save() method: I've chosen to have a single method that can either insert or update an entity, based on its id. The problem with this approach is that, firstly, the method has more than one responsibility, making it a bit confusing to understand what it really does. Secondly it relies on the entities id, that needs to be a long, making it a very specific and weak implementation. The advantage is that from the outside (client code), you don't need to worry about the details. Just simple call save() and you are done. If you prefer a more generic and more cohesive implementation, make the getId() method return a generic type and split the save() method into a create() and update() methods. My idea here was just to make it simple to use.

EntityManager dependency: Here is where the biggest problem lies. For it to work well, every time a new instance of a entity is created, either by using new EntityXYZ() by hand or when it is created by a framework (e.g. as a result of a JPA / Hibernate query), we want the entity manager to be injected automatically. The only way I found to make it work is using aspects with AspectJ and Spring.

Configuring AspectJ and Spring

My idea here is not to give a full explanation about the whole AspectJ and Spring integration, mainly because I don't know it very well myself. I'll just give the basic steps to make this example work.

First add @Configurable to the Entity. This will tell that the entity will be managed by Spring. However, Spring is not aware of instances of the entity, in this case, Traveller, being created. This is why we need AspectJ. The only thing we need to do is to add the following line to our Spring context xml file.

<context:load-time-weaver />

This makes AspectJ intercept the creation of beans annotated with @Configurable and tells Spring to inject the dependencies. In order to the load-time weaver (LTW) work, we need to override JVM's class loading so that the first time our entity classes are loaded, AspectJ can kick in, the @Configurable annotation is discovered and all the dependencies are injected. For that we need to pass the following parameter to the JVM:

-javaagent:<path-to-your-maven-repository>/.m2/repository/org/springframework/spring-instrument/3.0.5.RELEASE/spring-instrument-3.0.5.RELEASE.jar

The snippet above is what we must use if using Spring 3.0.x. It works fine inside Eclipse but apparently it has some conflicts with Maven 2.2.1. If you run into any problems, you can use the old version below.

-javaagent:<path-to-your-maven-repository>/.m2/repository/org/springframework/spring-agent/2.5.6/spring-agent-2.5.6.jar

Another thing that is a good idea is to add the aop.xml file to your project, limiting the classes that will be affected by AspectJ. Add the aop.xml to src/main/resources/META-INF folder.


 
  
  <include within="com.lscc.ddddemo.model.entity.*" />
  <include within="com.lscc.ddddemo.model.entity.builder.*" />
  <exclude within="*..*CGLIB*" />

NOTE: The exclude clause is important to avoid some conflicts during the integration test. AspectJ sometimes tries to do some magic with the test classes as well causing a few problems and the exclude clause avoids that.

Making the integration test work

I'll be using MySQL, so I'll need a database with the following table there:

CREATE TABLE `traveller` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(45) NOT NULL,
`country` varchar(45) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB

As I'm using JPA 2.0, we need a persistence.xml that must be located at src/main/resources/META-INF


 
    
        com.lscc.ddddemo.model.entity.Traveller
 
        
            <property name="hibernate.show_sql" value="true" />
            <property name="hibernate.format_sql" value="true" />
 
            <property name="hibernate.connection.driver_class" value="com.mysql.jdbc.Driver" />
            <property name="hibernate.connection.url" value="jdbc:mysql://localhost:3306/lscc-ddd" />
            <property name="hibernate.connection.username" value="root" />
 
            <property name="hibernate.c3p0.min_size" value="5" />
            <property name="hibernate.c3p0.max_size" value="20" />
            <property name="hibernate.c3p0.timeout" value="300" />
            <property name="hibernate.c3p0.max_statements"
                    value="50" />
            <property name="hibernate.c3p0.idle_test_period"
                    value="3000" />

In our Spring configuration, we also need to tell Spring how to create an EntityManager, providing the EntityManager Factory:


        
    
        <property name="persistenceUnitName" value="testPU" />
    

    <tx:annotation-driven transaction-manager="myTransactionManager" />
 
    
        <property name="entityManagerFactory" ref="entityManagerFactory" />

In order to separate unit tests and integration tests in my Maven project, I've added a different profile for the integration tests in my pom.xml:


   with-integration-tests
   
    
     
      org.apache.maven.plugins
      maven-surefire-plugin
      
       always
       -javaagent:/Users/sandro.mancuso/.m2/repository/org/springframework/spring-agent/2.5.6/spring-agent-2.5.6.jar
      
      2.5
      true
      
       
        integration-tests
        integration-test
        
         test
        
        
         
          **/common/*
         
         
          **/*IntegrationTest.java

So now, if you want to run the integration tests from the command line, just type:

mvn clean install -Pwith-integration-tests

If running the tests from inside Eclipse, don't forget to add the -javaagent paramenter to the VM.

Conclusion

So now, from anywhere in your application you can do some thing like:

Traveller traveller = new Traveller();
traveller.setName("John");
traveller.setCountry("England");
traveller.save();

The advantage of using the ActiveRecord:
- Code becomes much simpler;
- There is almost no reason for a DAO layer any more;
- Code is more explicit in its intent.

The disadvantages:
- Entities will need to inherit from a base class;
- Entities would have more than one responsibility (against the Single Responsibility Principle);
- Infrastructure layer (EntityManager) would be bleeding into our domain objects.

In my view, I like the simplicity of the ActiveRecord pattern. In the past 10 years we've been designing Java web applications where our entities are anaemic, having state (getters and setters) and no behaviour. So at the end, they are pure data structures. I feel that entities must be empowered and with techniques like that, we can do it, abstracting the persistence layer.

I'm still not convinced that from now on I'll be using ActiveRecord instead of DAOs and POJOs but I'm glad that now there is a viable option. I'll need to try it in a real project, alongside with Command Query Responsibility Segregation (CQRS) pattern to see how I will feel about it. I'm really getting sick of the standard Action/Service/DAO way to develop web applications in Java instead of having a proper domain model.

By the way, to make the whole thing work, I had loads of problems to find the right combination of libraries. Please have a look at my pom.xml file for details. I'll be evolving this code base to try different things so when you look at it, it may not be exactly as it is described here.

https://github.com/sandromancuso/cqrs-activerecord

Interesting links with more technical details:
http://nurkiewicz.blogspot.com/2009/10/ddd-in-spring-made-easy-with-aspectj.html
http://blog.m1key.me/2010/06/integration-testing-your-spring-3-jpa.html

Bad Code: The Invisible Threat

2010-09-27T22:41:00.000+01:00

One of the first things said by the non-believers of the software craftsmanship movement was that good and clean code is not enough to guarantee the success of a project. And yes, they are absolutely right. There are innumerable reasons which would make a project to fail ranging from business strategy to competitors, project management, cost, time to market, partnerships, technical limitations, integrations, etc.

Due to the amount of important things a software project may have, organisations tend not to pay to much attention to things that are considered less important, like the quality of the software being developed. It is believed that with a good management team, deep hierarchies, micro management, strict process and a large amount of good documentation, the project will succeed.

In a software project, the most important deliverable is the software itself. Anything else is secondary.

Many organisations see software development as a production line where the workers (developers) are viewed as less skilled than their high-qualified and much more well paid managers. Very rarely companies like that will be able to attract or retain good software developers, leaving their entire business on the hands of mediocre professionals.

Look after your garden

Rather than construction, programming is more like gardening. - The Pragmatic Programmer

Code is organic, not mechanical. Like a garden, code needs constant maintenance. For a garden to look nice all year round, you need to look after its soil, constantly remove weeds, regularly water it, remove some dead plants, replant new ones, trim or re-arrange existing ones so they can stay healthy and look nice as whole. With basic regular maintenance, the garden will always look great but if you neglect it, even if for a short period, the effort to make it nice again will be much bigger. The longer you neglect it, the harder it will be to make it look nice again and you may even loose some or all of your plants.

Code is no different. If code quality is not constantly looked after, the code starts to deteriorate. Bad design choices, lack of tests and poor use of languages and tools will make parts of the code to rot. Bit by bit other parts of the code will also be contaminated up to the point that the whole code base is so ill that it becomes extremely painful to change it or add new features to it.

The Invisible Threat

When starting a greenfield project, everything is great. With a non-existent code base, developers can quickly start creating new features without the fear of breaking or changing any existing code. Testers are happy because everything they need to test is new, meaning that they don't need to worry about regression tests. Managers can see a quickly progress in terms of new features added and delivered. What a fantastic first month the team is having.

However, this is not a team of craftsmen. This is a team of average developers structured and treated like unskilled production line workers.

As time goes by, things are getting messier, bugs start to appear (some with no apparent explanation) and features start taking longer and longer to be developed and tested. Very slowly, the time to deliver anything starts to stretch out. But this is a slow process. Slow enough that takes months, sometimes, over a year or two to be noticed by the management.

It's very common to see projects where, at the beginning of a project, a feature of size X takes N number of days to be implemented. Over the time, as more bad code is added to the application, the same feature X (or a feature of the same size) takes much longer to be implemented than it used to take at the beginning of the project. As the quality of the code decreases, the amount of time to implement a new feature, fix a bug or make a change increases. The lower the quality, the higher the number of bugs, the harder is to test and less robust and reliable the application becomes.

Some people say that they just don't have time to do it properly but, in general, a lot more time and money is spent later on on tests and bug fixing.

Hostage of your own software

When the code base gets into the situation where changes or additional features take too long to be implemented or worse, developers and managers are scared to touch existing code, an action must be taken immediately. This is a very dangerous situation to be since business progress is being impeded or delayed by the software instead of being helped by it.

To keep business progress, schedule and budget under control, high quality code needs to be maintained at all costs.

Organisations may need to cancel the implementation of some features or postpone changes just because of the amount of time and money that they may cost to be built. Having poor quality of code responsible for it is totally unacceptable.

The biggest problem here is that bad code is invisible to everyone besides developers. Other members of the team will just realise that something is wrong when it is too late. This means that it is the developers responsibility to look after the quality of the code. Some times, developers expose the problem to project managers but the request for having some time to "re-factor" the code is often ignored for various reasons, including a lack of understand of the impacts of bad code and the inability of developers to explain it. On the other hand, when developers come to a point where they need to ask for some formal time to do refactoring, this means that for one reason or another, they neglected the code at some point in the past.

Hire craftsmen not average developers

With the amount of literature, tools, technologies, methodologies and the infinite source of information available on the web, it is just unacceptable to have a team of developers that let the code to rot.

Craftsmen are gardeners and are constantly looking after the code base, quickly refactoring it without fear since they are strongly backed by a good battery of tests that can test the entire application in just a few minutes. Time constraints or change in requirements will never be used as excuses for bad code or lack of tests due to the good design principles and techniques constantly used throughout the application.

Having an empowered team of craftsmen can be the difference between success and failure of any software project.

Quality of code may not guarantee the success of a project but it can definitely be the main invisible cause of its failure.

Beyond the manifesto: The Software Craftsmanship Attitude

2010-09-14T23:19:00.000+01:00

Being an aspiring software craftsman goes way beyond than just saying it. I'll quote my own definition of software craftsmanship from my previous post.

Software craftsmanship is a long journey to mastery. It's a lifestyle where developers choose to be responsible for their own careers and for improving their craft, constantly learning new tools and techniques. Software Craftsmanship is all about putting responsibility, professionalism, pragmatism and pride back into software development.

Software craftsmanship is all about attitude. The attitude of raising the bar of professional software development starting with our own skills and professionalism.

The responsibility shift

Not long ago, I was speaking to a developer and he was complaining about his company, saying that they didn't have a good career plan, that he did not have any training, that he was not given the opportunity to learn new technologies and, of course, that he was not paid enough. Apparently, from his perspective, his employer was responsible for his career.

Imagine that we need a doctor. Would we pay a doctor to learn while he cut us open or give us a diagnosis? Would we pay an engineer to learn while he draws the plan for our new house? Would we go to a concert and pay the musician to learn how to play the guitar during the show? What about a chef in a restaurant?

So why is the employer's obligation to pay us training courses and pay us to learn new technologies and tools while we are working on a project? Should the employers be responsible for what we learn and what we don't learn.

Software development is not a 9 to 5 profession. To be a professional software developer, we need to take our own time and money to keep learning and improving. As professionals, we should be paid for what we know, our ability to learn fast and for the quality of the work we do. We own our careers and are responsible for them. Working for a customer/employer that helps us with our career in terms of training, books, conferences, seminars, etc, is great but should be considered a bonus.

... but how can we learn and keep ourselves up-to-date?

Different developers have different preferences but here is a list of ways I find useful.

Literature

Books, many books. Having your own library is essential. Books give you a good overview of a specific technology or subject quickly. No, it does not mean you will be proficient and next day you will be a specialist. What a book will give you is an understanding of what a technology or subject is about. It will be up to you then to decide if you want to practice what you've learned and become proficient. If you don't have the habit, try reading 3 to 4 technical books per year. Once you get the habit, try one per month. The most common excuse is that you don't have time. The cool thing about books is that you can read them during periods of "dead" time, like on the tube, bus, in your dentist's waiting room, on the bed before going to sleep, toilet, etc.

Blogs are now one of my favourite types of reading. They tend to fit more the software craftsmanship model since they are much more personal and in general, related to personal findings, opinions, successes and failures. Reading blogs from more experienced professionals and subject matter experts is a good, quick and free way for apprentices and journeymen to learn from multiple master craftsmen at the same time. But don't think that just experienced professionals should write blogs. Every software developer should write his or her own blogs, sharing their experiences and findings, helping to create a great community of professionals.

Technical websites are also good in order to keep yourself up-to-date with what's going in the market. New technologies, techniques, etc.

Practice, practice, practice

Pet project(s) is for me, by far, the best way to learn and study. A pet project is a real project but without the boring bits. There are no deadlines, does not need to make money, you control the requirements and most importantly, you use the technologies and methodologies you want, whenever you want, wherever you want. You are the boss. Pet projects give something for you to focus and help you to understand why and how you can use certain technologies. It gives you the experience you need to apply what you've learned into real projects. Pet projects are meant to be fun.

Contributing to open source projects can also be a great thing. There are thousands of them out there. Find a project that is related to what you want to learn or know more about and download the source code. Start running and reading the tests, if any. Inspect and debug the code. If you want to contribute, start small. Add some documentation and write some tests. Then, check the list of features to be implemented and pick a simple one and give it a go. You can also propose and implement a new and small one to start with.

Pair-programming can be a great experience. Many developers are afraid to try or think they will feel uncomfortable. That's what I thought as well before I tried. Today I really enjoy it. Pair-programming gives you an opportunity to discuss your ideas, to learn new tricks and techniques from your pair and the resulting code is much better. Pairing with someone from your team is good but pairing with someone that you barely know can be a very interesting experience.

If learning a new language or new technique like TDD / BDD or trying different approaches to OOP or functional programming, try a code kata. Code kata is a small exercise that in general can be solved in a few minutes or in a few hours. Code katas were created to help developers to focus in a problem while improving their skills. You can find a good source of code katas at codingkata.org and codekata.pragprog.com

Community and Events

Be part of your local community. Software craftsmanship is all about a community of professionals, learning and sharing with each other and elevating the level of professionalism and maturity of our industry. Join your nearest user groups and participate in their events. User groups are the best way for making contacts, sharing ideas and learning things. Many user groups promote free talks, coding activities and social events. A great aspect of being part of a community if the feeling that you are not alone. There are many people out there having the same problems you've got and many others that will happily share their solution to it.

Raising the bar

The main changes proposed by the software craftsmanship movement are related to the developers attitude. In summary, it's about being proud about your work, the code you produce and the software you create. It's about constantly trying to improve your skills and learn new ones.

An aspiring software craftsman is intolerant of bad code will constantly apply the Boy Scout Rule.

If you find that the good code you wrote one year ago is still good enough today, it means you didn't learn anything during this period and this is unacceptable.

Productive partnerships

The attitude of a software craftsman goes beyond good code. The relationship between a software craftsman and his or her customer (or employer) is of a productive partnership and not employer / employee. As a productive partnership, it's our role to constantly question the requirements and propose improvements and new features. It's our job to warn our customers about the problems we see. Due to the knowledge of the code and technology, developers are well positioned to help their customers improve their business.

However, sometimes some customers or employers don't want or don't see the advantages of this partnership and will treat developers as production line workers that should just do what they are told. Coding monkeys. In cases like that, any aspiring software craftsman should move on and find another job. Staying in a place where your skills are not appreciated is a career suicide.

Raising the bar of our industry is on our own interest. The better we become in writing code and delivering valuable software, the better our lives will be, professionally and financially.

PS: If you are in London or nearby, join the London Software Craftsmanship Community.

Software Craftsmanship

2010-09-03T17:40:00.000+01:00

So what is software craftsmanship?

A better metaphor: In a very simplistic way, we can say that software craftsmanship is a better metaphor for software development than software engineering, as I wrote in a previous post. Software craftsmanship sees software as a craft and compares software developers to the medieval blacksmiths. Apprentices would work with more experienced blacksmiths, travelling from place to place, working with and for different masters, learning different tools and techniques, improving their craft until the point they were good enough to become master themselves. (There is more to it, but let's keep it simple for now).

Wikipedia Definition: is an approach to software development that emphasizes the coding skills of the software developers themselves. It is a response by software developers to the perceived ills of the mainstream software industry, including the prioritization of financial concerns over developer accountability. (read in full)

I personally don't like too much the Wikipedia's definition. It's very dry and I don't think it captures the essence of what being a software craftsman means to a software developer.

A more personal definition: Software craftsmanship is a long journey to mastery. It's a lifestyle where developers choose to be responsible for their own careers and for improving their craft, constantly learning new tools and techniques. Software Craftsmanship is all about putting responsibility, professionalism, pragmatism and pride back into software development

A software craftsman cares and is proud of his or her work and is extremely professional and pragmatic when it comes to its implementation.

The Software Craftsmanship Movement

The software craftsmanship movement is basically an evolution of ideas that started probably in the late 90ies, early 2000 with the publication of The Pragmatic Programmer by Andy Hunt and Dave Thomas (1999) and Software Craftsmanship: The New Imperative by Pete McBreen (2001). In 2008, Uncle Bob proposed "Craftsmanship over Execution" (originally Craftsmanship over Crap) as the fifth value for the Agile Manifesto. In 2009, the Manifesto for Software Craftsmanship was created, defining the values of the movement and international Software Craftsmanship conferences emerged in the US and UK.

In the manifesto, the essence of the software craftsmanship is capture in its subtitle: Raising the bar. The manifesto was idealised by very experienced developers that had enough of project failures mainly caused by poor management, ill processes and, of course, badly-written code.

Developers are taking the matter into their own hands and are trying to change how the industry sees software development not just proposing new and revolutionary processes but showing customers that they care about what they do and that they want to work together with their customers in order to produce great and long-lived software.

The values of the Software Craftsmanship Movement

Not only working software, but also well-crafted software

Working code is not good enough. Think on a 5 year old application (it could be 2 or 10) where we are scared to change some of its parts (fix bugs, add new features, etc.) because we don't understand how it works and have no confidence that we will not break anything else. This application is working software but is it good enough? Well crafted software means that regardless how old the application is, developers can understand it easily, side effects are well known and controlled, high test coverage, clear design, business language well expressed in the code and adding or changing features does not take longer that it used to take at the beginning of the project, when the code base was small.

The code must be maintainable and predictable. Developers must know what is going to happen when changing the code and must not fear to change it. Changes should be localised and not cause impact in other parts of the application. Tests will guarantee that nothing else was broken.

Not only responding to change, but also steadily adding value

This is not just about adding new features and fixing bugs. This is also about constantly improving the structure and cleanliness of the code. The software must be seen as an asset and the constant maintenance of it will make it more valuable during its lifetime, instead of letting it rot and devalue.

The Boy Scout Rule (defined by Uncle Bob) states that we should always let the code a bit cleaner than we found it. This is a paraphrase of Boy Scouts rule: leave the camp cleaner than you found it.

If customers want to keep benefiting from adding and changing features quickly, they will need high quality code to enable them to do it.

Not only individuals and interactions, but also a community of professionals

This is somehow related to the idea of apprentices, journeymen and masters, where software craftsmanship masters will mentor apprentices and help them in their journey. The software craftsmanship community is responsible for training the next generation of professionals. Knowledge and ideas must be shared and discussed within the community in order to keep moving the software development industry forward.

Not only customer collaboration, but also productive partnerships

Software craftsmen need successful projects to build their reputation and are proud of their achievements. Successfully delivering high quality software is essential for any software craftsman journey. With this in mind, software craftsmen will do whatever they can for a project to succeed. They don't act like simple employees that just do what they are told to do. They want to actively contribute to the success of the project, questioning requirements, understanding the business, proposing improvements and productively partnering with their customers or employers. This is an interesting shift of perspective, if you like, and the advantages for the customer and for the project success are enormous. A well-motivated team has a much bigger chance to make any project succeed. However, if the customer is not prepared to have this partnership and sees software development as an industrial process and the least important part of the project, this customer will never have real software craftsmen working for him for too long. Not getting involved with the business, not questioning requirements, not proposing improvements and not knowing the customers needs is not a partnership. Real software craftsmen make the customer needs, their needs.

Conclusion

Recently, Uncle Bob Martin said during an interview:

The original torch of the Agile message has changed hands, and is now being carried by the Software Craftsmanship movement. These are the folks who continue to pursue the technical excellence and professionalism that drove the original founders of the Agile movement.

The Software Craftsmanship Movement is another step forward towards better and healthier software projects. Instead of just focusing on processes, it focuses on the quality of the software that is produced and most importantly on the attitude and competence of the people involved.

Software Craftsmanship brings pride and professionalism into software development.

Software Engineering: The problem with the production line

2010-08-22T23:35:00.000+01:00

Is software engineering the best approach for developing software? Does it apply for the majority of the software projects or just a very few of them?

Software Engineering was an answer for the perceived "software crisis", back in 1968, in the First NATO Software Engineering Conference and it was created to solve the problems of extremely large NATO and U.S. Department of Defence projects. In the majority of these projects, the hardware was still being designed and with no hardware to test, there was plenty of time to investigate requirements and write the software specifications. Hardware controlled by the software was generally worth billions of dollars like in the case of space shuttle and the Safeguard Ballistic Missile Defence System. People's lives and national security were also at stake.

The IEEE Computer Society's Software Engineering Body of Knowledge defines "software engineering" as:

Software engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software.

Software Engineering can be very effective when developing safety critical systems like the one for the space shuttle as described on "They Write The Right Stuff" - Fast Company article, from 1996:

The last three versions of the program - each 420,000 lines long - had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.

Of course that this looks really impressive but there is more to it:

Money is not the critical constraint: The group's $35 million per year budget is a trivial slice of the NASA pie, but on a dollars-per-line basis, it makes the group among the nation's most expensive software organizations.

Another extract from this same article when they were discussing the process:

And the culture is equally intolerant of creativity, the individual coding flourishes and styles that are the signature of the all-night software world. "People ask, doesn't this process stifle creativity? You have to do exactly what the manual says, and you've got someone looking over your shoulder," says Ted Keller (senior technical manager). "The answer is, yes, the process does stifle creativity."

Many of the NATO and US Department of Defence took years, some over a decade to complete. Many had hundreds of people involved and almost half of the time was spend in requirements and design specifications, with uncountable reviews and approval cycles. Of course that, due to their reliability and quality, many are still in use today.

Software Engineering for the Masses

More and more hardware became cheap and business of all sizes needed software in order to survive and be competitive. The difference was that a the big majority of these businesses couldn't afford to pay $35 million dollars per year and neither wait for too many years to start benefiting from their software. Also, many of those software projects were not manipulating expensive hardware or dealing with life-threatening situations. Edward Yourdon, in his book Rise and Resurrection of the American Programmer, wrote:

I'm going to deliver a system to you in six months that will have 5,000 bugs in it - and you're going to be very happy!

It was clear that software engineering processes had to be adapted in order to satisfy a more impatient and lower budget legion of businesses.

The "Good Enough Software" Era

Over the decades, many new software engineering processes and methodologies were created in order to make software development faster and cheaper. The most adopted were the ones based on iterative and incremental development that evolved into the agile software development.

Agile software development and "good enough software" were an amazing improvement in bringing costs down, mitigating risks with quicker feedbacks and much faster time to market.

Regardless the methodology or process used, software projects are still failing. Some fail because they were over-budget, others because they were not delivered on time, others failed to satisfy the requirements and business goals, et al.

The main problem is that for decades, software development was seen as a production line. That's the software engineering perspective of software development. Processes and methodologies are generally much more focused in making this production line more productive, creating different management and organisational styles than actually trying to make the employees more capable. Developers are treated as mere brain-damaged programmers and are at the bottom of the food chain.

Good enough is not always good enough

Using agile, lean or any methodology is not enough. It is all about the people involved in the project and their willingness to succeed and be proud of what they produce. If developers are seen as the least important and cheapest members of a software project, the maximum that this team is going to produce is mediocre software, regardless of methodology.

A software project would have much better chances to succeed with "good people with a bad process" than "mediocre people with a good process". Good people would always find a way to improve the process, be productive and produce something that they are proud of. Mediocre people accepts whatever is established, even when it is not good enough, and will produce just what they were asked for.

In a software project, if a company wants a good software, they will need good and empowered software developers. Instead of 10 mediocre people with a good process, it would be better to have 3 or 4 good people and empower them to deliver the project.

A process, imposed by managers and people that have no clue how to write software, will just guarantee that mediocre software is predictably delivered (if lucky). On the other hand, a team of self-organised and great developers would have a better shot at creating a more efficient way to produce great software, constantly trying to improve they way they work.

A process should never be more important than the people. Managers should facilitate the work of great software developers and not tell them what to do. It should always be harder to replace a good software developers than a manager since they are the ones that know the system inside-out. Managing a few well motivated, well paid and good professionals is always easier than manage many mediocre people.

Software development is a creative and highly skilled profession that takes years to master. While software development is treated like a production line, projects will continue to fail.

Source

Software Craftsmanship: The New Imperative - ISBN 0-201-73386-2, 2002
http://en.wikipedia.org/wiki/Software_engineering
http://en.wikipedia.org/wiki/Software_crisis
IEEE Standard Computer Dictionary, ISBN 1-55937-079-3, IEEE 1990
"They Write The Right Stuff", Fast Company, http://www.fastcompany.com/magazine/06/writestuff.html
Safeguard Program: http://en.wikipedia.org/wiki/Safeguard_Program
Stephenson, W. E. "An analysis of the resources used in the SAFEGUARD system software development"
Edward Yourdon - http://yourdon.com/about/
http://en.wikipedia.org/wiki/Iterative_and_incremental_development
http://en.wikipedia.org/wiki/Agile_software_development

Which type of barista are you?

2010-08-07T01:29:00.000+01:00

A colleague from New Zealand once was telling me about a type of coffee that was originated there. It's called Flat White Coffee. We were discussing about the difference between this coffee and all the other types of coffee with milk. Eventually we started talking about the quality of the drink and what makes it be better or worse. He mentioned that, of course, the quality of the beans and milk are very important for a good coffee but what makes a good coffee an excellent coffee is the barista's ability.

What makes a coffee to be a rubbish coffee then? If the coffee grains are rubbish, you will have a rubbish coffee. However, for a flat white coffee, coffee is just one variable in the equation. There are other variables like how the grains are roasted, steaming the milk at the right temperature, not adding sugar, how the milk is poured, the microfoam on top of the drink, etc. See the distinction from cafe con leche for details. Anyway, the point is, a rubbish (or careless) barista, with rubbish coffee will produce a rubbish coffee drink.

Rubbish Coffee

Clearly a business that serves a rubbish coffee will not survive if their main business is to sell coffee drinks. Also, this business will never attract customers that really appreciate a good coffee.

Average Coffee

Producing an average coffee is easy. You buy average coffee grains, hire an average barista and, hey presto, you have an average coffee drink. There is nothing wrong with an average coffee drink if you are not in the coffee business. However, if you are in a coffee business, your business will be just another one. No one will remember you and chances are that you will have occasional customers, but not regulars.

Good Coffee

Producing a good coffee is not that simple. However, it does not need to be expensive. You don't need to buy the best quality ingredients to be able to make a good coffee. Best quality ingredients are expensive and inevitably will make your coffee drinks more expensive as well. You can mitigate this situation hiring a good barista, or maybe someone that has the potential and willingness to become a good one. Clearly, the barista that prepared the coffee above is trying his best to make a good coffee. It's still not perfect (purely looking at the pattern on top of the drink) but it is definitely a much better coffee than the normal and average one that you get everywhere. It is clear that the barista cares about it and eventually he will be able to produce a very good coffee.If you are in the coffee business, producing anything less than a good coffee is just unacceptable.

Great Coffee

And then you have the great coffee, that is made with great coffee beans, carefully roasted, prepared in a very good coffee machine and by a great barista. Great baristas are proud of their ability to prepare a great coffee and they would not work for too long for a business where coffee making is not treated with the deserved respect.

Now imagine that you are the barista. But instead of coffee, you produce code. Imagine that the the coffee grains, milk and coffee machine are the tools you use to produce your code like the computer language, the IDE, the database, etc. Instead of serving coffee for your customers, you are producing software that will be used by your team mates, project sponsors, the company they (or you) work for, external clients, etc.

Like in the flat white coffee example above, of course that the tools we use are very important when producing a good software. However, more importantly, it is the quality of the software engineers that counts. A good barista can make a good coffee even when using average coffee grains, due to his or her ability to combine ingredients and prepare the drink. A bad barista can ruin the coffee even if he or she is using the best quality coffee beans. The biggest difference between the two baristas is how much they care about each cup of flat white coffee they prepare. Their pride and willingness to achieve the best pattern on top of each drink. The great feeling of achievement when they produce a great one. The happiness to see returning customers, queuing and waiting their turn to order the coffee that he, skilfully, prepares.

So, which type of barista are you?

One team, one language

2010-06-23T12:52:00.000+01:00

On a previous post I was discussing, among other things, how code often doesn't represent the business properly. The code "satisfies" the business requirements but doesn't express them very well. The main reason for that is because we, developers, like to abstract business terms and rules into technical implementations and patterns.

Very often, we discuss the user stories (requirement documents, use cases, whatever the methodology used is) with the "domain experts" and as soon as we understand what needs to be done, we map the requirements to a technical design (actions, services, entities, helpers, DAOs, etc) that is completely meaningless to the domain experts. Sometimes, even among developers themselves, different names and expressions are used to refer to the same thing. The main reason is that different developers talk to different domain experts and come up with different abstractions. As a result, the usage of different terms to describe requirements leads to confusion, duplication of code and unpredictable behaviour in the system.

The first step towards an expressive and domain-focused design is to have a common language among ALL members of the team. ALL means ALL: developers, domain experts (business analysts, users, product owner, etc), testers, project manager and anyone else involved in the project.

Developers very often say that domain experts don't understand objects and database and because of that, they need to "translate" business requirements into software design. However, we developers don't understand the business as well as the domain experts do, what more often than not, leads to imperfect and confusing abstractions.

The Ubiquitous Language

A language structured around the domain model and used by all team members to connect all the activities of the team with the software.

The Ubiquitous Language is one of the most important things, if not the most, in Domain-Driven Design. The main idea is that the whole team speaks a single language, that is the business language.

Business terms related to the software to be implemented must enter the ubiquitous language and each of these terms must be understood clearly by all members. During requirements gathering sessions and planning meetings, these terms must be captured and made available to everybody. Technical terms from the development team and business terms not relevant for the piece of software being implemented MUST NOT enter the ubiquitous language.

Capturing the ubiquitous language

In order to capture the ubiquitous language, it is mandatory that you work on a iterative software development environment. Trying to capture the ubiquitous language up-front could straitjacket the whole process, inhibiting team members to make the necessary changes along the way. As the language is used to express the business requirements, it is natural that it evolves during the lifetime of the project, where new terms are added, deleted and also re-defined.

Methodologies like Extreme Programming (XP) says that the only documentation should be the code. Not even comments on the code are appreciated, since they can easily get out of sync with the code. The code should be the only documentation since it is the only one that represents exactly what the system does.

On the other hands, we have UML (Unified Modeling Language), that in theory, should be a great candidate to document the ubiquitous language since the whole purpose of UML was to document requirements and express them in a language that is common to developers and business people.

In summary, showing code during discussions with domain experts, testers and other members of the team during a design session is not exactly a fantastic idea. Also, using just UML, because of its bureaucracy, rules and details is also a bad idea. The whole UML notation could easily straitjacket the creative process during the exercise.

There are many discussions about what would be the best way to capture the ubiquitous language. My preferred way is to draw diagrams (boxes and arrows mixed with some well understood UMLish notation) where each box represent a "domain object" (aka domain concept). A domain object can be anything that is expressed by the business, like client, organisation, product, route specification, sales system, etc. They would all be boxes. Add to it a few arrows linking the boxes and with just a couple of words explaining how they related to each other. Sometimes a mixture of a class and sequence diagram (or an active diagram) can be very helpful, but don't get to picky about any notation. Preferably, draw on a white board, take a picture and store that on the wiki. For further sessions, just open the wiki and re-draw just the bit of the design that is important to the feature being discussed. Make the necessary adjusts on the white board, take another picture and stored it on the wiki again. There is much more to that, if you want to dive into agile modeling, but I will leave it to another post.

This goes way beyond transforming nouns and verbs into classes and methods. Taking the examples above, for example, client, organisation and products could be transformed in entities; route specification could be a strategy class used by a routing service (that would also need to be added to the diagram and to the ubiquitous language); sales system would be an external system that we need to integrate to, etc.

Making the code more expressive

The code should reflect all concepts exposed by the model. Classes and methods should be named according to the names defined by the domain. Associations, compositions, aggregations and sometimes even inheritances should be extracted from the model.

Sometimes, during implementation, we realise that some of the domain concepts discussed and added to the model don't actually fit well together and some changes are necessary. When it happens, developers should discuss the problems and/or limitations with the domain experts and refactor the domain model in order to favour a more precise implementation, without ever distorting the business significance of the design.

Ultimately, the code is the most important artefact of a software project and it needs to work efficiently. Regardless of what many experts in the subject say, code implementation will have some impact on the design. However, we need to be careful and very selective about which aspects of the code can influence design changes. As a rule, try as much as you can to never let technical frameworks limitations influence your design, but as we know, every rule has exceptions.

A common implementation problem that very often get in the way is mapping objects to databases using ORM tools. In this case, bending the model a little bit in favour of a more realistic relation among entities is not a bad thing. Just make sure that changes like that are represented in the model and understood by everyone involved.

DOs and DON'Ts

Don't try to model everything. Focus on the core of your application. We are not working on a waterfall or Unified Process project here.
Try to model just the key concepts of the domain problem;
Do not clutter your models with too much details. Keep it focused on the main responsibilities;
Limit your discussions and changes in the model to the business concepts (domain objects) related to the user story being discussed;
Don't add architectural concepts like DAOs, Actions, etc. We are not writing implementation diagrams.
As your application grows, break the application into multiple models (domains), explicitly defining the context and boundaries within which a model applies. This is called Bounded Context in Domain-Driven Design.
Avoid thinking purely on the implementation when designing your model. Understanding and modeling the business is the most important thing here.

Challenges of a Model-Driven Design

Model-Driven Design (MDD) is more an art than a science. It takes a lot of practice and willingness to get it going and get it right. Refactoring towards deeper insights must be seen as a positive and essential part of the project development. TDD and continuous integration are also essential for any agile and domain-driven application.

The goal of MDD is having the software expressing a deep and supple design.

Last buy not least, developers with good Object-Oriented Design skills are needed in the project. The lack of design skills could easily transform ANY software project in a total failure in the long term.

Source

http://domaindrivendesign.org/

http://domaindrivendesign.org/node/91|

http://domaindrivendesign.org/node/132

http://domaindrivendesign.org/node/121

http://www.agilemodeling.com/

The Wolf in Sheep's Clothing

2010-06-12T03:51:00.000+01:00

In the last few years, I've noticed that the majority of the projects that I've participated roughly followed the same design. Speaking to colleagues and friends, the great majority said that they were also following the same design. It's what I call "ASD" design (Action-Service-DAO).

by Action we mean a Struts Action, Spring Controller, JSF backing bean, etc. Any class that handles an action triggered by the user interface.

We can argue that there is nothing really wrong about this approach since if we map that to a three-tier architecture, that is followed by the majority of the applications, the ASD classes would be in the right places, keeping a good isolation between the tiers.

One of the problems of this approach is when services are created randomly, with different granularities, low cohesion and with a weak representation of the business domains. Services end up being function libraries, almost like an utility class where all "functions" related to a specified "module" are grouped. When it comes to DAOs, the situation is not very different. DAOs are developed in a way that they become utility classes where sometimes we find a single DAO with all queries, inserts, delete, updates for an entire module or sometimes you find one DAO per entity. Either way, regardless what the query returns (or what it is its intention), or any rules related to updates or deletes, the methods will go to the same class. As the application grows, the code becomes something like this:

Looking at the picture above, can we really say that it is object-oriented programming? As services and DAOs don't strongly represent business concepts, the code becomes procedural. I don't want to bang on about the advantages of OOP over Procedural code. I believe that too many people have already discussed that over the past 30 years. My point here is to discuss what a "design" like that can do to an application.

Following bad examples

Unfortunately, in software development, bad examples are easier to be followed than good examples. With a non-expressive business model like that, the services get overloaded with methods. When adding new features to the application, developers need to go through all the methods of a service to see if there is already a method that does what they need. Due to the lack of cohesion and method explosion in the services, many developers will just add a new method in there, that can be very similar to existing ones, instead of re-factor the existing ones. This leads to services with confusingly similar methods and a lot of duplication. As a chain of "bad" events, the more methods a service has, the more classes depending on it the application will have. Many dependencies means that re-factoring the class would be harder, discouraging any one willing to improve the quality of the code.

Duplication does not happen just inside a single service. It's also very common to find different services with very similar methods, if not the same. In general, this is due for the lack of clarity on the responsibility of each service. According to the feature that developers are working on, they will choose a service to add (or reuse) the methods they need. If the responsibility and granularity of each service is not clear, business rules related to a certain area of the business will be spread all over the place since different developers will think that the method will belong to different classes.

Exposing DAOs to the presentation tier

Another side effect of this approach is that since the DAOs are also "utility classes" with no business meaning (it just has an architectural meaning), some developers can easily expose the DAOs to the actions, without going through the services. Let's give more meaningful names and more details about the Action 4 and DAO 4 shown above. Let's call them ClientAction and ClientDAO.

Here the user interface needs to display a list of clients. ClientDAO has a method that returns a list of clients. In theory this may make sense. The most used argument in favour of this approach instead of adding a "ClientService" in between the ClientAction and the ClientDAO is that the service would have a method that just delegates the call to the DAO. The service layer, in this case, would be considered redundant and just pointless extra work.

Looking at this isolated scenario, having a service in the middle really looks a bit overkill and unnecessary. However, we will probably want to do more things with a client. We will probably want to create a new client, delete or update an existing client.

Here is where the problems begin. Very rarely, we have a pure CRUD application. In general, many business rules have to be performed before or after any changes are made in the persistence tier (generally a database). For example, before inserting a new client, a credit check needs to be performed. When deleting a new client, we need to make sure that there is no outstanding balance for that client and close his or her account. We also need to archive all the orders for this client. Whatever the application's business rules dictate. Multiple entities (or database operations) may be involved in a operation like that. We need to be able to define transaction boundaries. Besides CRUD operations, we will also have all business methods like check if client has credit, add orders to a client, etc. This is too much for a single DAO to handle. DAOs should not perform any business logic and should be hidden from the presentation tier. It should be hidden (encapsulated) even from different services. DAOs should just deal with the persistence. Business logic and transaction boundaries should be controlled by a class with business responsibilities, that in the case of this "poor" design, it would be a service.

So even having a few methods in the service that just delegate the responsibility to a DAO, it is still a price worth paying. The service should hide (encapsulate) the details of it's implementation. A client code does not need to know how and where the data is persisted.

OK, enough of this. Exposing DAOs to the presentation tier is WRONG. Let's move on.

Moving away from the procedural code (baby steps)

The first thing to do to move away from the procedural code is to make your code more expressive and aligned to the business. We need to narrow the gap between developers and domain experts (business analysts, users, product owner or whoever knows the business rules) so we all start speaking the same language. The code must be written in a way that it expresses the business and not just architectural concepts that means nothing to the business. Actions, Services and DAOs are technical terms with no connection to any business term.

There are a few techniques that can be applied (or even combined) in order to make it possible. In future posts, I'll be talking about some of these techniques in more details. I know, it's frustrating that I will not tell you right now how to do it after have spending all this time criticising the ASD procedural design. The important thing for now is to know that what looks a good solution, since it is used by many different people and in many different projects, may not be as good as it seems.

In the meantime, there is something that we can start doing to our code in order to reduce the mess and make the necessary refactorings easier in the future.

Preparing your code for an easier transition

The following advices will help us to get our code to a point where it can be easily refactored into a more domain (business) focused approach in the future. It will still be a bit procedural but will be much more well organised and a notion of components (business components) will start to emerge.

1. Identify key concepts (domains) in your application. (e.g. Client, Order, Invoice, Trip, Itinerary, etc)
2. You can create / refactor services for each one of the key domains.
3. Try to keep a well balanced granularity for all services.
4. Avoid services for small parts of the domain. For example, do not create a service for line items. Line items should be handled by the OrderService.
5. Services are not allowed to manipulate multiple domains (with exceptions of whole-part relations - compositions)
6. DAOs are encapsulated by the services and never accessed by any other class.
7. Not every service must access a DAO, but any DAO must be accessed by one and only one service.
8. Services must delegate operations to other services if the operation is related to a different domain from the one the service is handling.
9. Services talk to each other. DAOs never talk to each other.
10. Parts (like a line item) are never exposed by a service. Service always exposes the whole (like Order). Parts are accessed via the whole. (e.g. order.getLineItems();)

With the following rules, our procedural code starts looking more like meaningful objects, with a reasonably well defined interface and some significance in terms of business.

As mentioned before, this is still a bit procedural but this design already solve a some of the problems discussed earlier like having more meaningful and specific services. The responsibility and boundaries of each service are more defined, making it easier for developers to look for implemented methods and re-use what it is in there, reducing duplication.

In future posts I'll be talking about the next steps towards a more expressive and business focused code, less procedural and more object-oriented.

If you are curious about the next steps and can't wait for my posts, have a look at:
http://domaindrivendesign.org/
http://en.wikipedia.org/wiki/Behavior_Driven_Development
http://en.wikipedia.org/wiki/Model-Driven_Architecture
http://en.wikipedia.org/wiki/OOD

Object-Oriented Design Principles - Part 2

2010-06-06T19:22:00.000+01:00

In the first part of this Object-Oriented Design principles series, I covered the Single Responsibility Principle (SRP). I had also covered the Liskov Substitution Principle (LSP) on a previous post. So let's move on to another Object-Oriented Design Principle.

The Open Closed Principle (OCP)

Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification

We know that requirements will change, new requirements will come and we will need to change some existing code during the life of a project. However, when a change in one place results in a cascade of changes in other classes, that's a sign that the design is not quite right. Changes causing side effects are undesirable since it makes the program unstable, rigid and fragile where parts can not be re-used.

The Open Closed Principle target all problems described above. Being open for extension means that modules can be extended in order to make them behave in a new or different way. Closed for modification means that we should not change existing code unless that we are fixing a bug.

There were two proposed ways to achieve the OCP and both use inheritance:

Dr. Bertrand Meyer, that coined the Open Closed Principle in 1988, proposes the use of inheritance. When a new feature or a changing on an existing feature is needed, a new class must be created, inheriting from the old one. The new class does not necessarily keep the same interface.
A few other authors redefined the OCP in the 90ies. Basically they suggested the use of abstract base classes and concrete implementations. The abstract class would define the interface that would be closed to modifications. The concrete classes would implement the abstract class interface (open for extension) and multiple implementations could be created and polymorphically substituted for each other.

Nowadays, with the evolution of frameworks with dependency injection capabilities, a better approach would be having a client class pointing to an plain Java interface and inject the implementations.

OCP is an old OOP design principle and is one of the most important ones, due to the problems it solves. It can be an interesting approach in cases where the bureaucracy of changing old code and deployment to the production environment is very high. Some companies may ask for code reviews, a long test cycle (non-automated), documentation, etc. With OCP, no existing code is changed and just new code is added.

However, in a more agile environment, good test coverage, IDEs that have good re-factoring tools (like Eclipse, Idea, etc) I wouldn't be too worried about changing classes and interfaces but this would not invalidate all the advantages of the OCP.

Source
http://en.wikipedia.org/wiki/Solid_%28object-oriented_design%29
http://en.wikipedia.org/wiki/Open/closed_principle
http://www.objectmentor.com/resources/articles/ocp.pdf
http://en.wikipedia.org/wiki/Design_by_Contract
http://en.wikipedia.org/wiki/Information_hiding

Object-Oriented Design Principles - Part 1

2010-06-03T16:21:00.000+01:00

Almost every developer that I know would be able to give a reasonable explanation about inheritance, encapsulation and polymorphism. However, there is much more to Object-Oriented Programming (OOP) than that. In order to come up with a good and clean design, we need to bear in mind some Object-Oriented Design (OOD) principles. Although many of these principles were already published in books and blogs, they are known to just a very small percentage of developers. I don't want to show my age here, but very rarely we find young developers talking about OOD Principles.

There are quite a few OOD principles out there, created/coined by different developers and academics, but I will be listing here just the OOD principles that I consider to be the most important ones.

The following 5 principles together are known by the mnemonic acronym "SOLID". They were first put together by Robert C. Martin (Uncle Bob) in the early 2000s and should be applied at class level. They are:

SRP — The Single Responsibility Principle
OCP — The Open Closed Principle
LSP — The Liskov Substitution Principle
ISP — The Interface Segregation Principle
DIP — The Dependency Inversion Principle

The Single Responsibility Principle (SRP)

There should never be more than one reason for a class to change.

The SRP is the simplest OOD Principle and probably one of the hardest to get right. It is also one of the most violated principles.

Let's have a look at the following class:

public class TripItinerary {
    public void addPlace(Place place) { ... }
    public void removePlace(Place place) { ... }
    public List<Place> getPlaces() { ... }
    public void displayOnMap() { ... }
    public Place findPlaceByName(String placeName) { ... }
}

The class above has 3 different responsibilities, that means, 3 different reasons to change:

Store the places to be visited.
Display the itinerary on a map.
Finds a place by name.

Storing places on the itinerary may have rules like not adding repeated places, keeping the sequence that places will be visited, etc.
Displaying on the map may vary according to the map API being used like Google Map, Bing Map, Yahoo Maps, etc.
Find a place by name may involve calling a web-services to see if the place exists, if there are more than one place with the same name, checking the type of place (city, town, waterfall, monument, castle, etc).

Ideally we would have three different classes to do that, each one with its own responsibility.

public class TripItinerary {
    public void addPlace(Place place) { ... }
    public void removePlace(Place place) { ... }
    public List<Place> getPlaces() { ... }
}

public class ItineraryMapService {
    public void displayOnMap(TripItinerary tripItinerary) { ... }
}

public class PlaceService {
    public List<Place> findPlaceByName(String placeName) { ... }
}

With these approach, we can change the internals of all classes without having the risk of breaking any of the behaviour of the other classes. Without this separation, the design becomes fragile and might break in unexpected ways when changed.

When thinking about a single responsibility, think cohesion at class level. Before creating a class, we need to define what its responsibility should be and the reason for its existence. Before adding any other public method to an existing class, check if this new method relates to the other public methods (the class interface). When creating public methods for a class, have them at least at a communicational cohesion level.

On the next posts, I'll be covering the remaining SOLID OOD principles.

Object-Oriented Design Principles - Part 2

Source
http://en.wikipedia.org/wiki/Solid_%28object-oriented_design%29
http://en.wikipedia.org/wiki/Single_responsibility_principle
http://www.objectmentor.com/resources/articles/srp.pdf

The Liskov Substitution Principle (LSP)

2010-06-02T01:50:00.000+01:00

The Liskov Substitution Principle was initially introduced by Barbara Liskov in a 1987 conference keynote address entitled Data abstraction and hierarchy. LSP is also part of SOLID, a group of five Object-Oriented Design Principles put together by Robert C. Martin in the early 2000s.

Functions that use pointers or references to base classes must be able to use objects of derived classes without knowing it.

The LSP's summary above looks quite obvious. If the calling code was written to use a base class, replacing it with a sub-class (inheritance) should make no difference to the calling code. That means, the calling code should not be changed and should be totally agnostic about which implementation is being used.

LSP's importance is noticed mainly when it is violated. If a subclass causes changes on the calling code, that means, the calling code needs to test which subclass it is dealing with (instanceof, casting, etc), the code is violating the Liskov Substitution Principle and also the Open Closed Principle. This violation causes high coupling, low cohesion and a cascade of changes.

Violation of Liskov Substitution Principle

public void drawShape(Shape s) {
    if (s instanceof Square) {
        drawSquare((Square) s);
    } else if (s instanceof Circle){
        drawCircle((Circle) s);
    }
}

The Liskov Substitution Principle also imposes a few rules that the sub-classes must obey. It bears a certain resemblance with Bertrand Meyer's Design by Contract in that it considers the interaction of subtyping with pre- and postconditions

...when redefining a routine [in a derivative], you may only replace its precondition by a weaker one, and its postcondition by a stronger one.

This means the sub-classes must accept everything that the base class accepts (pre-condition) and must conform to all postconditions of the base class.

Example of a more subtle violation

public class Rectangle {
    private int height;
    private int width;

    public Rectangle(int height, int width) {
        this.height = height;
        this.width = width;
    }
    public int getHeight() {
        return this.height;
    }
    public void setHeight(int height) {
        this.height = height;
    }
    public int getWidth() {
        return this.width;
    }
    public void setWidth(int width) {
        this.width = width;
    }
}

public class Square extends Rectangle {

    public Square(int size) {
        super(size, size);
    }  
    public int getHeight() {
        return super.height;
    }
    public int getWidth() {
        return super.width;
    }
}

In the code above, a Square IS A Rectangle. Note that a rectangle can have different sizes for width and height, but in a square, height and width must be of the same size. What would happen in the following code is executed ?

Rectangle r = new Square();
r.setHeight(5);
r.setWidth(6);

We should not allow this to happen since that would make the Square object invalid. In a square, height and width must always be the same. A quick fix for this would be to override the setter methods:

public class Square extends Rectangle {

    public Square(int size) {
        super(size, size);
    }  
    public int getHeight() {
        return super.height;
    }
    public int getWidth() {
        return super.width;
    }
    public void setHeight(int height) {
        super.height = height;
        super.width = width;
    }
    public void setWidth(int width) {
        super.width = width;
        super.height = width;
    }
}

Although this approach would fix the problem, is a violation of the Liskov Substitution Principle since the methods will weaken (violate) the postconditions for the Rectangle setters, which state that dimensions can be modified independently.

One interesting thing to note is that if we analyse the Rectangle and Square classes in isolation, they are consistent and valid. However, when we look at how the classes can be used by a client program, we realise that the model is broken.

Every time you get yourself adapting a sub-class so that it does not break or its state does not become invalid if used in the context of a super-class, this is a clue that the sub-class should not be a sub-class at all.

In this example, maybe a Square is not a Rectangle. Square should not have height and width to start with. It should just have size. And since a Square is not a Rectangle, it should never be used in a Rectangle context.

public class Square {
    int size;
    
    public Square(int size) {
        this.size = size;
    }  
    public int getSize() {
        return this.size;
    }
    public void setSize(int size) {
        this.size = size;
    }
}

We can not validate a model in isolation. A model should be validated just in terms of its clients.

Source
http://en.wikipedia.org/wiki/Liskov_substitution_principle
http://www.objectmentor.com/resources/articles/lsp.pdf

Empowering your entities

2010-05-24T23:44:00.000+01:00

The first thing I learnt when I started with Object-Oriented Programming was that an object should have state and behaviour. And indeed, that's how it was back then. However, since J2EE (Entity Beans) and other ORM tools like Hibernate, iBATIS, TopLink, JPA, etc, everything changed. We were so focused in mapping tables to objects, configure the relationships, lazy loading, eager fetching, primary keys and everything else that you can map and configure on the ORM tools that we simply forgot that those entities could and should also have some behaviour.

In general, for applications that use a database, the entities are the key focus of the application. They are the main components of an application's core domain and their names and methods should represent, and also help to define, a common language used by all members of the team and not just developers (See Ubiquitous Language - Domain Driven Design).

Let's see a few situations where your entities could be empowered.

1. Default values / initialisation

Let's have a look at a fairly simple piece of code and see how many problems it may cause:

Listing 1.1

// Somewhere in the code 
DiscussionGroup dicussionGroup = new DiscussionGroup();
discussionGroup.setCreationDate(new Date());
discussionGroup.setLastUpdate(new Date()); 
discussionGroup.setAllowAnonymousPosts(false);

// Somewhere else in the code
if (discussionGroup.getAccess().equals(Access.PUBLIC) {
   this.displayDiscussionGroupInfo(discussionGroup);
};

Members members = discussionGroup.getMembers();
for (Member member : members) {
     // Do something
}

This code above has the following problems:

creationDate and lastUpdate setters being called right after creation. Let's assume both attributes are mandatory on the database. Wherever in the code a DiscussionGroup is created, we need to remember to set them both. In Failing to do so, we will get an error when persisting the entity.
The check to verify if the discussion group is public may throw a NullPointerException if the attribute is not initialised.
Like the access check, the iteration over the discussion group members can also fail if the list of members is not initialised.
The calling code needs to know about the internals of the entity and cater for that, like setting values to mandatory fields and also do null checks for some attributes.

To solve the problems, just initialise all the attributes you can, I mean, the ones where a default value would make sense. Always avoid having getters returning null.

Listing 1.2

// DiscussionGroup.class
public class DiscussionGroup {
    private Date creationDate;
    private Date lastUpdate;
    private boolean allowAnonymousPosts;
    private Access access;
    private List<Member> members;
    
    public DiscussionGroup() {
        this.creationDate = new Date();
        this.lastUpdate = new Date();
        this.allowAnonymousPosts = false;
        this.access = Access.PRIVATE;
        this.members = new ArrayList<Member>();    
    }
}

2. Operation that just involves an entity's attributes.

Look at the code below.

Listing 2.1

// OrderService.class
if (order.getTotal() > 0 
        && order.getOrderItems().size() > 0
        && order.getAuthorizationDate() != null 
        && order.getClient() != null) {
    this.processOrder(order);
}

The if statement checks attributes from the order object so that it can decide if the order can be processed or not. Note that this logic is outside the order object, making the two classes tightly coupled and making a poor use of encapsulation. In this case, we could re-factor this code like that:

Listing 2.2

// Order.class
public boolean isReadyForProcessing() {
    this.getTotal() > 0 
        && this.getOrderItems().size() > 0
        && this.getAuthorizationDate() != null 
        && this.getClient() != null);
}

// OrderService.class - Somewhere in the code
if (order.isReadyForProcessing() {
    this.processOrder(order);
}

This re-factored code makes the code more expressive, easy to read and easy to test. This also makes the code to better represent the business rules.

3. Children manipulation

Look at the listings 3.1 and 3.2.

Listing 3.1

// Add an entry to a diary.
Diary diary = new Diary();
Entry entry = new Entry();
entry.setDate(new Date()); 
entry.setText("Today I went ... ");
diary.getEntries().add(entry);

Listing 3.2

// Remove an entry from a diary.
ListIterator<Entry> entriesIterator = 
        diary.getEntries().listIterator();
Entry entry;
while (entriesIterator.hasNext()) {
    entry = entriesIterator().next();
    if (entry.getDate().equals(someDateVar)) {
        entriesIterator.remove();
    }
}

The main problem with Listing 3.1 and Listing 3.2 is that the parent class Diary is exposing its internals, the entries. Once again, this is a major encapsulation breach. As a principle, the parent should never let a stranger manipulates its children without its consent. Besides all the code duplication that this may cause, in case that I want to add/delete entry in different parts of the application (for some reason), this also makes the code to be very fragile where the parent may break since it was not notified that someone changed its children.

This could be easily fixed if we add this logic to the parent class.

Listing 3.3

// Diary.class
public Entry addEntry(Date date, String text) {
    Entry entry = new Entry();
    entry.setDate(date);
    entry.setText(text);
    this.entries.add(entry);
    return entry;
}

public Entry deleteEntry(Date date) {
    ListIterator<entry> entriesIterator = 
            this.getEntries().listIterator();
    Entry entry;
    while (entriesIterator.hasNext()) {
        entry = entriesIterator().next();
        if (entry.getDate().equals(someDateVar) {
            entriesIterator.remove();
            break;
        }
    }        
    return entry;
}

The calling code would be like:

Listing 3.4

// Add an entry to a diary.
Diary diary = new Diary();
Date date = new Date();
diary.addEntry(date, "Today I went ... ");

// Remove an entry from a diary.
diary.deleteEntry(date);

Conclusion
Entities don't need to be just dumb classes with state and no behaviour. They should contain business logic that is related to their attributes and children. This will promote encapsulation, reduce code duplication, make your code more expressive and easy to read and also much easier to test.

Sources:
http://domaindrivendesign.org/
Domain Driven Design - by Eric Evans
Clean Code - by Robert C. Martin

MVC and Multi-tier architecture

2010-05-09T03:28:00.000+01:00

Over the years, working in different web-based Java projects, I noticed that there is a big confusion about the boundaries and overlaps between MVC, that is an architectural pattern, and a multi-tier architecture (also known as n-tier architecture). The main confusion is in identifying what is controller, what is model and what is application (business) tier. The results of this confusion, to name just a few, are:

Unclear design;
Poor re-usability (loads of copy and paste);
Non-cohesive classes and methods;
Business logic all over the place;
Difficult to test;
Difficult to measure the impact of changes and improvements;
And people's favourite, actions (if using Struts-like web frameworks) with hundreds, if not thousands of lines.

I'll quickly refresh our memories about multi-tier architecture and MVC so that later we can see how they relate to each other.

Multi-tier architecture

Often called n-tier architecture, the multi-tier architecture is a logical way to separate the different responsibilities of your application. The most common multi-tier architecture is the three-tier architecture, which will be the one that I'll be focusing on.

The tree-tier architecture is divided in the following tiers:

Presentation tier: It is responsible to interact with the user, displaying information and providing ways where the user can input data and perform actions.
Application tier: It is responsible for the coordination of the application, its business logic, decisions, calculations and evaluations. It executes commands, actions and moves data between the presentation and data tiers. It is also known as business tier, logic tier or middle tier.
Data tier: It is responsible to retrieve and store data. Data can be stored in a database, xml, file system or even other system. It is also known as persistence tier.

Each tier must be as independent as possible from each other, where a good practice would be to provide interfaces as "facades" to each tier. Organising your classes according to these logical tiers would make the code more cohesive, loosely coupled, easier to understand and easier to test. This approach would also help to improve greatly the re-usability and would make changes to be more localised, minimising the impact on the rest of the code.

A multi-tier architecture would be for applications that access other systems (RPCs, webservices, etc), access multiple sources of data, or uses any sort of middleware.

MVC and Java MVC frameworks.

I will assume that people reading this post already know MVC so I'll give just a brief and generic description of how MVC and its variations like Model 1 and Model 2 work. If you need more information about it, please refer to the links at the end of this post.

The general behaviour of the MVC pattern is:

User performs an action on the view (screen, page). This action can be anything like clicking a button, clicking a link, selecting an item from a drop down list, etc. Data may be submitted, in case of a form.
Controller receives the request or event and invokes the model.
Model will perform some business logic, persist or fetch some data.
Model return the result of this operation to the controller. This result may include some data.
Controller, according to the result from the model, invokes the next view. The next view can be the same one that originated the request or a different one.
View is rendered. The view may display any data returned from the model.

Java MVC frameworks like Struts, Spring MVC and alike are based on what we call MVC Model 2, that is a variation of the original MVC pattern.

Fitting MVC into a Three-tier Architecture

In a traditional java web application, View and Controller will belong to the Presentation tier and Model will belong to the application and data tiers.

So far so good, but when using Java Web frameworks like Struts, Spring MVC, JSF, etc, the catch is to be able to identify what is controller and what is model.

Understanding the role of the "action" classes.

Struts was one of the first and one of the most used Java MVC frameworks. When it was released, back in 2000 (version 1.0 in 2001), many developers did not get the whole MVC Model 2 thing and very quickly started misusing the framework and sacrificing some important architectural patterns. The situation got worse when other frameworks also based on Model 2 were released, since the same bad old habits from Struts were used to develop applications with frameworks like WebWork, Spring MVC, etc.

In Struts, when some action is performed on the page, an "action" class is invoked. This action class is probably the source of the whole problem. What exactly is this action class? I mean, what's the purpose of this class and where does it belong, taking into consideration the MVC pattern and the three-tier application?

Since the View is done by the JSPs, the Controller is done by the servlet (that is configured on the web.xml), many would answer that this class is the model. This would explain why we find so many actions with thousands of lines and full of business logic.

However, the action class DOES NOT belong to the Model.

All requests are handled by the same servlet (Front Controller design pattern), and then the respective action is called (Command design pattern). The result of this action will be the view to be displayed. So, in summary, what is the responsibility of this action class? The action is triggered by the view, does some thing and decides which view will be displayed next. This is exactly what a MVC Controller does, meaning that action classes are also part of the Controller, working almost as an extension or helpers for the main servlet.

When Spring MVC came out, one of the first things that I noticed was that they called the equivalent Struts Action class, "Controller". That's right. In Spring MVC, you create controllers instead of actions, what makes much more sense. However, even naming the classes as controllers, some developers kept adding business logic to them.

In summary, the action class must just invoke the model (could be a service, session bean, business object, etc.), get the result, set it into a context (session, request, etc) and invoke the view. Action classes should be small, clean and without any business logic, as a Controller class should be.

Component-centric frameworks and its "backing bean" classes

In component-centric frameworks like JSF, Tapestry, Wicket and alike, the pages (generally XHTML) have components that are bound to Java classes (known as backing beans in JSF). These components can be input texts, drop down lists, tables, etc, or even the entire page. Basically each component on the page can be bound to a Java class, that would behave like a model and sometimes controller for these components.The backing beans are responsible to hold the state of the components and also handle events, validation, conversions, trigger business logic, update/refresh other components, fire events, listen to events, etc.

Now that we know that Struts Actions and Spring Controllers belong to the Controller part of the MVC, where do the backing bean classes (JSF like) belong to?

When we talk about component-centric web frameworks and also add AJAX into the mix, the line between controller and model becomes a little bit blurred.

The backing bean may handle navigation and in this case, it would act as a controller. When acting as model for its view component, although it is a model, it is a model for that specific view component. That means, the logic that this backing bean should perform would be related to rendering the view component or invoking other view components (events, re-render, etc) and not exactly application business logic, keeping this managed bean coherent. Any application business logic like making calculations, fetching or storing data, make a web service call, etc, should be delegated to a business class belonging to the application tier.

So in the case of a component-centric framework, managed beans would belong to the presentation tier, even being models for view components.

NOTE: All classes related to the java web frameworks like validators, converters, forms, etc, also belong to the Presentation tier.

Summary

When developing applications it is important to keep your code cohesive and loosely coupled. The first step is to make a quick analyses and define the logical tiers. In case of a web application with database access, it will not differ too much from a three-tier architecture. If integrating with other systems or accessing multiple data sources, application and data tiers may be broken down into more tiers.

When using Java web frameworks, regardless if they are page-centric (Struts like) or component-centric (JSF like), chances are that everything related to the framework (forms, converters, validators, actions, controllers, managed beans, etc) will belong to the presentation tier and should not have business logic. Business logic and data access should be delegated from the presentation tier to the application tier. This would allow us to keep our managed beans and actions (controllers) very small and clean.

In case of too much view logic (enabling/disabling components, populating tables and drop down lists, validations, etc), use helper classes for the actions and managed beans (see View Helper design pattern)

For the model, application and data tier, they will be covered in future posts since they can vary a lot from application to application.

Source
http://en.wikipedia.org/wiki/Multitier_architecture
http://en.wikipedia.org/wiki/Model-view-controller
http://java.sun.com/blueprints/patterns/MVC-detailed.html
http://www.ibm.com/developerworks/library/j-jsf1/
http://www.javaworld.com/javaworld/jw-12-1999/jw-12-ssj-jspmvc.html
http://java.sun.com/developer/technicalArticles/J2EE/despat/
http://java.sun.com/blueprints/corej2eepatterns/Patterns/ViewHelper.html

Cohesion - The cornerstone of OO

2010-04-24T02:45:00.000+01:00

Cohesion is probably the most important concept of Object-Oriented Programming since it promotes a good level of encapsulation, separation of concerns and responsibilities, re-usability and maintainability.

Definition

Cohesion (noun) : when the members of a group or society are united.
Cohesive (adjective) : united and working together effectively.

Cambridge Dictionary

In computer programming, cohesion is a measure of how strongly-related and focused the various responsibilities of a software module are.

Wikipedia

Cohesion at method level

Coincidental (worst): Performs multiple operations and some times they are not related to each other.
Conditional: According to an if statement, different attributes are modified or different values are set to the different attributes.
Iterative: Several attributes (Array variables) are modified as a result of a looping.
Communicational: More than one attribute is modified according to only one input.
Sequential: More than one variable (object) modification result in the change to only one attribute.
Functional (best): Method modifies fewer than 2 attributes.

Cohesion at class level

Coincidental (worst): Methods grouped arbitrarily and have no significant relationship (E.g. Util classes with methods handling strings, lists, mathematical calculations)
Logical: Methods grouped because they logically are categorized to do the same thing, even if different by nature (E.g. grouping all I/O handling routine, all database selects, inserts, etc.).
Temporal: Methods grouped by when they are processed at a particular time in program execution (E.g. validates data, persist the data, create audit information, notifies user via email).
Procedural: Methods grouped because they always follow a certain sequence of execution. (Verify is user exist, performs login, write logs, retrieve user's detail)
Communicational: Methods grouped because they work on the same data (E.g. Insert, delete, validate, update a client entity).
Sequential: Methods grouped because the output of one method can be used as an input of other methods. (reads a file, process the file).
Functional (best): Methods grouped because they all contribute to a single well-defined task. (parsing XML)

Applying cohesion in the real world

Depending of the type of software you are writing, you will need to compromise a little bit. It is not always possible to have all methods and classes at the highest cohesion level (functional).

If you are building a framework or a very generic part of your system, chances are that the majority of your classes and methods will be at sequential and functional levels. However, when writing a more commercial application, I mean, an application where there are business logic, database access, users, etc, there is a good chance that many of your classes and methods will be more at the communicational level.

In a more simplistic way, each class and each method should have a single responsibility. A technique that I use for that is to write a brief description (javadoc) for each class and method before writing the methods of the class or the body of the method. This forces me to think what the responsibility of the class or method that I'm creating is and as soon as I realise that the class or method is doing more than what I described, I know that I need to break it down in more classes or methods (private or public ones).

Some people use other criteria, like number of public methods per class or number of lines per method. This sort of metrics are helpful because it makes you re-analyse your code and can be a good indicator that something is not quite right. A class with many public methods is an indication that the class may be doing too much and does not have a single responsibility, having a low cohesion. A method with many lines is also an indication that this method may be doing too much. One of the problems with this approach as a cohesion measure is knowing how much is too much. Is 10 lines of code per method too much? What about 20? Is 10 public methods in a class too much? Number of methods per class or number of lines of code per method don't necessarily tells much about how cohesive the class or method is, but they can be used as a "smell detector". Thinking on a single purpose for each class and each method before you implement them will help you to keep your classes and methods small without much effort.

The more cohesive your code is, the more reusable, robust, easy to test and reliable it is.

Source
http://en.wikipedia.org/wiki/Cohesion_%28computer_science%29
http://www.waysys.com/ws_content_bl_pgssd_ch06.html
http://en.wikipedia.org/wiki/Single_responsibility_principle