Wednesday 15 August 2012

The best approach to software development

Today, talking about doing a big design up-front (BDUF) sounds a bit ridiculous, right? Who would do that? That's not craftsmanship, is it?

However, in the past, that would be considered the norm. Writing requirement documents, drawing architectural and very low level detail diagrams was the right thing to do. Well, that’s what very smart guys proposed on the 1968 NATO Software Engineering Conference and it worked for NASA and the US Department of Defense. I’m sure they know what they are doing and if it works for them, it will definitely work for our small CRUD application or one page website. And then it happened. It became a religion and the majority of projects in the following decades were developed like that.

No, but not nowadays. We've learned the lesson, right? We wouldn't make this mistake again.  

After watching a few talks in conferences and InfoQ, we understood that this is not a good thing. We’ve also read in some books that we should do TDD. The design should emerge from tests.

And of course, we should adopt an Agile methodology as well. Let’s adopt Scrum to start with. There are many books about it, certifications and even entire conferences dedicated to it. Of course we should adopt TDD an Scrum because that’s the best approach to manage and develop and software.

Oh, but what about all this lean stuff? Eliminate waste, limit work in progress, system thinking, theory of constrains, Kanban. I heard it worked really well for Toyota so we should definitely do that as well. Why? Jesus, you just don’t get it. Of course that’s best approach to manage and develop software.

God, how could I forget? I was also told that I really should speak to my customer. We should really discuss the requirements so we can have a better understanding of what we should build. Are you using BDD? No!!! Wow! How can you develop software without that? Why should you use it? The answer is simple. That’s the best approach to manage and develop software. Don’t tell me that you thought that BDD was a testing tool. You are so yesterday. That was version one. BDD version three is all about communication. It's about software development done right. Yes, I know it sounds alien but apparently we are supposed to speak to people. God, how on Earth we haven’t thought about that before? How did we develop software all these years? If you don’t use BDD, you are just doing it wrong. Why? Because that’s the best approach to manage and develop software. Duh!

Outside-In TDD, Inside-Out TDD, ATDD, Classic TDD, London School TDD? Really? Are you still discussing that? Don’t tell me that we you are still writing tests. What? Why are you wasting time writing unit tests? It doesn’t make sense any more. You should spike and stabilize. What if you don’t know what you are doing or where you are going? What if you just want to explore your options? Why are you writing tests? Oh, I get it. You were told that this was the best approach to manage and develop software. Nah, forget that. Unit tests are for losers. We write small services and just monitor them. If they are wrong, we just throw them away and re-write. And THAT is the best way to manage and develop software.

Architecture and design patterns? What??? Who are you? My grandfather? Scrap that. That’s for programmers from the 80ies and 90ies. In the real world we have our design emerging from tests. No, stupid. Not using normal TDD. God, which planet do you live? We use TDD As If You Meant It. We use this technique and PRESTO, the design emerges and evolves nicely throughout the life span of the project regardless of how many developers, teams and design skills. Every one can see code smells, right?

And what about DDD? Domain Driven what? Nah, never heard about it. Hmm.. hold on. I think I heard something about it many years ago, but probably it was not important enough otherwise we would have more people today saying that DDD is the best approach to manage and develop software.

Noooo. No no no no. No, I didn't hear that. Get out of here. Did you just say that you are still using an Object-Oriented language? STATICALLY-TYPED???? No, sorry. This conversation is a waste of my time. It's because of people like you that our industry is shit. The only way for this conversation to get even worse is if you tell me that you still use a relational database. Haven't you heard that functional programming is a MUST DO and just retards and programmers from the 80ies and 90ies use relational databases. Functional languages, NoSQL databases... Repeat with me. Functional languages, NoSQL databases. What a douche bag.

Ah, trying to be a smart ass? Yeah, functional programming appeared long ago in the 50ies and developers in the 60ies and 70ies preferred to use OO instead. But you don't know why, do you? DO YOU? They did use OO because they were a bunch of hippies that didn't take anything seriously. They were that sort of people that went to Woodstock, got high on LSD and had brain damage after that. No, don't take this whole OO stuff seriously. We are finally getting back to reality now. Functional programming and NoSQL databases are definitely the best approach for software development.

Dogmatism, religion, context, curiosity, inquiring mind and pragmatism

Before I conclude, I just want to clarify that by no means I'm criticizing any person or group of people behind some of the methodologies, technologies or techniques mentioned above. These people have done an amazing job thinking, practicing and sharing their own ideas of how software development could be done in a better way and for that we should all be grateful. Our industry if definitely better with their contribution.

My main criticism here is about how the vast majority of developers react to all these things. It is not just because someone, somewhere wrote a book, recorded a video or gave a talk in a conference about something that it will make that thing right, in all contexts. Quite often, we fail to question things just because the people promoting it are relatively well known. We fail to understand the context where a methodology, technology or technique should be best suitable for. We fail, quite often, to use our own judgement because of the fear to be ridiculed by our colleagues. We should stop being dogmatic and religious about things. This just leads to stupid decisions. Doing things for the sake of doing or because someone else said so is just plain wrong and stupid. 

Being a good developer means to be inquisitive, curious and pragmatic. Never religious. Never dogmatic. Curious means that we should be eager to learn about all the things mentioned above and many many more. Inquisitive means that we should investigate and question all the things we learn. Pragmatic means that we should choose the right tools, being technologies, methodologies or techniques, for the job.

Context matters. 

Whenever you see people saying that we should or shouldn't do something, ask them why. Ask them about the context where they tried to do (or not to do) what they are saying. 

Software development is not the same thing of producing cars. Once the car is ready, you don't go back to the manufacturer and ask them to add another wheel or put the engine somewhere else. Developing software for expensive hardware is not the same thing as developing a simple web application with two pages. Hardware has an specification that you need to code against. Quite often, you don't even have access to the hardware because it is just not built yet. The cost of a bug in production is not the same for all applications. The cost of a few bugs in a social networking or cooking website can be very different from the cost of a few bugs in a trading or financial system processing millions of transactions per day. Working with a small team, every one co-located and with easy access to customers is very different from working on a project with 10+ teams spread in 5 countries and different timezones. 

Read and learn as much as you can. However, don't assume that everything you read or watch applies in every context. Make informed decisions and trust your instincts.

The bad news is that there is no best approach to software development. Maximum we could say is that there are certain technologies, methodologies and techniques that are more suitable to a specific context.

In case you are really trying to find the best approach to software development in general, I hope you don't get too frustrated and good luck with that. If you ever find it, please let us know. It's always interesting to know about different approaches. Maybe unicorns really exist. Who knows?

This post was inspired by conversations during many London Software Craftsmanship Community (LSCC) Round-table meetings, conversations during SoCraTes 2012 and also conversations with colleagues at work. Thanks everyone for that.


Unknown said...

Sounds like my internal dialogue before I begin any new piece of work! Definitely agree that we need more pragmatism but, at the risk of introducing more dogma, I think we also need better ways of navigating the plethora of options. And I think underlying all this is the fact that, historically, our industry has been perceived as judgemental and intolerant of ignorance. As we improve that so we enable more personal pragmatism.

Unknown said...

I have a different opinion. What I see is pragmatism becoming the new dogma. An excuse to shutdown conversations rather than open them up. What you deride as the constant change in emphasis is for me proof that we're doing something right. Allowing mutation to be the driver of evolution. We put out our ideas for peer review and the benefit of others. Subsequently they may get adapted, adopted or abandoned. We may need to get better at understanding what would success look like. But that's come to me from exposure to system thinking.

Also I'd really like to caution against the fashion for deriding the Lisp and Smalltalk communities. Via a roundabout route including GOOS I discovered former Smalltalkers like David West and Sandi Metz who describe eloquently Object Orientated Design in a way that surpasses all my previous attempts to acquire knowledge on the subject.

I agree with you Sandro and would implore people to continue being curious and take the initiative to discover as much as you feel able to by engaging with those in the community that also exhibit those qualities.

Jeff said...

I didn't get the impression from the 1968 NATO paper that BDUF was advocated. Quite the opposite!

Design and implementation proceeded in a number of stages. Each stage was typified by a period of intellectual activity followed by a period of program reconstruction. Each stage produced a usable product and the period between the end of one stage and the start of the next provided the operational experience upon which the next design was based. In general the products of successive stages approached the final design requirement; each stage included more facilities than the last. On three occasions major design changes were made but for the most part the changes were localised and could be described as ‘tuning’

Dijkstra also made some telling comments about the need to get test part of the development process (not something that just happens at the end).

... I am convinced that the quality of the product can never be established afterwards. Whether the correctness of a piece of software can be guaranteed or not depends greatly on the structure of the thing made. This means that the ability to convince users, or yourself, that the product is good, is closely intertwined with the design process itself.

In fact, the more I read that paper the more I'm convinced that we haven't had too many terribly new ideas about the way we write software. We've just re-badged them and tried harder at selling them.

Ligeirinho said...

"Whenever you see people saying that we should or shouldn't do something, ask them why. Ask them about the context where they tried to do (or not to do) what they are saying."
We should apply this in every aspect of our lives!

Jean said...

BDUF works, it just doesn't scale.

Kit Plummer said...

BDUF works, it just takes too long and costs too much.

Anonymous said...

Functional programming languages work really well for massively parallel tasks (see e.g. MapReduce). The reason they're seeing a resurgence in popularity lately is because CPUs are no longer scaling up in speed, and are now scaling horizontally in cores, which is exactly the kind of optimization that functional programming excels at.

Unknown said...

-1 navel gazing, no code sample

Anonymous said...

God says...

consentings subtilty delighted companions entirely pierce pardoned genuinely girded officer sharper literally quoth contradictions strifes Avenue growing Our tenets requires conference Fame gasped assenting native reclaim endeared magical foresee lifted

Oliver Uvman said...

Fanatics are stuck in local maxima.

rstackhouse said...

Sounds similar in many ways to Fred Brooks' "No Silver Bullet" article and the "Context is King" mantra from our Agile benefactors.

suruz said...

AS you say selcting methodology is very much depends on context of software we develop.You can not just say procedural progamming is better
than OOP.It depends.As graphics engine programmer, I can say today's most graphics engines are data driven.Huge array of data(which can be a vertext buffer object) is passed to engine and with in a single CPU draw call data is passed to GPU. CPU to GPU data transfer is a expensive can not just issue GPU transfer call procedurally for every single data, which was OpenGL fixed function pipeline.Today's OpenGL is Object driven
not because it can not be handelded procedurally
but because of effeciancy of system hardware.

flash said...

Would of been more interesting with out the explanation at the end.

Wonder what kind of comments you would gotten then ? (Actually I don't wonder, I know)

Context Context Context

Rick said...

This stuff is precisely why I don't jump on the bandwagon of the latest save-the-world paradigm. It, too, will pass. If it lasts five years, I'll look at it,maybe even adopt it! Otherwise, I'll do what I've been doing since 1977: if/then/else. (As long as I don't have to go back to keypunching, everything else is negotiable!)

Niklas Schlimm said...

Context driven development then? ;) Very much like this article ... thx!

Kirk said...

For all the noise on the internet about how software should be developed I have yet to interview at a single company that has much in the way of any automated testing.

Really makes me wonder where the rest of the world really is.

Unknown said...

Great post. Navigating through all this vocabulary makes you realize how complicated software development is and how being reactive/pragmatic can help you. Question everything!

Jdon said...

you donot know anything about DDD, DDD is the best approach to design software. do you think software donot need design? directly develop it ? are you from other planets?

Mohamed said...

thank u

Seguridad Informática said...

After 45 years in development, the two outstanding guideline I learned to follow was:
1) Know the customers business and what he wants you to deliver. You deliver what he wants, and if you think there is a better way, you propose it verbally, and react based on feedback.

As for coding style, Functional or object oriented, topdown, or bottom up (as doing device driver development), use the best appropriate method.

In the end, it is based on the following remaining guideline:


and take the time to make certain that anyone could follow your solution.

Shawn Michaels said...

I, if truth be told, would like to thank so much for your advanced thought just about this good subject. This blog is exemplar of one fine piece of writing. This is my first occasion I visit here. I will post a link to this page on my blog. I am sure my visitors will find it very useful. BY - software development company

Moulay Mohamed Skalli said...

Whatever people chooses there must be one/many rational reason(s). Questioning the process is only a first step necessary but not enough. Not enough because people have their comfort zone that they don't like to step out of it. Some times people have their own agenda that can be driven by success, failure, fear, fame, revenge, limited vision, or cowboy. This dilemma can be viewed in so many ways depending which eyes are looking at it....I have more to say about this but these sentences should make us all reflect.

Nikos Maravitsas said...

Hi Sandro,

Great bog! Is there an email address I could contact you in private?

Djordje said...

Sandro, great post. I love it.
I dont want to get too philosophical bit ir just want to say that:
Software is science, and our appro
ach will always be the scientific method, which is based in hypothesis(wiki positivism). Your post perfectly describes it in a pragmatical way and at the end reminds us that, there is no best way(true because science is reduccionist and the results will always depend on their contexto) Thanks for your contributions, to the software community, i find your blog, very inspiring.

Orelinfotech said...

Context driven development, Very much like this article ... thanks for it.

Software Development Company Odisha

Anonymous said...

As you say selecting methodology is very much depends on context of software we develop. You cannot just say procedural programming is better. Well apart of that thanks for this sharing.

web development professionals in scottsdale

Unknown said...

I like your blog and its content. Nice post..

Post a Comment