Complexity Accidental vs Essential

Today it is hard to find team or organization that is not following agile but building software has not become easy, projects are missing schedule , over budget and it is also flawed.


Why it is so hard to build software ?

If you ask this questions to any engineer then 90%+ will say requirement, but is that the full truth ?

Lets try to decompose software construction. Every feature has 2 important component that decides whether feature will be successful or not.

• Essential complication (ec)

• Accidental complication( ac)

We will do some Functional programming refresher.

Feature = f( Essential Complication) + f(Accidental Complication )

Essential complication comes from domain like if you are building software for medical industry then it is complex. Accidental complication is complexity added by engineers, process & management to build feature.

Essential complication are hard to reduce because of domain, but to some extent it is possible to reduce by using good decomposition techniques. Decomposition is hard skill and comes from experiment of some fail projects.

Accidental complication can be controlled but it is not linear function, accidental complication is not same in every part of system and gets more complex over time. This also gives feedback on how much bad job we have done as engineer or product team.
Complexity comes in various forms like communication in team, less understanding , difficulty reusing some feature , extending program to new function, management problems etc.

Now we know accidental complication is exponential, so lets write formula again.

Feature = f( Essential complication) + f(Accidental complication * Unknown)) 

Now it will become clear why something takes many times longer than estimate or guess. Product owner also has part to play in adding accidental complication by marking assumption on importance of feature.

What can be done ?
If we need some predictability or consistence in delivery then we have to continuously work on reducing accidental complexity. Lets look at ways to keep to keep this in control.

• Using higher level languages.
• Incremental development by growing the software not building it.
• Good buy vs build decision. 
• Unified programming environment. 
• Raid prototype to refine requirement.
• Listen to design pressure.
• Test driven development.
• Stop "Get it out of the door" mindset.
• Reduce "surgical strike effort" in delivery.

Very insightful quote from Frederic Brooks, both customer and engineer has to learn what to ask, expect, and commit otherwise only option is broken system.

“An omelette, promised in two minutes, may appear to be progressing nicely. But when it has not set in two minutes, the customer has two choices—wait or eat it raw. Software customers have had the same choices. The cook has another choice; he can turn up the heat. The result is often an omelette nothing can save—burned in one part, raw in another.”
― Frederick P. Brooks Jr., The Mythical Man-Month: Essays on Software Engineering

Mythical-Man-Month by Mr Brooks is must read for every product owner , project manager and engineer.

If you like the post then you can follow me on twitter.

All models are wrong

Above quote by George Box is commonly used in Statistic and Machine learning but this post has nothing to do with Machine Learning!

I am going to share some thing about leadership models not machine learning model.

Some of the extremely successful leaders like Steve Jobs , Steve Ballmer, Jeff bezos, Larry Ellison , Elon Musk , Jack Ma , mark zuckerberg has played key role in transformation of our industry and life but do you want to use their leadership style or want to work with them ?

All of these leader are very successful and they came up with ideology,principles that worked in the system they were in and as engineer we know each system is optimized for very specific goals.

So before you start taking some of the traits from any leader you have also think about the system in which they we using that style and if you are not in that type of system then this leadership is going to do more damage than help.

With that we have to rephrase George Box statement to include leadership 

All leadership models are wrong but some are useful.

Today leadership has to be more open,collaborative and inclusive. 

Leadership comes after followership but many time we think that it is boolean, you are either leader or follower, being good follower is first stage of leadership and you have work on continuous improvement of followership skills.

Leadership and followership are two side of same coin but many leadership discussion, books etc puts very less focus on followership, so as individual you never deliberately practice follower skill. 

Followership

Followership has been studied for years and has many models, wikipedia link has good summary of the common ones and remember each model is optimized for system in which it works.

One of the model proposed by Ira Chaleff is very interesting, it divide followers in 4 types and later 5th one was also added.

As per him model looks like this

It has 2 dimension support & challenge.

Resource is low support and low challenge. They do what is asked for, they do enough to retain position.

Implementer is high support and low challenge. This is 10x engineer in team, always up for anything required to meet deadline. Leaders love this type of follower but side effect is that they will not speak up when things does not make sense or are not strategic. 

Individualists is low support & high challenge. This type has "Me" mindset and shows hostility towards someone or something. This type of follower are difficult to manage and needs constant realignment. 

Partners is high support & high challenge. This is sweet spot and very hard to find. This type of follower challenge leaders to make sure they understand big picture of team or organization. Once they are aligned then they take full responsibility and work whole heart to make it happen. This type of individual are good candidate for moving into leadership role.

Conclusion

Both leadership and followership are important to be successful and having self awareness of what type of leader & follower you are is very important. This gives nice model to understand people and their behavior. 

Don’t comment bad code—rewrite it

In this post i will share my experience of "comments in code" that i have got by reading code , writing code and reading books.

Lets start with famous quote

“Don’t comment bad code—rewrite it.”
  —Brian W. Kernighan and P. J. Plaugher

Lots of comment in code looks just like above image and it is distraction.

Comments are lie 
Most of the time comments are not in sync with code, no one cares about it so no one maintains it.

Code is refactored
Code is under constant improvement, it changes , evolves and dies but code comments just sticks and become orphan.

Bad code has more comments
Once we write code that is hard to understand then we try to explain it by comments and it tells lot about our failure to write expressive code.

Noisy comment
Lets look at at some code+comment from public domain.

https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java

/**
	* Create a new HdfsAdmin client.
	*
	* @param uri the unique URI of the HDFS file system to administer
	* @param conf configuration
	* @throws IOException in the event the file system could not be created
	*/
	public HdfsAdmin(URI uri, Configuration conf) throws IOException {
	FileSystem fs = FileSystem.get(uri, conf);
	if (!(fs instanceof DistributedFileSystem)) {
	throw new IllegalArgumentException("'" + uri + "' is not an HDFS URI.");
	} else {
	dfs = (DistributedFileSystem)fs;
	}
	}

This code snippet is from popular open source java project, many comment in enterprise is also like this. This adds more noise than value.

Journal comments
Many scripting code, data base code, python code is based on this pattern, this looks like team has no trust on source control and they have taken over this responsibility. It looks something like below

/*
01-Apr-2019 - First version with skeleton code by someone
10-Apr-2019 - Config support added  by someone
....
...
*/ 

Marker comments
This starts with something like below

// Starting init

//Make DB call 

this is also very common in scripting, DB stored procedure or in real enterprise type programs. This patterns tell that it is shouting to extract small function but gets overlooked.

Commented code
I am sure you have seen this many time and got frustrated that why this dead code exists.
It looks something like this.

//Keeping this for backup
//fastCode()


superFastCode() 

Author code
IDE adds block of comments to every file that is added to project, it looks like below

/*

@Ashkrit
*/

Author names is just at the top of the file but he no more maintains it and does not work for team or organization but occasionally gets credit/blame for his creation.



Conclusion
Code Comment mostly adds no value, it comes between you and code, it is one extra distraction.It also promotes bad habit for young programmer because they think "adding comments makes code better".

In some case comments are useful when you are writing public APIs but still be careful on what you write, software engineers are smart they can understand if code does not have accidental complexity.

I will leave you with another famous tweet by Kevlin Henney

Uncle bob has whole chapter in Clean Code: A Handbook of Agile Software Craftsmanship about comments. I recommend to read chapter, it covers things in more details.

If you like the post then you can follow me on twitter.