The lazy programmer

I was having a chat with an old time ThoughtWorks developer, the topic was trending towards voluminous work and long days at work. I interjected with the point that the developers should not be willing to work long and hard hours, instead they should be lazy so that there are better tools and automations coming out of them, rather than checking boxes on long todo lists for the day.

He thought for a while and then replied “You are right, the CruiseControl continuous integration tool was born out of laziness”. He went on to explain that one of the developers felt that it was too annoying to walk up to a computer, pull code out and run the build & tests. So the person put a build loop on that machine to do that task. Someone else put a web interface to it and there a new tool was born. It left a lasting legacy in the continuous integration space. (Cruise control home page).

Why do people relate long working hours to prosperity?

The industrial revolution required a good deal of unskilled labourers who were given instructions and repetitive tasks to be done. The more they do, the more money the company makes. So overtime was rewarded with more money and people tend to stay longer to get paid overtime. The invention part of automating the repetitive tasks were left to someone else, the thought of hard and long work is rewarding stayed on even though there was a chance of a new invention that could take this entire category of job away.

After many decades of advancements in the industrial space, automations have taken a majority of space. The place where the automation as of now is not able to get into are creative spaces or knowledge work. If a job requires more than few simple steps then it is beyond the mechanical skills and involves cognitive skills. The moment when even rudimentary cognitive skills are involved, then no longer any of those incentives and hour based pay work. It is explained well in the video created from the work of Dan Pink, the author of the book Drive.

Wisdom gets passed on through generations, bosses and workers alike, people were conditioned from the childhood that hard and long work is the only way prosper. When that person becomes the boss, demands the hours and when that person is the worker, obliges to it.

Programming is a step further, it involves complex thinking which requires us to bring deeper parts of our brain to work. Andy Hunt in his book The Pragmatic Programmer talks about L-mode and R-mode, which is about using our linear mode of the brain or the rich/random mode of the brain. Though there is value for linear mode, programming benefits a great deal from the R-mode. Staring outside the window, doodling, watching the fountain, a stroll could all be more productive activities than staring at the screen and furiously typing commands for long hours because new ideas pop out when you are least expecting and unprepared.

A programmer has to be lazy, should not jump into the task at hand instead approach programming with a mindset of ‘No code is the best code’. Laziness will make us look for to remove mundane repetitive tasks out of the way which will also pop up more creative ideas through R-mode. Workplaces should also help ease the norms of equating the number of hours in seat to productivity.

Push the problem out of your foreground mind, and just “hold it lightly”. Then go for a walk, etc. That’s when insights and breakthroughs come to me.— Henri Poincaré

Hand me downs

Why not follow the boy scout rule when moving on?

pass on

Boy scout rule is well known in extreme programming, people are advised to leave the code in a better state than they found it. I observe this mostly works well for programming but not elsewhere. I have always admired teachers especially the ones that teach the basics. I made it a point that if I understand something after a good deal of effort then I would make it easy for another person by simplifying it. I kept doing this at school and college, I helped people learn tough parts of algebra, chemistry and physics through easy analogies.

When I graduated and got my job, I still carried on with this work of simplifying the tough things I learnt. There was a sudden change in team composition and I had to take up the work of a sysadmin, which was difficult for someone who was in technical support and testing for a year. I spent a lot of time learning to fit in the new role and in the process making sure that the next person getting on to this will require less transition, this is the time I was in for a rude shock; I was told by one of the senior members of the team ‘if you proceed this way of simplifying things and sharing your knowledge, you will soon be out of your job’.

The intentional complexity was hard to accept, especially when the company was trying hard to reduce dependencies on people. The high complexity created many silos in the team which made replacements harder eventually causing the team’s growth to slow down. It gave a false sense of security as people were called experts in their tasks, but not learning anything new as the learning curve was too steep.

Not working together with peers or communities will lead to phase called ‘Expert beginner’ which prevents someone from becoming competent. There is a good writeup about ‘Expert Beginner’. People take pride in the complexity of the their work and put through the new comers through the same phase so that the learning curve is steep and there is still value for expert beginners.

I read an article from ‘British Bird Lovers’ which is about how red robins which are territorial in nature lost out on the learnings it had got to open sealed bottles. The birds which learnt and kept the knowledge to itself gained a lot, but its successors did not learn any. The article finishes well saying ‘Birds that flock together appear to learn faster and increase their chances to evolve and survive’.

The general tendency for people is to pass on what was handed down to them as it is, especially if they spent a lot of effort to make it work for them. If someone has a tough time getting on board at workplace, s/he will tend to keep the on-boarding process the same, largely due to the fear of someone else overtaking or replacing them. In the process creating a culture of territory and stagnation instead of co-operation and shared growth.
Eric Schmidt mentions in his presentation ‘How google works’, that the only way to consistently succeed is to attract skilled people, work as a group, care for the workplace. We should only hand down the best, leave the place better than we find, if we do it that way then together we move ahead.

Cost of testing last

sashimi-472778_640At a fine dining restaurant if I complained that I did not like the food; immediately the chef, waiter and the food taster run to the dining table to assure that I don’t feel disappointed and provide me an alternative as soon as possible. If you take a look behind the kitchen door then we will come to know that they take lots of precautions to make sure that this never happens at the dining table. Chefs have a tough time to keep taste and quality up to the mark irrespective of the availability of ingredients and limited time.

A typical restaurant is staffed with a chef, chef’s cooking assistants, butcher, waiters, managcook-196932_640er, janitors, food tasters and a lot of machines which speed up the process of cooking. Cook’s best use of time is spent cooking the food, but does it mean the cook should not get a feel whether the food is cooked and is it as per the customization the waiter described?

Let us a take a scenario where the manager is able to hire more food tasters and shifts the burden of the quality to the food taster. Cooks will be able to create lots of dishes as per the recipe given by the chef without giving a second thought about the balance of ingredients, texture and the level of cooking required. The dish will ready to present in a finished state when it reaches the food taster, if any fault is found then the dish had to done from the beginning which will increase the workload of the cook further and reduce the output considerably as the reworks are expensive, eventually will make the customer wait more or increase dining table incidents.

What looked like a clever idea on paper backfires as cooking for fine dining is not something that could be made into a template and responsibilities could be in silos; it is a complex system with a strong feedback loop at every stage. The scene at programming is similar at many cases, developers are encouraged to write only the code and leave the testing to the QA as the perception is that the best use of a developer’s time is to make her code. Seasoned programmers know better that unit tests are also code, if you are not testing then you do not know what you program.

Cynefin_as_of_1st_June_2014Programming is an emergent practice, which matches the complex domain in the Cynefin framework by Dave Snowden. Each project will be unique and a pattern will emerge over time which will be efficient for the team if there is a strong feedback loop. Trying to separate out of responsibilities like testing to just one small group will push the team into a cycle of code, test and fix with long feedback loops, eventually causing delays. Quality has to be assured at every stage of application development, responsiveness determines who is competitive in a complex environment.

Image courtesy: http://pixabay.com & http://www.wikiwand.com/en/Dave_Snowden

Rocking horse

Building software is always a task of trying to hit a moving target. The complexity keeps growing on every parameter if not managed well. Projects which build softwares that have to talk to each other are often built by different teams under different managements, leaving less scope to find schedule and contract to match for smooth integration. Many times I have observed that there is so much of dependency on each other that the projects look like they are progressing so well until they hit a dead lock on integration.

rocking-horseIn my view such projects are like rocking horses, for a child it gives an illusion that s/he is in control of the horse and its movements but there is no real progress, it is just moving not progressing. The illusion of movement in a project is formed when developers implement the requirements which is neither validated nor integrated but just added to growing source code, followed by an all out big bang effort to integrate and push it to production. The result of rework and last mile dash leaves the teams burnt out and demotivated that there is hardly enough energy to move on to the next work.

Research say that we have two kinds of thinking which I encountered in the book ‘The Pragmatic Programmer’, it says that people who work with the brain (knowledge workers) need to make use of both focussed and diffused modes of thinking which is mentioned as L-mode and R-mode in the book. Procrastination is the key to R-mode as it props up the answers we are looking for, at a moment when we are not thinking about it. It has worked for me very well as I have observed that I get great ideas in shower, while driving, while having coffee staring outside the office window.

Sadly management is trained to look at people furiously typing at their computers for long hours as work and any thinking time saharais deemed as a luxury. This problem gets compounded when management in the guise of agile leaving out all the people aspects behind; which help in looking back, taking stock of situation and plan well but use it solely for micromanagement. People if lost will usually walk in circles, it is a proven fact that people lost in a desert in the night time or an overcast condition without the direction markers will walk in circles, more about it in this link from Nat Geo.

Directionless movement is just a movement for the sake of movement like a rocking horse, there is no real progress.

Pull request is the new hammer

Building software is not an engineering problem, it is a communication problem between engineers and business. Agile Manifesto‘s first and foremost point is about the individuals and interactions. It is imperative that the feedback loop should be as short as possible to reduce the cost of late rework. Each team in an organization would be unique in terms of composition, their communication and means of interaction will be fine tuned for that particular group. The teams should be guided through non negotiable philosophies rather than intricate set of policies, restrictions and tools.

Cliche – ‘Any code can be understood by machines, code should be written such a way that it should be readable by humans’

MatrixCode is read many times more than it is written, so it is everyone’s desire to keep the code base very neat to adhere to good coding standards. One way of ensuring this is to do a code review. With Git the collaboration got easier that you ask anyone to clone your repo, do some work and contribute back through a pull request. Pull request is the only way of contributing code to a project which is otherwise a closed read only repository. The owner reviews the pull requests and decides what goes into his/her repo. The feedback cycle in this manner is too slow but works well for social coding.

Cliche – ‘If you have a hammer then all your problems look like nails’

It is becoming increasingly difficult to gather the right requirements, translate them quickly into working software and that too in an efficient hammer-147840_640manner. It is necessary for us to be able to roll up information from the environment and translate that into the software. To help the evolution we should be able to keep the feedback loop as small as possible so that it is very easy to add the right functionalities at a good pace. I happened to read a nice writeup on the importance of rolling up information from the environment. Many people fail to visualize what their intent is in selecting a tool in the first place, pull requests work very well in controlling what goes into someone’s personal project. It will also be successful in team setups, but it comes with a cost. The cost is the speed of evolution, as it will take longer and longer to sync the commits when the codebase grows.

Cliche – ‘Give a carpenter a latest power drill instead of a hammer, he will use it as a hammer’

penguin-158551_1280In the SOA context it is a double whammy where there are many micro repos and pull requests are used to control the quality of the code. Choice of tools and technologies are always like rock, paper & scissors; what works in one context will not work in the other. Choose the right one considering the intent and moving targets in all dimensions in the system, change the approach if it does not fit in or the situation has changed.

It is just a road not a roadmap

Roadmap A typical roadmap is something which gives a chosen possible path with alternate routes, options and milestones. But these alternatives and options stop at the map level and do not move into product roadmaps in the industries. It is very common to see product roadmaps published with milestones and when to expect them but all of them are linear. It is not a roadmap, it is just the road ahead which is more or less static. Responding to change is possible only when you have a roadmap, else every problem that comes in the way puts us in a hardship as we were not prepared to respond with alternatives.

How do we come up with a roadmap for software projects? Just like maps the requirements have to be visually triaged. We need to collect them at granular level such that it can be captured recorded in one or two sentences that can be written in bold on an index card. These granular requirements are popularly called as stories which represent an action of an end user or the ability of the system.

Once we collect most of the possible actions and abilities then we lay them all together on a large table or put it up on a wall to see them all at one glance. This helps us to triage quickly and discard duplicate ones and add some more if they were found to be missing. Once we have come up with an initial list then we collect the cards as per the priorities and arrange them linearly. If a requirement can be done in parallel then a parallel stream is introduced. In this manner we will be able to have a visual data on what are the dependencies and how much can be done in parallel.

We should play around the sequence and parallel streams to come up with multiple options for milestones and these multiple options together constitute a roadmap. Every few months it is better to revisit to the roadmap as software projects are always a moving target. This is nothing but the release planning meeting mentioned at XP website. We have to be aware of the options and not just rely on the first release plan that we have created, if we rely only on the first draft then it is just the road ahead; we will achieve all the milestones as per the plan but it may mostly lead to a not so desired location.

Packaging functional tests using TestPackage

TestPackage provides a clean way to package your functional tests and run them as a jar file with sharding and package level run options. All that we need to do is created a maven/gradle subproject for functional tests and have the testpackage’s dependency and manifest file entry to get an executable jar package.

Once the jar is created all we need to do is run ‘java -jar the-test-package.jar package.that.contain.tests’. This helps a lot in continuous delivery where the functional test artifact can be created at the build stage and promoted to run at the functional test stage without having to access the source code and rebuild later.

Using Log4J2 with SLF4J in a Spring 4 application

In one of the spring web applications we decided to use Log4J2 with SLF4J; when we deployed the web app in tomcat server along with the required jars we stumbled on the issue multiple binding for SLF4J and spring dependencies not getting satisfied. We had used Gradle and there were transitive dependencies which included log4j-12xx.jar and slf4j-log4j.jar as some of the dependencies in the project were using log4j version 1 for logging. Here is how we were able to start logging using log4j2.

Exclude the log4j-12xx.jar and slf4j-log modules from the gradle build

configurations {
 all*.exclude group: "org.slf4j", module: "slf4j-log4j12"
 all*.exclude group: "log4j", module: "log4j"
}

Add the following jars

org.slf4j:log4j-over-slf4j|route all the log4j v1 to slf4j
org.slf4j:slf4j-api|slf4j api
org.apache.logging.log4j:log4j-slf4j-impl|log4j2 bridge
org.apache.logging.log4j:log4j-core|log4j2 core
org.apache.logging.log4j:log4j-api|log4j2 api

Add the log4j2.xml configurations to the classpath or as specified at log4j manual

Build it like a sports team

Is it easy to get a crash course in football by Pele or Maradona for a week and produce a world cup winning football team?

Answer is NO. Then why do lots of people in the corporate world think of hiring scrum trainers & expert developers to train their team for a week and then expect their team to be undergo a transformation at a magical scale?

Football

German football team made it a point to transform their team and it took them a lots of years before they were able to reach the pinnacle. A quick side by side comparison of what is causing agile transformation to fail.

Vision

Football: Someone was there owning this entire transformation, the German football association spent a lot of time identifying talent in their teens and groomed them.

Office: In the corporate world switching jobs every few years have become common, but there is no passing on of the context, resulting in the new person taking charge, starting from all over again, as well as frustrating existing good performers who have to rebuild the perception.

Infrastructure

Football: Players expecting state of the art training facilities, fitness programs and new shoes are not a luxury, it is a necessity.

Office: Computers have become so cheap compared to the salaries, yet the policy of providing the best tools and good work environment are archaic.

Coach vs Management

Football: Coaches are given their due powers to help the team achieve the goal. It is very easy for anyone to comment on how professionals should play their game, there would be no use adhering to the metrics if the team cannot win. Winning is the only measure for management.

Office: Often coaches are seen as part of the management or management tries to heavily influence coaching which results in a team which will work either for metrics or to please higher ups without the actual result that it had aimed for.

Training

Football: Just the ability to kick the ball does not make a footballer. Training will be introduced to increase physical strength & stamina, better mental wellness, injury prevention, tactics and strategy.  A heavy investment is made in the training facilities.

Office: In the software industry a generation is about 2-3 years. Computer science degrees are nowhere near what is state of the art in the industry. At many places the on boarding process is either very shallow or not up to date with the recent developments, leaving people to learn most of the things hard way. Given too many things to learn and the information overload, this results in inefficient learning and application of knowledge on the job. We need to prepare people to find answers that are not available on Google.

Team composition

Football: Rookies don’t learn by watching greats from the bench, instead they play along with the veterans. Every sports team makes sure to have the right composition with a mix of rookies, emerging players and veterans. That is how they sustain a winning team.

Office: Architects and Leads often do not code or not part of the team every day, this means that most of the time the team just looks up to for advice or waits for reviews. There should be a good mix of people at all experience level so that there are enough people to try new things, enough people who have mastered few things and enough people who challenge change.

Above all – Persistence

Nothing in the world can take the place of persistence. Talent will not; nothing is more common than unsuccessful men with talent. Genius will not; unrewarded genius is almost a proverb. Education will not; the world is full of educated derelicts. Persistence and determination are omnipotent. The slogan ‘press on’ has solved and always will solve the problems of the human race. ~ Calvin Coolidge

Image courtesy of Salvatore Vuono at FreeDigitalPhotos.net