Information Snacking

buddha-231607_640Still at my home town some people equate being fat to prosperity. A fat person is someone who had steady access to food even at times of drought and famine, hence dying of hunger was ruled out. People ate whatever they can so that starvation does not kill them. Food revolution increased the availability of food and people started to eat well; when the life expectancy of every one improved, obesity became an issue. People were now dying because of overeating, unhealthy snacking and not maintaining a healthy life style.

The same is applicable to information, the differentiation between wise and knowledgable is not understood well. Just few decades ago in India, information was flowing slowly through government run TV channels, newspapers and books. The commute times were shorter with predictable work schedules, leaving more time for interaction with people and also enough time to read. I vividly remember the elders at home reading a novel in the evening, weekends or while standing at a long queue. There were reading desks at home where people read something and make notes.

If you summarize a book, often you will end up with a few lines which was the message from the book, but it will have a deep meaning to us if we had come up with those lines after reading and summarizing our thoughts, than those lines arriving in tweet.

french-fries-461705_1280We are right now in the era of information snacking where we are increasing our knowledge to a great deal, learned to drink it out of a firehose but not able to apply it effectively when needed. More and more people are addicted to ‘pull to refresh’ and snack on one liners, emails are annotated with TL;DRs and prefer to read a lot of tweets than read a book.

Manage the distractions before you are ruled by them.

TV was cloud-709148_1280one big distraction which was overtaken by social media, staying away from this for quality information consumption and time for family is a difficult but not an impossible thing to do. It is like going to gym toning up the body complemented by a healthy diet. Start slowly by turning off all notifications, keeping only one or two channels like telephone as an instant communication system; in other words change to a pull mode than being pushed with information. Dedicate slots for reading books, newspapers or other things which has long text, periodically clear out the social media backlog; indulge yourself once in a while but come back to a healthy routine.

book-419589_1280Why read long texts? Reading has helped me gather thoughts, play thought experiments, ability to write & articulate well; beyond that it took my mind of the constant buzz that follows a work day. Maintain a list of books to read, motivate other friends to read and share your learnings and stories. There are so many books that are rich in content and only limited time to read, sharing the book experience with others is rewarding as we get lots of information from others in a condensed but more interactive form as well as encourage us to read more and share. Stay away from information snacking, it just feeds empty knowledge. Consume information with a planned diet.

 

Release planning unboxed

Requirements

Release planning is largely an empirical activity. Capturing the requirements effectively is the first step. Capture the thoughts of product owner or client in a free flow, map them to user journeys and fine grain them to small requirement statements. Each of these granular requirements which can be expressed in a sentence or two will be the basic building blocks of the final product. It is popularly called as stories and getting them right is the first step of a successful release planning.

Visual triaging

sticky-note-681016_1280

Write each of the stories on to index cards or stickies, preferably colour coded based on parent features and put them up on a wall or a large table so that you can see all of them at once. This helps to see the spread of the requirements and allows to dive into deeper details for ambiguous entries.

If the ambiguity does not resolve with deep dives we quickly get the developers to do a spike (proof of concept) to validate our understanding. The proof of concepts have to be very quick but deep enough to provide us an understanding to estimate the effort involved.

Sizing

When talking about sizing, people immediately try to force fit their stories into fibonacci series or arbitrary small, medium, large. Instead we need to create a real world analogy for sizing the stories as mentioned in one of my previous posts. The last few years I have been using Scooter, Car and Bus as my sizing references for stories; and as I expected most of my sizing meetings have shrunk to less than half of what it used to be when using fibonacci or t-shirt sizes. If you have outliers which do not fit in any of the categories we temporarily park that in a trailer or ship category if very large or in bicycle if it is too small. The outliers are revisited and the stories are rewritten to see how they can be fitted into the sizing slots. The very large stories always have been unpredictable and skew the time taken for a point metrics, so we have to break them before the end of the sizing phase.

Velocity estimation

Remove the sizing details from the stories, pick developers in a round robin fashion and let them pick what can be done in five ideal days. That is there are no meetings on that day, no holidays, high availability of all machines and tools, no dependencies.  The developers each should do around 5 iterations of picking stories and the result has to be captured like below.

Developer 1

Week 1 S,S,M
Week 2 S,S,S,S
Week 3 M,M
Week 4 L
Week 5 S,L

Developer 2

Week 1 S,S,S,M
Week 2 M,S,S
Week 3 M,S,M
Week 4 L,S
Week 5 M,M

When we collect all the developers’ estimate and average out, we will be able to come up with the ratio of S,M,L like 1:2:4 , 1:3:7 or anything but not necessarily fibonacci. If we had taken fibonacci there would be too many arguments whether a story is a 2 or 3, but we avoid that argument if we let the velocity estimate give us the ratio. The ratio then becomes the points for a size, if the ratio turns out to be 1:3:7 then our S is 1, M is 3 and L is 7.

Realism infusion

It is not possible to plan for work in ideal days; there will be meetings, holidays, sick time, vacations, down times etc. We need to account for those. If the velocity estimate which we can call it as raw velocity comes out to be 10 points per week then we deduct the appropriate numbers to get a highly probable velocity. Typically this works out to be around 70% of raw velocity but it purely depends on the teams.

Prioritization

Clients will always say that they want all the features they have envisioned, but there are many stories where an app ends up where most of its user use only the 20% of it, the remaining is a long tail with very little returns to the effort involved. The product owner has to come up with the must haves for an initial set of features. There are many popular techniques used, MoSCoW prioritization method is one that is frequently used. If there are many people who will prioritize then each of them should come up with their own priorities and the chief product owner takes a call with a common denominator. The explanation is simple but most of the clients struggle to come up with a prioritized list as the illusion is every thing is a priority.

Sequencing

The stories are sequenced against a weekly timeline put up on the wall or a large table. The stories are then picked up in the order of feasibility with priority given to difficult and important stories. This step will give us lots of options on the number of parallel streams of work that can be run, what are the dependencies between stories, the order in which the features can be completed, the team ramp up and ramp down plans. The kind of mix and match we can do depends on the granularity of the stories and the amount of dependencies between them.

After the last step has gone through few rehashing we will be able to come up with an initial release plan to start with, this plan should always remain as the reference and re-planning has to happen on a monthly or fortnightly basis to keep adapting to the learnings we get as move along. Many people treat the initial release plan as a sacred rigid plan but that was just a guideline to tell us how to approach development. It is a waterfall project if the initial release plan was delivered as is.

The lazy programmer

I was having a chat with an old time ThoughtWorks developer, the topic was trending towards voluminous work and long days at work. I interjected with the point that the developers should not be willing to work long and hard hours, instead they should be lazy so that there are better tools and automations coming out of them, rather than checking boxes on long todo lists for the day.

He thought for a while and then replied “You are right, the CruiseControl continuous integration tool was born out of laziness”. He went on to explain that one of the developers felt that it was too annoying to walk up to a computer, pull code out and run the build & tests. So the person put a build loop on that machine to do that task. Someone else put a web interface to it and there a new tool was born. It left a lasting legacy in the continuous integration space. (Cruise control home page).

Why do people relate long working hours to prosperity?

The industrial revolution required a good deal of unskilled labourers who were given instructions and repetitive tasks to be done. The more they do, the more money the company makes. So overtime was rewarded with more money and people tend to stay longer to get paid overtime. The invention part of automating the repetitive tasks were left to someone else, the thought of hard and long work is rewarding stayed on even though there was a chance of a new invention that could take this entire category of job away.

After many decades of advancements in the industrial space, automations have taken a majority of space. The place where the automation as of now is not able to get into are creative spaces or knowledge work. If a job requires more than few simple steps then it is beyond the mechanical skills and involves cognitive skills. The moment when even rudimentary cognitive skills are involved, then no longer any of those incentives and hour based pay work. It is explained well in the video created from the work of Dan Pink, the author of the book Drive.

Wisdom gets passed on through generations, bosses and workers alike, people were conditioned from the childhood that hard and long work is the only way prosper. When that person becomes the boss, demands the hours and when that person is the worker, obliges to it.

Programming is a step further, it involves complex thinking which requires us to bring deeper parts of our brain to work. Andy Hunt in his book The Pragmatic Programmer talks about L-mode and R-mode, which is about using our linear mode of the brain or the rich/random mode of the brain. Though there is value for linear mode, programming benefits a great deal from the R-mode. Staring outside the window, doodling, watching the fountain, a stroll could all be more productive activities than staring at the screen and furiously typing commands for long hours because new ideas pop out when you are least expecting and unprepared.

A programmer has to be lazy, should not jump into the task at hand instead approach programming with a mindset of ‘No code is the best code’. Laziness will make us look for to remove mundane repetitive tasks out of the way which will also pop up more creative ideas through R-mode. Workplaces should also help ease the norms of equating the number of hours in seat to productivity.

Push the problem out of your foreground mind, and just “hold it lightly”. Then go for a walk, etc. That’s when insights and breakthroughs come to me.— Henri Poincaré

Hand me downs

Why not follow the boy scout rule when moving on?

pass on

Boy scout rule is well known in extreme programming, people are advised to leave the code in a better state than they found it. I observe this mostly works well for programming but not elsewhere. I have always admired teachers especially the ones that teach the basics. I made it a point that if I understand something after a good deal of effort then I would make it easy for another person by simplifying it. I kept doing this at school and college, I helped people learn tough parts of algebra, chemistry and physics through easy analogies.

When I graduated and got my job, I still carried on with this work of simplifying the tough things I learnt. There was a sudden change in team composition and I had to take up the work of a sysadmin, which was difficult for someone who was in technical support and testing for a year. I spent a lot of time learning to fit in the new role and in the process making sure that the next person getting on to this will require less transition, this is the time I was in for a rude shock; I was told by one of the senior members of the team ‘if you proceed this way of simplifying things and sharing your knowledge, you will soon be out of your job’.

The intentional complexity was hard to accept, especially when the company was trying hard to reduce dependencies on people. The high complexity created many silos in the team which made replacements harder eventually causing the team’s growth to slow down. It gave a false sense of security as people were called experts in their tasks, but not learning anything new as the learning curve was too steep.

Not working together with peers or communities will lead to phase called ‘Expert beginner’ which prevents someone from becoming competent. There is a good writeup about ‘Expert Beginner’. People take pride in the complexity of the their work and put through the new comers through the same phase so that the learning curve is steep and there is still value for expert beginners.

I read an article from ‘British Bird Lovers’ which is about how red robins which are territorial in nature lost out on the learnings it had got to open sealed bottles. The birds which learnt and kept the knowledge to itself gained a lot, but its successors did not learn any. The article finishes well saying ‘Birds that flock together appear to learn faster and increase their chances to evolve and survive’.

The general tendency for people is to pass on what was handed down to them as it is, especially if they spent a lot of effort to make it work for them. If someone has a tough time getting on board at workplace, s/he will tend to keep the on-boarding process the same, largely due to the fear of someone else overtaking or replacing them. In the process creating a culture of territory and stagnation instead of co-operation and shared growth.
Eric Schmidt mentions in his presentation ‘How google works’, that the only way to consistently succeed is to attract skilled people, work as a group, care for the workplace. We should only hand down the best, leave the place better than we find, if we do it that way then together we move ahead.

Cost of testing last

sashimi-472778_640At a fine dining restaurant if I complained that I did not like the food; immediately the chef, waiter and the food taster run to the dining table to assure that I don’t feel disappointed and provide me an alternative as soon as possible. If you take a look behind the kitchen door then we will come to know that they take lots of precautions to make sure that this never happens at the dining table. Chefs have a tough time to keep taste and quality up to the mark irrespective of the availability of ingredients and limited time.

A typical restaurant is staffed with a chef, chef’s cooking assistants, butcher, waiters, managcook-196932_640er, janitors, food tasters and a lot of machines which speed up the process of cooking. Cook’s best use of time is spent cooking the food, but does it mean the cook should not get a feel whether the food is cooked and is it as per the customization the waiter described?

Let us a take a scenario where the manager is able to hire more food tasters and shifts the burden of the quality to the food taster. Cooks will be able to create lots of dishes as per the recipe given by the chef without giving a second thought about the balance of ingredients, texture and the level of cooking required. The dish will ready to present in a finished state when it reaches the food taster, if any fault is found then the dish had to done from the beginning which will increase the workload of the cook further and reduce the output considerably as the reworks are expensive, eventually will make the customer wait more or increase dining table incidents.

What looked like a clever idea on paper backfires as cooking for fine dining is not something that could be made into a template and responsibilities could be in silos; it is a complex system with a strong feedback loop at every stage. The scene at programming is similar at many cases, developers are encouraged to write only the code and leave the testing to the QA as the perception is that the best use of a developer’s time is to make her code. Seasoned programmers know better that unit tests are also code, if you are not testing then you do not know what you program.

Cynefin_as_of_1st_June_2014Programming is an emergent practice, which matches the complex domain in the Cynefin framework by Dave Snowden. Each project will be unique and a pattern will emerge over time which will be efficient for the team if there is a strong feedback loop. Trying to separate out of responsibilities like testing to just one small group will push the team into a cycle of code, test and fix with long feedback loops, eventually causing delays. Quality has to be assured at every stage of application development, responsiveness determines who is competitive in a complex environment.

Image courtesy: http://pixabay.com & http://www.wikiwand.com/en/Dave_Snowden

Rocking horse

Building software is always a task of trying to hit a moving target. The complexity keeps growing on every parameter if not managed well. Projects which build softwares that have to talk to each other are often built by different teams under different managements, leaving less scope to find schedule and contract to match for smooth integration. Many times I have observed that there is so much of dependency on each other that the projects look like they are progressing so well until they hit a dead lock on integration.

rocking-horseIn my view such projects are like rocking horses, for a child it gives an illusion that s/he is in control of the horse and its movements but there is no real progress, it is just moving not progressing. The illusion of movement in a project is formed when developers implement the requirements which is neither validated nor integrated but just added to growing source code, followed by an all out big bang effort to integrate and push it to production. The result of rework and last mile dash leaves the teams burnt out and demotivated that there is hardly enough energy to move on to the next work.

Research say that we have two kinds of thinking which I encountered in the book ‘The Pragmatic Programmer’, it says that people who work with the brain (knowledge workers) need to make use of both focussed and diffused modes of thinking which is mentioned as L-mode and R-mode in the book. Procrastination is the key to R-mode as it props up the answers we are looking for, at a moment when we are not thinking about it. It has worked for me very well as I have observed that I get great ideas in shower, while driving, while having coffee staring outside the office window.

Sadly management is trained to look at people furiously typing at their computers for long hours as work and any thinking time saharais deemed as a luxury. This problem gets compounded when management in the guise of agile leaving out all the people aspects behind; which help in looking back, taking stock of situation and plan well but use it solely for micromanagement. People if lost will usually walk in circles, it is a proven fact that people lost in a desert in the night time or an overcast condition without the direction markers will walk in circles, more about it in this link from Nat Geo.

Directionless movement is just a movement for the sake of movement like a rocking horse, there is no real progress.

Pull request is the new hammer

Building software is not an engineering problem, it is a communication problem between engineers and business. Agile Manifesto‘s first and foremost point is about the individuals and interactions. It is imperative that the feedback loop should be as short as possible to reduce the cost of late rework. Each team in an organization would be unique in terms of composition, their communication and means of interaction will be fine tuned for that particular group. The teams should be guided through non negotiable philosophies rather than intricate set of policies, restrictions and tools.

Cliche – ‘Any code can be understood by machines, code should be written such a way that it should be readable by humans’

MatrixCode is read many times more than it is written, so it is everyone’s desire to keep the code base very neat to adhere to good coding standards. One way of ensuring this is to do a code review. With Git the collaboration got easier that you ask anyone to clone your repo, do some work and contribute back through a pull request. Pull request is the only way of contributing code to a project which is otherwise a closed read only repository. The owner reviews the pull requests and decides what goes into his/her repo. The feedback cycle in this manner is too slow but works well for social coding.

Cliche – ‘If you have a hammer then all your problems look like nails’

It is becoming increasingly difficult to gather the right requirements, translate them quickly into working software and that too in an efficient hammer-147840_640manner. It is necessary for us to be able to roll up information from the environment and translate that into the software. To help the evolution we should be able to keep the feedback loop as small as possible so that it is very easy to add the right functionalities at a good pace. I happened to read a nice writeup on the importance of rolling up information from the environment. Many people fail to visualize what their intent is in selecting a tool in the first place, pull requests work very well in controlling what goes into someone’s personal project. It will also be successful in team setups, but it comes with a cost. The cost is the speed of evolution, as it will take longer and longer to sync the commits when the codebase grows.

Cliche – ‘Give a carpenter a latest power drill instead of a hammer, he will use it as a hammer’

penguin-158551_1280In the SOA context it is a double whammy where there are many micro repos and pull requests are used to control the quality of the code. Choice of tools and technologies are always like rock, paper & scissors; what works in one context will not work in the other. Choose the right one considering the intent and moving targets in all dimensions in the system, change the approach if it does not fit in or the situation has changed.

It is just a road not a roadmap

Roadmap A typical roadmap is something which gives a chosen possible path with alternate routes, options and milestones. But these alternatives and options stop at the map level and do not move into product roadmaps in the industries. It is very common to see product roadmaps published with milestones and when to expect them but all of them are linear. It is not a roadmap, it is just the road ahead which is more or less static. Responding to change is possible only when you have a roadmap, else every problem that comes in the way puts us in a hardship as we were not prepared to respond with alternatives.

How do we come up with a roadmap for software projects? Just like maps the requirements have to be visually triaged. We need to collect them at granular level such that it can be captured recorded in one or two sentences that can be written in bold on an index card. These granular requirements are popularly called as stories which represent an action of an end user or the ability of the system.

Once we collect most of the possible actions and abilities then we lay them all together on a large table or put it up on a wall to see them all at one glance. This helps us to triage quickly and discard duplicate ones and add some more if they were found to be missing. Once we have come up with an initial list then we collect the cards as per the priorities and arrange them linearly. If a requirement can be done in parallel then a parallel stream is introduced. In this manner we will be able to have a visual data on what are the dependencies and how much can be done in parallel.

We should play around the sequence and parallel streams to come up with multiple options for milestones and these multiple options together constitute a roadmap. Every few months it is better to revisit to the roadmap as software projects are always a moving target. This is nothing but the release planning meeting mentioned at XP website. We have to be aware of the options and not just rely on the first release plan that we have created, if we rely only on the first draft then it is just the road ahead; we will achieve all the milestones as per the plan but it may mostly lead to a not so desired location.

Packaging functional tests using TestPackage

TestPackage provides a clean way to package your functional tests and run them as a jar file with sharding and package level run options. All that we need to do is created a maven/gradle subproject for functional tests and have the testpackage’s dependency and manifest file entry to get an executable jar package.

Once the jar is created all we need to do is run ‘java -jar the-test-package.jar package.that.contain.tests’. This helps a lot in continuous delivery where the functional test artifact can be created at the build stage and promoted to run at the functional test stage without having to access the source code and rebuild later.