Regression towards the mean

I came across this sentence ‘Regression towards the mean’ in the book ‘The drunkard’s walk’. It means that if something bad, unusual or extraordinary happens then it mostly followed by an average acceptable one. It mentions that people are often fooled by this phenomenon, it gives a false sense of control over situations and outcomes. People think that rebuking someone for bad behaviour results in good behaviour, but according to statistics acceptable behaviour always follows bad behaviour irrespective of getting yelled at or not.

A flight instructor’s example is given, the flight instructor thinks that yelling at a pilot making a serious mistakes makes them fly the plane better next time and praising a pilot for a good flight results in a mediocre flight next time; so the instructors resort to only yelling and no praise, but studies show that irrespective of an instructor’s reaction the behaviour of the pilots always fluctuated, a good one was followed by an average one and a bad one was also followed by an average one.

angry-33059_1280

This shows that we should learn to respond to situations than react to it, if there is something not acceptable it is better to express the impact on us and leave it, than to do something to make sure the person repents for the mistake. It is a tough habit to leave as rebuking and a good behaviour following is very much a positive reinforcement, but if achieved it creates a good atmosphere to allow natural fluctuations take its course without worrying too much about it.

Sharing Knowledge vs Learning Together

Knowledge of group learners grows multifold than lone learners. This is understood by many college students where group study is common. I as a student, part of a group, divided subjects among ourselves, study the reading materials, understand it, condense it and present it back to the group. In this way every one understood the crux of what is to be learnt in a short amount of time. Somewhere I read ‘the moment we get a job, we stop learning’. That is because, learning is a conscious activity and when we don’t learn together, it becomes too tough to continue learning and many just don’t learn anymore.

lesson-1014915_1280

Majority of the books have some fluff, it is to comply with the publisher’s mandate for the thickness of the book to justify a price. It is difficult to sift through the pages and mine the wisdom we want from it. It takes time  and speed reading is not easy for a lot of people. In a group setting it is ideal to read a book, summarise the learnings and present it back to the group either as a summary or the application of the learnings. If it evokes enough curiosity, then people will make some time to read it in detail.

The above applies to videos, courses and any other learning form or material. What about the blogs, they are short, can we just forward the links? There are lots of ways in which we receive links; twitter, hacker news, reddit so on. If we merely forward the links that interests us to our groups, it has no bigger impact than people who have already subscribed to the feeds. What will be beneficial to people who receive our link would be, a short description of what it is, how did I apply (or can apply), why this would be beneficial, next steps if possible.

By making it easy for people to understand many topics in a condensed form, we improve the breadth of the knowledge of the whole group. If anyone is curious then they can deep dive for the depth in a particular subject. It is impossible to get out of date in a group that is continuously learning.

Birds that flock, seem to learn faster. They increase their chances to survive and evolve more quickly

Learning together requires effort, it won’t happen if we don’t do it consciously.

Micro Habits

Anyone who had taken a reliable public transport would have formed micro habits. The first time I am in a new place, taking a public transport to work is always an experiment. As few days pass, the onward connections and crowd patterns get clear and I do my optimisations. I learnt to cut down my travel time 15 minutes every day by always boarding the last coach in the train so that when I alight at my station the coach is always next to the staircase which connects me to the next platform; I quickly walk to the next platform and board the train waiting there. If I miss this train, the next connection was 15 minutes later. Even if the coach is crowded, I bear the discomfort for a while because I know I am going to catch my connection on time.

At work and home also lots of micro habits can be formed. As the name suggests, these are not big ones and often takes only a small effort to make it happen. The first requirement to form micro habits is to have a set routine. Wake up at the same time every day, leave for work at the same time every day, leave office at the same time every day. By having a set routine we tend to observe lots of things and we also reduce the variables, so the number of decisions that needs to be made on a day reduces, because the routine automates most of our decisions.

Micro habits should not be confused with micro optimisation. Micro optimisations are very small gains for a large effort. Micro habits end up in great gains in small timely efforts. All it takes is to have a discipline to follow a set routine, observe the surroundings and make those small adjustments. It may not be easy at first, but once we start practising it becomes part of our thoughts.

Small changes can produce big results…but the areas of highest leverage are often the least obvious.

– Peter Senge, The Fifth Discipline

 

As we grow

It is natural for every kid topeter-886132_1280 dream of growing up to a respectable status, have lots of possessions and make decisions. What I did not realise was the amount of responsibilities that came with growing up. The first big desire for any one to grow up is the desire to be independent, not be told what to do instead do what they want. It was always about the desire to be independent but oblivious to the price to pay for being independent. People will stop worrying about us less, we will be expected to stand for ourselves and care for dependent ones as grown ups. Being a grown up is a responsibility than a perk.

family-829133_640

At work similar thing happens, we join as an intern with a big dream. As an intern we will be told what to do and there is a limited scope of influence at work but at the same time there are less decisions that needs to be taken. As we gain experience the sphere of influence of our action increases, so do the expectations from the others to become effective individual contributors.

The term ‘individual contributor’ is largely misunderstood, it does not mean ‘leave me alone to do my work at my style’, instead it is a responsibility to reach the individual goals while working along with others. If there is a need for constant nudge from the managers, co-ordinators or facilitators to be reminded about one’s work and progress then it is a telltale sign that an individual has not grown up. When gaining experience what will matter most is the ability to accept responsibility and be accountable for signed up work, the independence that comes up with that is just a side effect not a right by itself.

Learning to drive

Kent Beck gives a good introduction to adoption of XP through an analogy of learning to drive. A brief extract can be found here. The learning to drive analogy is applicable to any discipline, where the instincts have to take over conscience in other words the motor/muscle memory will play a bigger part than a conscious effort, but when demanded we need to rewire our learnings. Music is also analogous to this.

The steps in becoming a good in something.

Start immediately

Procrastination affects a lot of us and one of the key catalyst for someone to procrastinate is the idea of perfection. This is because of the fear of failure and being judged, so people end up reading and studying about something instead of learning by doing. Unless we get into something mission critical or any action that will cause any irreversible consequence then we should not hesitate to start. By starting immediately, we will be forced to learn the absolute minimum that is required to start instead of getting equipped with a world of knowledge.

Be predictably slow at first

There is a sense of satisfaction in moving up the levels, so people immediately step up their levels without getting a good understanding of what they learnt. When I was learning to drive, I was never allowed to shift to the 3rd gear until I was able to drive the vehicle without getting stalled in the lower gears. Same goes to my piano tutor who never allowed me to step up the tempo until I never missed a note. This made sure that whatever I had got it into my head is done the right way, I was very angry with my tutors as they never allowed me to step up quickly but in hindsight it makes sense; unlearning something learnt wrong takes a long time than learning it to do the right way.

mini-one-862815_1280

When stepping up, change only one parameter at a time

We learn more and more faster and deeper when we are able to understand the effects of our actions. The more we are able to identify the cause and effects, the better is our ability to sense the patterns and apply learnings at a more abstract level. Stepping up the learning through one parameter a time allows us pinpoint the cause and effect, helping us to save energy. Someone shifting to 3rd gear and also driving in rain for the first time on a slippery road will not lead to effective learning.

Once comfortable, keep getting exposed to new conditions.

When we are comfortable in a new skill, chances are high that we get stagnant. Routinely exposing us to newer challenges helps to sharpen as well as prepare us for unknowns. The comfort zone is where the instincts take over the conscience, but when the situation demands we need to switch back to the conscience and make quick judgements from our experience. Good drivers are not usually worried about bad weather or poor visibility, instead they know when to stop driving and when to carry on.

Listen to others, over time we may pickup bad habits.

Habits change as soon as the feedback stops, I used to drive without co-passengers for an extended period of time and developed a tendency to jump over potholes and speed breakers. This annoyed my co-passengers and until I heard from them, I never realized that I had developed a bad habit. It is very difficult to sustain best practices or get a new one, getting a mentor or coach or even just plainly listening to other’s observation will help us a lot.

Never underestimate the power of a coach or a mentor

The coach in the operating room published by the New Yorker magazine tells about how an accomplished surgeon realised that even the best tennis players had coaches and decided to take a coach with him to the operating room. The doctor was doing a thyroid operation which he has done many times and the coach had never done that type of surgery so there was no expectation on useful feedback. At the end of the surgery the coach surprised with a lot of detailed observations which were overlooked and missed by the team. If we are to be professionally better at something then get coaches, does not matter who we are or how successful we are.

When learning stops, we regress

Conspicuous Busyness

During my visit to a library in my neighbourhood, I usually see two contrasting groups. One is a group of retired men mostly in their 60’s and 70’s and other is a group of college students. The contrast is not in the age but the life in the togetherness of these people. The retired men’s group is lively, people are chatting and arguing, cracking jokes at each other. The student’s group on the other hand was too silent with sporadic comments like ‘Hey check what I have just shared’. The student’s group was hanging out in the virtual world in texts and memes.

We always seek joy, we will always find ways to remain happy but we give more weightage to frivolous joy than momentous ones. Short term goals or immediate gratification gives joy because of the reward cycle of our brain; we are trained to get satisfaction by being busy and believe we can get great output just by rote activity, just a vestige of industrial revolution practices.

I read this story somewhere sometime ago — A poet who has not produced much in the recent times took a boat and rowed to the middle of a lake. He thought the distance from the people and quiet atmosphere will lead him to good train of thoughts. It was nightfall so he lit a candle, pulled out his notebook to write but no thoughts occurred to him. He was flipping between his old poems for inspiration and also closed his eyes tyring too hard to concentrate. Hours went by and nothing happened, he thought reading some book will help him and started to read, but it got windy and he had a hard time to keep the candle from getting out. After several minutes of struggle to keep the candle burning, he gave up in frustration and the wind blew the candle out.

moon-165487_640Until the candle was burning bright, he was missing a beautiful sight of the full moon which gracefully illuminated the entire lake, the valley and the surrounding hills. That whole evening he had been waiting for an inspiration to write and it was just lying in front of him. The wind by blowing out the candle, did the trick; the sight in front of him was so spectacular that it inspired the poet to produce one of his greatest works.

 ‘I am not producing anything creative, so why ask me not to browse and consume information always? I enjoy this more than our conversations’ — This is a common retort when you remind people to be present in a situation and not fiddling with their gadgets. It is gadget to be very specific as books are not very destructively distracting, books cannot cater to a short attention span and can not provide on demand entertainment.

A study was done on rats for addictive behaviour, a lever was provided in a cage which provided a dose of cocaine when pulled. On discovering this, the rats kept pulling the lever for more and more doses of cocaine until it killed them. Our neural pathways for reward is not very different from that of rats, we would soon get hooked on to joyful behaviour even though it will cause us harm in the long run. The only way to get away from this is to consciously avoid the ‘pull to refresh’ hooks in the gadgets and set some quality time for leisure. Fortunately delayed gratification is an acquired skill, there are enough studies like ‘The standford marshmallow’ experiment suggest that people disciplined enough are way more successful.

Leisure is one of the best rewards, it is very hard earned and should be spent wisely. There was once a time when people predicted that the world will be with so many machines that people can accomplish their work within few hours of work everyday and pursue their hobbies; fast forward to the digital revolution, the work and leisure no longer has boundaries, in fact there are no boundaries and distances at all.

If socializing & leisure is forgotten due to blurred boundaries then the only reward we have is to keep ourselves busy. We reach out to our pockets to consume information to gratify the need to have a sense of usefulness. That leads us nowhere, it is just being busy for the sake of busyness, some sense of self worth.

It is okay to say that we have free time, of all the years I spent when I look back; the only thing in memories are those mindless chats, sports, games, hobbies etc. None of the memories like going through social feeds for hours together is so memorable, those seem like a blip when looking back compared to the memorable ones however small in duration they were, seems very vivid. By being deliberately busy we are just passing time, not living.

Dev huddles – The andon cord of software development

HelpToyota introduced the andon cord in their manufacturing lines to help people to stop the production line and alert the people around if an abnormal situation arises. It is a cord that hangs above the head within easy reach of any to help to immediately gather the attention of the others. While the rest of the industry was treating production line is something that never should be stopped and let quality control take care, this one put power in the hands of people on the line to take a call to improve quality by attacking the problem at its source.

The idea then widely got adopted in different forms like ‘Help’ buttons in various manufacturing sectors. It was easy to adopt in manufacturing as we can see what is going wrong and gather together to immediately fix the problem. In software development it is not obvious when to pull the andon cord and how to collect thoughts of people on what to solve.

huddle

Dev huddles are the answer to the andon cord in software development. Huddles are very common in any team sports, team members quickly huddle to celebrate a goal or discuss a plan. The team also optimises over time to communicate very effectively in fewer words and quick time.

When to call for a dev huddle?
We need to call for a dev huddle when

  • Our programming has deteriorated from a flow to brute force or trial/error method. Manuals did not help to resolve and even a quick help from another team member did not help.
  • We find a badly developed code and need to bring it to attention of the team when it is still fresh in mind.
  • We are about to make a major change in the code base and every body needs to be aware of the incoming change so they are not surprised.

How to make sure that we don’t waste other’s time?

Team time interruption is expensive, so before calling for a dev huddle we need to make sure that we have all our show and tell pieces ready for discussion. One example is when stuck at a problem, quickly jot down the problem and the steps tried to resolve that did not work on the whiteboard in diagrams or words; then call for the huddle. If the resolution or direction is not found within 10~15 minutes, break the huddle to come and do a detailed research on the problem.

Why should dev huddles work?

Dev huddles work on the idea of crowd wisdom, the average output of a group is always higher than best individual in the group in most cases. Ideas & solutions can come from any one provided they are given a good explanation about the problem. Dev huddles also help in knowledge sharing in a terse & effective manner.

The big bridge that was replaced twice and no one noticed

I got a chance to walk on the Golden Gate bridge, could not stop admiring the beauty of the surrondings and the bridge. One can see what the bridge is subjected when taking a walk on this one, I arrived at one edge of the bridge when it was bright & sunny and by the time I reached the other end it was misty & foggy. It is constantly pounded by waves & high speed wind; and takes a lot of traffic. It left me curios to find the engineering behind it, after a while I stumbled on a documentary made by Nat Geo titled ‘Impossible bridges – Golden Gate’.

The engineers thought that it is impossible to build such a bridge because the water was too deep and winds were too fast and the distance is too long to cover. Even if built people wondered about how could it be sustained. Nat Geo’s documentary showed how the crew worked over the years to strengthen the bridge, rework the floor without disrupting much traffic flow. There were innovations like suspended traveling platforms to work on the cables. The bridge withstood an earthquake and was  strengthened to withstand stronger earthquakes. At the end of the documentary, the narrator mentions, he is sure that all parts of the bridge has been replaced at least twice but due to the engineers’ skill, people never knew that they are looking at a new bridge.

Golden Gate

The same holds good for long running software projects, many softwares over the course of time change so much internally based on the people working on it, which Richard Gabriel calls as Habitability in his book ‘Patterns of software architecture’. If the engineers who took over the maintenance of Golden Gate have not replaced the bridge over the course of time we would have a big heap of metal lying on the sea bed due to structural erosion. It is the same analogy that applies to software called ‘big ball of mud’ due to software rot.

The biggest reason given for software rot is that there is not enough time. Architecture is a long term concern, little beyond the learning horizon as mentioned by Peter Senge in his book ‘Fifth Discipline‘. We are not able to observe the effects of software architecture in the near term as there are very urgent business needs that keep coming up and they are never tangible like bridges; visualising them requires experience and skill. The advantage a software has is that it can be made more robust even after it has been built hastily.

How do we find that the architecture & design needs a revamp?

  • If the change to the software looks simple and trivial but takes a bafflingly disproportional amount of time to fix.
  • If it is difficult to visualise the code and explain to others in simple abstractions.
  • If a technology upgrade looks infeasible without a rewrite.
  • If there is too much dependency on individuals and context.

The revamp is not just about modular design & clean code but also adequate test coverage along with a robust continuous delivery mechanism similar to how the Golden Gate’s engineers used suspended traveling platforms to carry out their repair work on the cables without affecting traffic. A successful well maintained app that takes advantage of new technologies and changes hands would definitely been written many times over and no one would have noticed it.

How far can we pile on debt?

We are inept at perceiving the effects of debt, we grossly underestimate compounding factors and fall into a debt trap. Debt is tempting, it gives that edge to own things which is not possible now but only with the future effort that we can conveniently borrow and buy. There is a saying in Tamil, buying something in loan is like lighting a fire with borrowed hay but to repay it has to be paid in wood for same volume. If instant gratification takes priority then debt that needs to be repaid will hit us hard and bring us to a grinding halt. Corporates love debt, because with a lower cost of interest they will be able to create a high value output. This is where debt helps, if we are able to create something that is more valuable that can repay the debt with interest and still leave us with lots of money then debt is good.

Developers take debt analogy to code and create technical debt to get features fast to production, one major drawback is that most of us don’t understand monetary debt and its impacts well even though it can be quantified; but take decisions on technical debt which is difficult to quantify.

Let us have a look at an analogy, Neo has a trash can in his area that has to be cleared periodically. Clearing the trash can whether it contains little trash or full can of trash takes the same effort, so he prefers till the trash can fills up. The trash accumulates in a way that it doubles every day if not cleared. A new person Brio takes up the place of Neo, Neo explains that the first day only a gram of trash gets generated after clearing, but if left uncleared it doubles every day. He used to clear it every 10th day as it is around 500 grams by that time.

Brio observes that the trash piling up day to day is trivial to care about, around the tenth day he notices that trash is 500 grams and barely fills the can, he decides to come back 10 more days later forgetting the fact that it doubles. 10 days later what was a small can trash turns into an unmanageable heap of mess which is about 500 kilograms. The power of compounding is too easy to miss in the initial days, it will hit us hard who are used to linear maths.

Same is applicable for technical debts in an application. Tradeoff decisions have to be recorded and visited at frequent intervals to see if it is causing any harm than the convenience it provided. Technical debt also paves way for ‘Broken Windows’. Software development is a social activity, it will be very easy to make a broken window the new norm when code changes hands. Run your tech retrospectives; review code, design and architecture as an entire team at frequent intervals, that will help to manage technical debt better. Remember, it is not like corporate debt which is manageable by numbers.