Pull request is the new hammer

Building software is not an engineering problem, it is a communication problem between engineers and business. Agile Manifesto‘s first and foremost point is about the individuals and interactions. It is imperative that the feedback loop should be as short as possible to reduce the cost of late rework. Each team in an organization would be unique in terms of composition, their communication and means of interaction will be fine tuned for that particular group. The teams should be guided through non negotiable philosophies rather than intricate set of policies, restrictions and tools.

Cliche – ‘Any code can be understood by machines, code should be written such a way that it should be readable by humans’

MatrixCode is read many times more than it is written, so it is everyone’s desire to keep the code base very neat to adhere to good coding standards. One way of ensuring this is to do a code review. With Git the collaboration got easier that you ask anyone to clone your repo, do some work and contribute back through a pull request. Pull request is the only way of contributing code to a project which is otherwise a closed read only repository. The owner reviews the pull requests and decides what goes into his/her repo. The feedback cycle in this manner is too slow but works well for social coding.

Cliche – ‘If you have a hammer then all your problems look like nails’

It is becoming increasingly difficult to gather the right requirements, translate them quickly into working software and that too in an efficient hammer-147840_640manner. It is necessary for us to be able to roll up information from the environment and translate that into the software. To help the evolution we should be able to keep the feedback loop as small as possible so that it is very easy to add the right functionalities at a good pace. I happened to read a nice writeup on the importance of rolling up information from the environment. Many people fail to visualize what their intent is in selecting a tool in the first place, pull requests work very well in controlling what goes into someone’s personal project. It will also be successful in team setups, but it comes with a cost. The cost is the speed of evolution, as it will take longer and longer to sync the commits when the codebase grows.

Cliche – ‘Give a carpenter a latest power drill instead of a hammer, he will use it as a hammer’

penguin-158551_1280In the SOA context it is a double whammy where there are many micro repos and pull requests are used to control the quality of the code. Choice of tools and technologies are always like rock, paper & scissors; what works in one context will not work in the other. Choose the right one considering the intent and moving targets in all dimensions in the system, change the approach if it does not fit in or the situation has changed.

1 Comment

Filed under Developer

It is just a road not a roadmap

Roadmap A typical roadmap is something which gives a chosen possible path with alternate routes, options and milestones. But these alternatives and options stop at the map level and do not move into product roadmaps in the industries. It is very common to see product roadmaps published with milestones and when to expect them but all of them are linear. It is not a roadmap, it is just the road ahead which is more or less static. Responding to change is possible only when you have a roadmap, else every problem that comes in the way puts us in a hardship as we were not prepared to respond with alternatives.

How do we come up with a roadmap for software projects? Just like maps the requirements have to be visually triaged. We need to collect them at granular level such that it can be captured recorded in one or two sentences that can be written in bold on an index card. These granular requirements are popularly called as stories which represent an action of an end user or the ability of the system.

Once we collect most of the possible actions and abilities then we lay them all together on a large table or put it up on a wall to see them all at one glance. This helps us to triage quickly and discard duplicate ones and add some more if they were found to be missing. Once we have come up with an initial list then we collect the cards as per the priorities and arrange them linearly. If a requirement can be done in parallel then a parallel stream is introduced. In this manner we will be able to have a visual data on what are the dependencies and how much can be done in parallel.

We should play around the sequence and parallel streams to come up with multiple options for milestones and these multiple options together constitute a roadmap. Every few months it is better to revisit to the roadmap as software projects are always a moving target. This is nothing but the release planning meeting mentioned at XP website. We have to be aware of the options and not just rely on the first release plan that we have created, if we rely only on the first draft then it is just the road ahead; we will achieve all the milestones as per the plan but it may mostly lead to a not so desired location.

Leave a comment

Filed under Uncategorized

Packaging functional tests using TestPackage

TestPackage provides a clean way to package your functional tests and run them as a jar file with sharding and package level run options. All that we need to do is created a maven/gradle subproject for functional tests and have the testpackage’s dependency and manifest file entry to get an executable jar package.

Once the jar is created all we need to do is run ‘java -jar the-test-package.jar package.that.contain.tests’. This helps a lot in continuous delivery where the functional test artifact can be created at the build stage and promoted to run at the functional test stage without having to access the source code and rebuild later.

Leave a comment

Filed under Developer

Using Log4J2 with SLF4J in a Spring 4 application

In one of the spring web applications we decided to use Log4J2 with SLF4J; when we deployed the web app in tomcat server along with the required jars we stumbled on the issue multiple binding for SLF4J and spring dependencies not getting satisfied. We had used Gradle and there were transitive dependencies which included log4j-12xx.jar and slf4j-log4j.jar as some of the dependencies in the project were using log4j version 1 for logging. Here is how we were able to start logging using log4j2.

Exclude the log4j-12xx.jar and slf4j-log modules from the gradle build

configurations {
 all*.exclude group: "org.slf4j", module: "slf4j-log4j12"
 all*.exclude group: "log4j", module: "log4j"
}

Add the following jars

org.slf4j:log4j-over-slf4j|route all the log4j v1 to slf4j
org.slf4j:slf4j-api|slf4j api
org.apache.logging.log4j:log4j-slf4j-impl|log4j2 bridge
org.apache.logging.log4j:log4j-core|log4j2 core
org.apache.logging.log4j:log4j-api|log4j2 api

Add the log4j2.xml configurations to the classpath or as specified at log4j manual

Leave a comment

Filed under Developer

Build it like a sports team

Is it easy to get a crash course in football by Pele or Maradona for a week and produce a world cup winning football team?

Answer is NO. Then why do lots of people in the corporate world think of hiring scrum trainers & expert developers to train their team for a week and then expect their team to be undergo a transformation at a magical scale?

Football

German football team made it a point to transform their team and it took them a lots of years before they were able to reach the pinnacle. A quick side by side comparison of what is causing agile transformation to fail.

Vision

Football: Someone was there owning this entire transformation, the German football association spent a lot of time identifying talent in their teens and groomed them.

Office: In the corporate world switching jobs every few years have become common, but there is no passing on of the context, resulting in the new person taking charge, starting from all over again, as well as frustrating existing good performers who have to rebuild the perception.

Infrastructure

Football: Players expecting state of the art training facilities, fitness programs and new shoes are not a luxury, it is a necessity.

Office: Computers have become so cheap compared to the salaries, yet the policy of providing the best tools and good work environment are archaic.

Coach vs Management

Football: Coaches are given their due powers to help the team achieve the goal. It is very easy for anyone to comment on how professionals should play their game, there would be no use adhering to the metrics if the team cannot win. Winning is the only measure for management.

Office: Often coaches are seen as part of the management or management tries to heavily influence coaching which results in a team which will work either for metrics or to please higher ups without the actual result that it had aimed for.

Training

Football: Just the ability to kick the ball does not make a footballer. Training will be introduced to increase physical strength & stamina, better mental wellness, injury prevention, tactics and strategy.  A heavy investment is made in the training facilities.

Office: In the software industry a generation is about 2-3 years. Computer science degrees are nowhere near what is state of the art in the industry. At many places the on boarding process is either very shallow or not up to date with the recent developments, leaving people to learn most of the things hard way. Given too many things to learn and the information overload, this results in inefficient learning and application of knowledge on the job. We need to prepare people to find answers that are not available on Google.

Team composition

Football: Rookies don’t learn by watching greats from the bench, instead they play along with the veterans. Every sports team makes sure to have the right composition with a mix of rookies, emerging players and veterans. That is how they sustain a winning team.

Office: Architects and Leads often do not code or not part of the team every day, this means that most of the time the team just looks up to for advice or waits for reviews. There should be a good mix of people at all experience level so that there are enough people to try new things, enough people who have mastered few things and enough people who challenge change.

Above all – Persistence

Nothing in the world can take the place of persistence. Talent will not; nothing is more common than unsuccessful men with talent. Genius will not; unrewarded genius is almost a proverb. Education will not; the world is full of educated derelicts. Persistence and determination are omnipotent. The slogan ‘press on’ has solved and always will solve the problems of the human race. ~ Calvin Coolidge

Image courtesy of Salvatore Vuono at FreeDigitalPhotos.net

Leave a comment

Filed under Help me help you, Workplace

Building applications ‘Ship Ready’

Last mile problem is ubiquitous. Most of the developers think that the deployment or production readiness is not their cup of tea and stay focussed at getting their output for an input given according to the specification. Usually the last week of the release is chaotic with many dry runs and hot fixes. It is so easy to avoid the last mile problem yet so many people avoid those ways. What if the inertia of setting up build scripts & configuration are taken away? Will it be easier for developers to build ship ready applications?

Spring Boot and Gradle is a great combo to get started in the morning and ship a prototype in the evening. Much of the inertia that is present in writing ant scritps and configuring a spring application is blown away by making very simple entries. Here are some examples that makes the life of a developer easy by eliminating the complexity involved in project setup and deployments.

gradle wrapper – This creates a wrapper script for gradle such that only the first developer who sets the project needs to install or have gradle on their machine. From the next developer onwards who checks out the code, they can start using gradle through gradlew script. No installations required except a JDK and a VCS.

gradle build – If the Spring Boot’s starter web is used then this creates a uber jar with an embedded tomcat and all the library dependencies. All we need to deploy is run ‘java -jar myapp.jar’ in the required locations. Configurations can be overridden either through command line arguments or through external property files.

gradle eclipse or gradle idea – Helps in setting up the project in IDE. Once imported in IDE running a web app in debug mode is just right click, debug application on the main class.

gradle bootRun – Command line way to quickly bring up the application and test it as a whole.

Building a 12 factor app with gradle and spring-boot is very easy and uses less effort. Get the project setup with this and pair it up with a continuous delivery tool, you get a ‘Ship Ready’ application right from the first day of development.

Leave a comment

Filed under Uncategorized

Engineering practices – What is that?

What does a boom in an industry bring in? It brings in a shortage of talent.

What does shortage of talent bring in? It brings in a great demand for the talent.

What does great demand mean in a shortage of talent? It increases the supply in an inorganic manner and inverse vandalism occurs.

In a country called Noviland, carpentry was an expensive business dictated by undersupply of wood as the wood cutters only had access to axes. This kept carpentry and arts related to wood accessible only to the elite and the academics who studied carpentry. This brought in many skilled and intelligent people to the field who were well respected and well paid, people were committed to workmanship. Even though the forest cover was huge in Noviland, technology of axes led to a limited supply.

A wood cutter’s daughter invents a circular saw that can be driven through the bicycle gears which helps to cut trees at 10 times the pace they were used to. Suddenly the supply of wood increased ten fold and there were not enough carpenters. The demand for carpenters grew way too much that it drove their paychecks very high, prompting people to switch careers. Those who learnt other trades, tools and works now did a simple study of hammer, nails & saw and jumped into the profession.

Maintaining old work was easy as everything was in place and simple maintenance tasks were all required to keep up. New work is were all the hell broke loose. People were now getting furniture that does not last long, wooden bridges collapsed on a few days of exposure to heat and rain. People who wanted to replace their furniture dreaded at the cost of replacement and replacement failures. There were way too many carpenters and finding passionate people was next to impossible. Carpentry became a high paying job where no passion or study was required, all it matters was to know about hammer, nails and saw. Some of them who did exceptionally well were immediately made a manager and given interns to manage thereby killing passionate carpenters in the interest of scale.

Software engineering in a few countries is at this state, merely having a crash course in Java and understanding of few libraries are enough to be called as a programmer. Programming is not just writing code, it is solving a communication problem which involves many aspects. Lots of productivity is lost because of the failure to understand and implement good engineering practices. Some people pick agile as a tool to enhance productivity but do nothing to improve the capability of the developers. There are so many practices which can help us develop and release software in days & weeks but many of us are still stuck at the timescale of months & years because of the ignorance of practices as well as reduced progression on the skill level.

Below is a table which helps us to find where we stand in terms of skills, I can say with confidence that even with 10 years of experience many of us will not rate ourselves proficient. We should let skill and passion dominate engineering fields.

 Table source: http://science.raphael.poss.name/programming-levels.html

 

Leave a comment

Filed under Developer, Workplace