First of all, I should say, I am neither a physicist nor a chemist, so this post is not about Physic nor chemistry, it’s about software engineering, however, viscosity is still a characteristic of fluids. In this article, I will define viscosity as a symptom in the software development process.
Based on a simple definition from Wikipedia the Viscosity is a physical property of fluids. It shows resistance to flow. In a simple example, water has a low viscosity, as it is “thin”. Syrup and tar, on the other hand, have a high viscosity, as they are “thick”. A way to test for viscosity is the speed at which the substance runs down a slope. Syrup would reach the bottom very slowly, and water would be much quicker.
So what if we map software engineering concepts and the definition of the Viscosity,
|Physics||Software related concept|
|Fluid||* Software development process |
* Development environment you are working in
* The ways you make a decision in your team
* Existing design, code and architecture
|Resistance||The amount of time and money you spend on doing something|
|Flow||* Get something done|
* Release something
* Debug something
* Do something right with a right approach
Why “Viscosity” matters?
When we talk about resistance in doing something or achieving a goal, the other side of the coin is talking about the amount of time and resources we need to meet the goal. The more resistance we have the more resources we need, and in most cases, it means waste. And, the more waste we have not only the more money we lose, but also the more entropy and chaos we will have in performing the activity which ends in an unstable and unreliable product, process, business, team and etc.
In term of software engineering, the viscosity can be divided into Environmental Viscosity and Technical Viscosity.
This type of viscosity is a reason that the developers cannot be productive, and they do not tend to make their best decision during the development process. Environmental viscosity has different reasons, here are the most important root cause for this type of viscosity.
1- The viscosity of tools and configurations
However, it seems using modern development tools help in making better software the amount of complexity and time you need to set up and configuring these tools in the development environments increase the viscosity dramatically. For example Suppose, you have a new guy in your team and she had to set up her PC and start the development if she is very lucky, you have a guideline for her to do the setup, although I can bet, this guideline not only are deprecated and out-of-date but also just cover the sunny scenarios and in case of blockage there is nothing to help that guy, so she has to spend a day or more just for installing and set up the tools.
2- Team culture and Communication Viscosity
Each team, as well as the organization as the whole, has a structure, sometimes it’s an official structure which based on that responsibilities are assigned to different people in the team. Moreover, decision making, impediments removal, problem escalating have an accepted process, maybe an unwritten one, but there it’s. Most of the time teams label their approach to do the development with a development framework or process like SCRUM, XP, AUP, etc. but they do the ceremonies in these frameworks based on their own localized conception, not the main idea in the source development method they used for the title themselves, in such a case, a not only a communication gap might be revealed very soon but also in a worst-case some conflicts might be happening in interactions because of misconceptions.
3- Debugging environment viscosity:
Even if you have tests for are individual parts of your software, you cannot be sure these parts will work properly with each other when you integrate them, the reason is rooted in the inherent the software complexity, which makes it a chaotic system, which the behaviour of the system cannot be predicted from the behaviour of the individual parts because their dynamic interaction is a major factor in the final results. So in this case, to see the system as a whole, regardless of the production infrastructural configuration, you try to do an end-to-end test on your local. It’s impossible to suppose you can develop software without an end to end test in your development environment, or local PC. But the problem appears when the undesirable complexity of setting up an environment to an end to end test is required for testing those part of the system that don’t need the complex integrated components. For instance suppose you use Azure as an infrastructural environment to run your application, and you have used the Azure Queue. Should a developer set up his environment to work with the Azure when he has nothing to do with it and just wants to debug functionality in the system which has nothing to do with Azure. In this case, added complexity of setting up access to Azure for debugging a simple functionality raise the viscosity of your debugging environment.
4- CI/CD environmental Viscosity
Undauntedly, CI/CD is a tenet for this era of software development, but as they are useful and helpful, using them without their effect on the development process might end in a hard to deal development environment. Suppose you have just one build script for your production and development environment, when a developer wants to push/check-in a single line of change all the heavy steps in a production appropriate build script must be executed and each push/check-in costs in a long-running build that takes more than half an hour to finished.
Technical viscosity in software development is the result of violating best practices in software architecture and design. A software with the most notorious design symptoms is the root of technical viscosity, which prevent applying the good decisions in the development. These symptoms based on uncle bob’s categorization are:
Rigidity: Rigidity is the tendency for software to be difficult to change, even in simple ways. A design is rigid if a single change causes a cascade of subsequent changes in independent modules. The more modules that must be changed, the more rigid the design.
Fragility: Fragility is the tendency of a program to break in many places when a single change is made. Often, the new problems are in areas that have no conceptual relationship with the area that was changed Fixing those problems leads to even more problems, and the development team begins to resemble a dog chasing its tail.
Immobility: A design is immobile when it contains parts that could be useful in other systems, but the effort and risk involved with separating those parts from the original system are too great. This is an unfortunate but very common occurrence.
Needless Complexity: A design smells of needless complexity when it contains elements that aren’t currently useful. This frequently happens when developers anticipate changes to the requirements and put facilities in the software to deal with those potential changes. At first, this may seem like a good thing to do. After all, preparing for future changes should keep our code flexible and prevent nightmarish changes later. Unfortunately, the effect is often just the opposite. By preparing for many contingencies, the design becomes littered with constructs that are never used. Some of those preparations may pay off, but many more do not. Meanwhile, the design carries the weight of these unused design elements. This makes the software complex and difficult to understand.
Needless Repetition: Cut and paste may be useful text-editing operations, but they can be disastrous code-editing operations. All too often, software systems are built on dozens or hundreds of repeated code elements. It happens like
Opacity: Opacity is the tendency of a module to be difficult to understand. Code can be written in a clear and expressive manner, or it can be written in an opaque and convoluted manner. Code that evolves over time tends to become more and more opaque with age. A constant effort to keep the code clear and expressive is required in order to keep opacity to a minimum.
Uncle bob in his seminal book “Agile Software Development, Principles, Patterns, and Practices” has added “Viscosity” in this list as well, but as this article deep dive into that topic, it has removed from symptoms list.
Not only this design symptoms, but also bad modularity which has a high level of coupling between modules results in a technical viscosity in the software development, this topic is detailed in “Clean Architecture: A Craftsman’s Guide to Software Structure and Design” by uncle bob, and you can find an overview of the concept here.
Solutions for decreasing Viscosity
- Setup a pc based on your configuration, and while it’s still virgin with no change, make an image from the system as the whole, and when you need to set up a new PC for development just restore the image.
- Having a clear induction meeting with the new team members, create a clear definition for the ways you do the things in the team, regardless of the labels you use for them. And clarifying the responsibilities and team problem solving can help to cure this threat.
- Use software architecture and design best practices to decouple your system and make it modular so that it can be configured to run in a lightweight environment which simplify the debugging process. Using this strategy, each developer can configure it’s under debugging code so that to just load and use those parts it needs.
- There must be different CI/CD strategies for each environment and purpose in addition to a wise branching strategy which helps to have a more agile and safe product line.
- To tackle technical viscosity you need to be loyal to architecture and design best practices. You have to avoid reinventing the wheel. Moreover, you have to pay back your technical debts, if you don’t tackle them on time, they will do it very soon, and at that time, it’s not dealing with “viscosity”, it’s swamping in the mud.
To sum up, regardless of the modern technology stack, you have in hand, and regardless of the title you have chosen for you software production process, there are many ways you can fail in creating software with good enough quality, in budget and in time. All the tools and process invented in the software production industry are for reducing the complexity and the production cost, if using them has increased the viscosity in your product line, there must be something to take care of and revise out there.