Complexity and Effort

Let us start with a simple diagram. On the horizontal axis (X) we represent the complexity of the software problem to be solved. On the vertical axis (Y) we represent the cost, meaning the effort required to build a software solution.

The scientific community agrees on one point: as problem complexity increases, the effort required to solve it increases as well. This growth is not linear: in many cases it is more than proportional, and in some situations it can even become exponential.

A highly simplified interpretation of software engineering is precisely this: finding ways to address increasingly complex problems while keeping effort within reasonable limits (following the green curve rather than the black one, see the next diagram). Over time, the discipline has developed languages, architectures, methods, and tools that shift the relationship between complexity and cost toward more manageable solutions.

Different Technologies, Different Curves: Point Q

The same type of diagram can also be used to describe the effect of adopting different techniques or technologies. To make the concept easier to understand, let us use a deliberately classic example: imagine that the black curve represents the typical complexity-cost relationship when using a structured programming language (but not an object-oriented one), while the green curve represents the same relationship when using an object-oriented language. The reasoning applies equally well to today’s technological choices: monoliths versus microservices, synchronous orchestration versus event-driven systems, manually managed infrastructure versus infrastructure as code.

The green curve intersects the Y axis at a higher point. This means that adopting more structured solutions requires a greater initial effort. Object-oriented programming is more demanding than programming with a structured language. Writing a microservices application is more complex than writing a monolithic application.

The two curves intersect at one point. Let us call that point Q on the X axis. Before Q, the less structured solution costs less. After Q, the more structured solution progressively becomes more efficient. This means that certain techniques only make sense for problems whose size is equal to or greater than Q.

This apparently simple fact explains many recurring discussions in the industry. Even highly competent people who are accustomed to problems smaller than Q perceive a set of techniques and methodologies as unnecessary overhead, when in reality those techniques only make sense for larger problems. It is not a competence issue: they are simply looking at the same curve from a different point on the X axis. One could say that every technology corresponds to a certain curve, and that for each one it is necessary to evaluate whether it should be applied to a specific problem.

The Effect of Experience

When a technique is used repeatedly, its initial cost tends to decrease. If a team has already learned object-oriented programming for one project, the cost of that learning does not need to be paid again in subsequent projects. Graphically, this is equivalent to shifting the green curve downward. The light green curve in the following diagram represents the same technology as the dark green curve, but applied by people who have already mastered it.

For this reason, more experienced professionals tend to use structured technologies even for relatively small problems: for them, point Q has shifted to the left. Those who are not yet familiar with those tools perceive the initial cost more strongly and therefore tend to prefer simpler solutions.

The difference does not concern technology alone. It concerns the competence of the people using it. The ability to achieve efficient results depends both on the chosen technology and on the people responsible for implementing and maintaining it.

The Red Curve: Technology Used Poorly

There is one final aspect of the diagram that deserves attention. When a powerful technology is adopted without the necessary competence to use it correctly, the result is not neutral. It is worse than what could have been achieved with simpler tools. In the diagram, this scenario corresponds to the red curve.

Technology alone is not the solution. The solution is technology connected to the way it is used. Java is an excellent object-oriented language, but using Java is not in itself a guarantee of writing quality applications. It is possible to write a Java program without errors, one that compiles correctly and can be executed, and that is nevertheless terrible in terms of maintainability and beyond. In another field, one might say that a good vocabulary and a good grammar book are not enough on their own to make someone a good writer.

The Role of Software Engineering

This issue is highly relevant in computing and involves three dynamics that influence one another and are worth distinguishing.

The first is commoditization. The functionalities that software offers today will tomorrow be considered the bare minimum by its users. To keep up, software products continuously offer more functionality, which means developers are forced to deal with increasingly large applications. The fact that users consider certain features obvious does not mean those features build themselves. Put differently: as time passes, the problems to be addressed become increasingly larger, and developers are practically forced to adopt new techniques and technologies, sometimes reluctantly or without having significant expertise.

The second is hype. Every era has technologies that become especially fashionable, and sales departments rely heavily on them to promote products. The logic is B2B2C: “buy my system because it uses Mongo” (instead of old and obsolete Oracle). Every historical period has its buzzwords: multimodal systems, blockchain, big data, IoT, enterprise service bus, metaverse, and more recently AI. None of these technologies is necessarily wrong. On the contrary, they most likely offer useful possibilities. The problem is that high hype usually arrives before mature techniques and methodologies for using those technologies properly. It is easy to misuse them, and organizations are often pushed to adopt them not because of technical reasoning but because of commercial pressure.

The third is maturity. Problems can also arise with technologies that are already mature and have exited the hype cycle, but the dynamic is different. A mature technology becomes easy to use: today, writing a REST web service is accessible to many people. The issue is that a technology that is easy to use is sometimes overused or used inappropriately. Technology makes it possible to write a web service. It does not explain when it is worth writing one. In this case, the risk is not incompetence but overengineering: adding complexity where it is unnecessary.

The problem arises when these three dynamics overlap and awareness of a technology’s actual purpose — and of the theory required to apply it appropriately — is lost. The risk is following the trends of the moment, choosing a language or database without real criteria or deep understanding. When this happens, people end up following the red curve of the diagram. Someone who does not know how to program in an object-oriented way and tries to do so anyway may produce a worse result than they would have achieved with a less complex language.

Software engineering means using the right technology at the right time in the appropriate way, because only then is it possible to address complex problems effectively. This is why our website does not contain a long list of buzzwords and acronyms corresponding to the latest trend. We are confident that we know many of them. We are even more confident that we will use the ones that are truly necessary.