I have written a lot about abstraction. I think that it is something at the root of design, at the root of understanding something. There are various benefits of abstraction and it can be effectively used to simplify.
A lot of times, especially in discussions, it has been implicitly considered that abstraction reduces complexity. It does not! Complexity is like total amount of energy – constant. I do not remember where I had read this, but it conveys the message. Complexity cannot be reduced. However, you can use abstraction to provide a view that someone else can comprehend. Simplicity is a perception of the user. One of the factors in this perception is the knowledge. If you can provide a view to the user which the user knows about, then it plays to the perception of simplicity. Abstraction can be effectively used to do this.
However, this does not mean that the actual complexity has been reduced. For example, a car manufacturer can provide the abstraction of a car to the driver who does not have to know and worry about the innumerable mechanical, electrical and electronics tricks that makes the car work. Once the car maker provides a view that a driver can understand he/she can drive, that is, use it. This does not mean that it has made it any easier to make the car. The complexity has not been reduced or eliminated, it is still in the place, it is just abstracted into something that the user can understand.
What this means is that abstraction can be effectively used to provide different views for different users according to their knowledge and skillset. This I feel is the key to providing simple and productive interfaces to users, and something that directly impacts the ROI of a software.
March 14th, 2007 at 3:00 am
The complexity of the problem clearly remains constant, but I wonder about the complexity of the solution. If you want to get across a stream, there are solutions more complex than others; a log placed to bridge the gap would be one of the simpler, whilst a trebuchet combined with an improved parachute one of the more complex.
Thus, it seems perfectly possible that the amount of complexity in a program could be reduced (as the program represents the solution, rather than the problem), and it also seems possible that, in some cases, abstraction could achieve this. Of course, this does nothing to reduce the complexity of the problem, but at least in theory, the problem is a fixed constant anyway.
March 14th, 2007 at 4:39 pm
I disagree with you on this one Abhijit. Abstraction does make it less complex to make a car. Concepts like engine, brakes, exhaust, seats, chassis, gearbox, headlights etc. are all abstracts and lets the manufacturer outsource, reuse or buy parts from elsewhere. The car manufacturer actually doesn’t need to know the details of how each part works or is implemented. He just needs to know they can be used.
Abstraction lets you break a complex problem into several simpler ones. You don’t have to keep everything in you head all the time… It lets you build upon what has already been done and focus on what extra functionality you want.. much simpler than starting from scratch. I can make a form in .net in a few lines of code, I don’t need to know anything about hardware, drivers and operating system. I find this less complex.
When it comes to user requirements I find that a lot of clients have very specific demands. Once I have helped then make the requirements more abstract, the requirements are much simpler to implement.
As abstraction lets us solve problems that are otherwise too complex, I think abstractions provide less complexity.
March 14th, 2007 at 4:50 pm
Weavejester, I agree with you that there can be simpler solutions. However, within a solution, the complexity is still constant. Abstraction enables us to present the solution in a way that it is simple to others. Without abstraction, it could have been more difficult. But the complexity is still there.
Rune, I agree with you when you say that it lets the manufacturer have less information. However, it does not mean that the complexity involved in building a car has reduced. Complexity in building a car for the manufacturer has reduced, maybe because it can now be outsourced, it has become simpler for the user to use because he/she understands what he sees. For the user, the manufacturer, who assembles the different components to build a car is a user of those components, the complexity sure decreases. However, not for the builder. Where I am coming from is that the software engineer who is trying to solve the domain problem cannot ignore its complexity on the basis of abstraction. Abstraction is for the user, with whom the piece of software will communicate with. Abstraction sure helps in designing good interfaces.
Abstraction is useful for the builder too, to analyze and hence simplify. The complexity is as it is, but abstraction helps the builder in consuming it piece by piece. The complexity will still have to be acknowledged and enough care taken while building the software.
I realised in some of my conversations that the decisions were simply taken on the basis of abstractions, without acknowledging the complexity involved. To use your example, it is easy to create forms in .Net, but the one who created the framework which lets you do that had to still consider all the complexity.
March 15th, 2007 at 9:46 am
An interesting discussion this.
I like the way WeaveJester puts forth the idea of the log vs trebuchet solution for crossing a stream.
I would say that the reason we are able to discuss a log vs trebuchet solution is because we have abstracted the problem from crossing a stream to traveling from one point to another.
Once we leave the abstraction, and get into details- the log may be washed away by the current, there may be no logs available, there may be no equipment to reach the log across the two points etc. So the solution of a log is an solution of an abstracted problem, which will still need to be verified as a solution of the real problem of crossing the stream.
As we go up the abstraction, the realm of possible solutions widens. However we would need to go back down the abstraction to reach a tangible solution.
So an solution to an abstracted problem would exactly be that- an abstract solution. In that sense, there is no running away from the complexity of a solution.
So I would say, problem abstraction does reduce complexity, thus opening up an array of abstract solutions, but then the abstract solution would need to be ratified against the real problem which re-introduces complexity.
March 15th, 2007 at 11:15 am
Wow, Rahul, you have put it in much better words than I would have. I think this discussion is better than the post itself
March 16th, 2007 at 12:00 pm
[...] Abstraction does not reduce complexity by Abhijit [...]
March 17th, 2007 at 2:24 am
[...] HR units? Abstracting doesn’t solve problems either, just moves them off to someone else. As Abhijit Nadgouda observes: A lot of times, especially in discussions, it has been implicitly considered that abstraction [...]
May 7th, 2007 at 7:43 pm
this discussion really is eye opening.
May 9th, 2007 at 2:27 pm
Abstraction is a mechanism available for human beings to tackle the complex problem. It is hardwired into our brain!
Our brain can effectively process few chunks of information effectively at a time. So, we tend to break the complex problem into many simpler ones and then solve those simpler ones and then ’synthesize’ these solutions and come up with the solution for the original complex problem.
So, please note that as some of the authors recognized, coming up with a solution for a complex problem involves both – breaking down and synthesis.
So, what does abstraction achieve?
It hides details from other components. So, details need to depend on
abstraction than the other way round. So, when you can change details without fearing about the effects these changes going to have on others. So, essentially what have achieved is – you have localized some of the complexity
behind a very simple, innocent looking facade. But, the effect is very profound. In software engg. parlance, you have made your design more ‘modular’.
Now about, complexity of the solution itself –
Complexity arises because of interrelationships between various modules of a system. If you can reduce the number of interrelationships ( called ‘coupling’ in software engg parlance ) you have essentially reduced the complexity of the system at the same time you have improved the characteristics of your architecture like – modifiability, fragility etc.
However, it is very important to distinguish between essential and accidental complexities. We should try to avoid accidental complexity at all costs.
May 21st, 2007 at 7:51 pm
Thanks for your input Prashant.