Sunday, October 11, 2009

Software Bureaucracy: Layering Optimizes Downward, Not Upward Collaboration

The Layering architecture style is one way to structure an application and separate concerns into a hierarchy of layers. The question is then: are layers efficient? aren't we adding more obstacles and making things more complex? What gives?

It All Started With Delegation
Layering is just organized delegation. Delegation is good. Before Layering, there was delegation. To enable a component to do more, it delegated the task to another component to be executed. That's one layer created already. Everything was fine with one layer (flat structure) until the tasks in that first layer became more in number or more complex and it became more efficient to delegate to yet another layer.

It is also important to note that a delegatee should be usable by any delegator. It should not be bound to one delegator. This promotes reusability, scalability and maintainability.

As you can see, each layer is created out of a need to optimize for the layer on top. There's nothing wrong with that. If we don't layer, and go with a flat structure instead, the top layer will have to communicate with all, decreasing its efficiency and defocusing it from its main responsibility, as a result, leading to ineffectiveness. Total failure.

Lower Layers Should Not Call Upward Ones (Software Bureaucracy)
Just like a delegatee should not be bound to one delegator, a layer should not know or be bound to one specific layer above it, or depend on the layers above to do its job.

As you have observed above, the whole strategy behind the layering architecture style it so focus the top layers on their main responsibility to achieve more and better results. The key to remember here is that layering was not fabricated to help the bottom layers.

What happens when the lower layers initiate the communication going upwards to each layer? Inefficiency, decreased performance and low maintainability. Because now, lower layers have to keep track and know each top layer in order to initiate communication with them and to do their job. This adds cyclic dependencies and chaos in general. Usually, these kind of software applications are not extensible and provide a very low return on investment.

When this mis-application of layering is done in the business world in corporate hierarchies, it is called bureaucracy, and that's not a good thing. I chose the term Software Bureaucracy as a term for this condition.

Because of this misunderstanding of the layering architecture some people dismiss the whole style altogether.

Then How to Communicate Upwards?
Communicating upwards is best done without dependency. Dependency on upper layers is at the core of the problem. The best way to implement that is through the publish-subscribe pattern or the Observer pattern. If any component is interested in the lower layers' work, then just subscribe to their events. This achieves the best of both worlds. Top layers are now in-the-know and lower layers do not have to depend on communication with the top layers. They just fire and forget. No obstacles in their way.

This architecture can be extended to the business world to benefit from. Keep the corporate hierarchy optimized but empower the lower levels and employees to make their own decisions without the need to depend on top managers. To keep top managers informed, employees can update one central corporate system about their current issues and status.

Who said that business cannot benefit for software architecture :)

Tuesday, October 6, 2009

Executable Architecture: Design Must Be Incremental

Incremental Design vs Complete Upfront Design
No matter how genius humans are, in my view, it is impossible to have a complete design that works perfectly from the first time when innovating a new software or inventing anything for that matter.

We hear the emphasis in real development situations being on implementation to be incremental but we rarely see incremental design being stressed out.

Increments must always be created twice: once in design and once in implementation. Why design? Design simply means think. Think, then execute. It is as simple as that. On one extreme, some developers got it backwards. They jump straight into coding first, then they think about it, afterwards, if they even do that. On the other extreme end, you got the ones who think about the total design first, bordering analysis-paralysis, then totally execute. The correct way is usually in the middle, neither extremes. Think (a.k.a. design) a little, execute a little, then think a little, then execute a little. From my personal experience and observation, this seems to be the optimal approach.

An analogy would be the journey of discovery. It is like entering a dark cave with a flashlight. You can only see (design) a few feet ahead. You don't know all the possible shortcuts or dangers that lie ahead. Every time you walk forward (implement) you might discover new crevices or nooks that you weren't aware of or dangers lurking to avoid.

Similarly, for new product ideas to succeed, the development approach is always a cycle or an iteration: design a little, implement a little, get feedback. Then, take that feedback, add on top of the first design another little incremental design, implement, feedback. Keep repeating until you're satisfied with the results and you achieve the complete work flow or requirement.

Once we have a complete product or we're in post-production, now we can manufacture or "assemble" as many pieces of this design as we want. As you can notice, unlike design, assembly can be planned in one big shot. This is because the design has already been proven to work and we are past the discovery phase.

What is executable architecture?
Having established the above, it follows that since architecture is a design activity, there's no such thing as an initial complete architecture on paper, then complete implementation. This is old style. It doesn't work. Avoid it like the plague. This was where the architect sat on a high pedestal and all he or she did was draw diagrams and hand them down for implementation. This perception of an architect is archaic.

The realistic and most effective way of creating a robust software product is to start with an architecture increment then implement that increment and then repeat as mentioned above. Start with a core architecture that addresses the significant use case scenario. Stop. Implement it. Get the feedback, then add on top of it. To distinguish this architecture from the paper-architecture I like to give it the term Executable Architecture. It can be executed and the results can be observed.

Software Architects: Keep your development skills sharp. The architecture is the actual code not the paper. The main benefit of paper or UML diagrams is to communicate, document and understand the architecture.