Evolution of Systems

In the discussion on evolution of systems, Hugues Bersini (Bersini05) states that:

Of the many types of systems organisations models, those that are in the form of networks and which are based on distributed architectures are the ones that survive and are economically most viable.

The study on dynamic evolution (Fung04) supports Bersini’s statement as the authors explain that one can easily achieve evolution in a component-based distributed system. In this paper, Hay Fung explains the abstraction of components and their connectors facilitates system structures to accommodate changes, and the process of accommodation becomes an innate property of the system.

Currently the software industry is in favour of the iterative and incremental development approach over the traditional waterfall model, in order to achieve flexible processes that handle requirements and reduce the risk by deploying smaller changes. In such an environment, dynamic evolution provides the flexibility in implementing changes to unforeseen and fluctuating business requirements which is very true for the type of economic climate we are now living in. We have moved form the industrial age to the knowledge age.

Evolution is a key factor to software systems, because these types of systems are never isolated, but implicitly constrained by their users, the communication with external systems (man or machines) and the infrastructures (software or hardware) with which they communicate (Oud02). Each adjustment or modification from any connecting peers (interactive participants) may change or affect the internal system, hence the state machine. The system may be provided with some resiliency and intelligence through predefined logic, to either protect itself from these changes or adapt itself to these changes, depending on the hostility of the environment. In other words some decision dominance may be given to the system, defining an aspect of autonomy, and since the working environment is non-deterministic, the system will need to adapt and re-adapt itself, changing its course many times during its life time (Oud02)

Unfortunately, investments in software systems are mostly focus on designing systems based on strict protocols and precise specifications and constraints, thus restricting the systems from its external communication. If we agree that software systems are never isolated, these types of design approach (strict protocols and precise specifications) go against the natural progress of software systems. The need for evolution in the design of complex enterprise system can be mapped to Lehman’s 8 laws of software evolution (Leh80).

We considered Lehman’s 8 laws of evolution to establishing a new discipline to the formulation of a framework which will attempt to incorporate some aspects of evolution within enterprise wide systems.

Law (3) Self Regulation - Global system evolution processes needs to be self-regulating to adapt to change. In mathematical terms, these regulations are pre-defined elements of a subset which are in turn elements of a global set, i.e. the enterprise architecture.

Law (4) Conservation of Organisational Stability - Unless feedback mechanisms are appropriately adjusted, recorded and statistically analysed, average effective global productivity rate in an evolving system tends to remain constant over product or service lifetime.

Law (5) Conservation of Familiarity – In general, the incremental growth and long term growth of systems tend to decline. Over the lifetime of a system, the incremental system change in each release is approximately constant.

The laws 3,4,5 of software evolution, has not been the main point of focus, in this blog article, since they have some dependencies on other laws and are more abstract in nature. For instance laws 3 and 4, self-regulation and conservation of organizational stability respectively, are dependent on robust feedback mechanism which is law 8. Law 5, conservation of familiarity is very dependent on law 6 and 7 which are continuous growth and decline in quality. As a result, the following laws have been addressed in the research.

Law (1) Continuing Change – “A program that is used in a real-world environment must change, or become progressively less useful”. A software system is never isolated and constantly communicates with external entities, which during the course of conversation changes the state of the software continuously. These changes cause variations within the internal system and are required to be managed and monitored to ensure that the changes tallies with the changes of the organisation. Continuous change is the basic law of evolution and is inherently implied even if most of the time, the client does not explicitly state it.

Law (2) Increasing Complexity - “As a program evolves, it becomes more complex, and extra resources are needed to preserve and simplify its structure”. As the number of communication agents (Service Components and network elements) increases, the communication model tend to become more complex and non-deterministic. Within an enterprise wide architecture, the law also applies to the message, as the content of the message tend to become more complex as the organisation matures from a B2C business model to B2B business models, extra resources are needed to preserve or simplify the structure of the systems in terms of manageability and serviceability. In an attempt to address the problematic of complexity, many decision makers are tempted to move towards a more manageable character of a common data model, the substratum of their service bus, yet on the longer term, this solution proves gruelling to scale and adapt to external systems. The very symmetric nature of the data model tends to hinder the evolution of systems.

Law (6) Continuing Growth - “The functional capability of systems must be continually increased to maintain user satisfaction over the system lifetime”. Scalability of the functionalities is key in the design of extendable software solutions. Although software solutions may be well structured internally, exploiting the capabilities of hierarchically structured class libraries, the programs have grown large and monolithic, with a high degree of interdependence of the internal modules. The size of the programs and their relatively high level of user oriented interfaces make them inflexible and discomfited to use for the construction of new functions and applications or new service interactions. As a result, new applications usually have to be constructed at a relatively low level, as compiled programs, which are expensive and inflexible. The high degree of integration at the class library level makes it difficult to construct heterogeneous applications that use modules from different systems, since there is a lack of abstraction. This problem can be addressed by distributed systems which allow software modules to be represented as either stand alone or pluggable components in a distributed application through defined interface or configurable resource adapters. This technique will require the compositions of meta models and the continuous process of abstracting (reinforcing the meta models from lesson learned) and concretizing (deriving pluggable components from the meta model to satisfy the custom requirements of specific scenarios), hence scaling the capabilities of the systems.

Law (7) Decline Quality – “Unless rigorously adapted to take into account for changes in the operational environment, the quality of a system will appear to be declining.” In order to manage the quality of the system, a quality process has to be put in place to improve both the operational environment and the system itself. In many studies directed to the evolution of computerised system (Zhan01, Mikk00 and Kout01), the aspect of quality management is very rarely considered. Yet, Lehman stresses that a decline in quality influences the viability of an adaptive system and for diverse reasons this is often the case for system that evolves. We firmly believe that the aspect of quality modelling and quality awareness has to be embedded within the system gene (at the code level if necessary). We refer to this model as the Run Time Quality Assurance (RTQA), modelled in the shape of an algorithm called the BipRyt algorithm (Mak04). The BipRyt algorithm was designed primarily to take into account the operational environment and steer the system, adjust its parameters to conform to the environment at run time, thus preserving the quality of the system as the system scales. The algorithm provides the aspect of autonomy to the collaborative participants of a system, which exploits a multi-purpose decision making mechanism for the management, control and optimisation of computational resources (e.g. node, CPU, channel, etc). The decision matrices regard all these participants as resources with diverse economical values (values which may be relative to each user).

Fundamentally the algorithm is founded on the basis that:

Rather than just using Quality modelling tools (such as the House of Quality, Analytical Hierarchy Process and Hypothesis Testing) as requirement and design activities, the BipRyt mechanism has embedded these approaches as a feature of the system (automating the methods of the tools) and made the system self-aware of its own quality attributes and react accordingly when there are SLA breaches or increased demand of resources

Law (8) Feedback System – “Evolution processes are multi-level, multi-loop, multi-agent feedback systems.” Feedback is a strong management tool to revise and improve a system. Since, software systems are never isolated, a robust feedback mechanism is required to validate the compliance of system functions over which the system developers have little or no control (Morr00). Due to the lack of control over the external components, the most economic approach is to implement feedback mechanism to obtaining statistics on how the components and the interface behaves, and then evolve in the light of that feedback. We have formulated and implemented an automated feedback mechanism (based on the concept of hypothesis Testing and Design of Experiment) within the decision process of resource management of service components of enterprise wide architectures to create an adaptive, decision making algorithm.

However, in order to design systems that evolve, one is also required to consider the aspect of autonomy within the distributed participants of the system. The study on the drive behind evolution of software architecture (Estu99) shows that systems are evolving under the pressure of a number of factors, which are: 1) distribution requires components to be more autonomous and to communicate through explicit means (not linked); 2) maintainability does not requires changing of the source code of components; 3) evolution of system and mobility require keeping components independent and autonomous and 4) Cost requires buying instead of building.

We believe that there is a 9th law missing in Lehman’s law of software evolution which is:

“For a system to evolve, it has to be distributed in nature with autonomous parts collaborating and exercising the 8 laws.”

What is a neurone on its own... what is an ant on its own? In my next blog, I will be discussing about autonomous Systems.

References:

(Bersini05) Bersini H, “Des réseaux et des sciences – Biologie, informatique, sociologie: l’omniprésence des reséaux”, Vuibert, Paris 2005

(Fung04) Hay Fung K, Low G, Bay P K, “Embracing dynamic evolution in distributed systems Software”, IEEE Vol. 21, Issue 2, pp. 49 – 55, March-April, 2004

(Oud02) Oudrhiri R, “Une approche de l’évolution des systèmes,- application aux systèmes d’information”, ed.Vuibert, 2002

(Leh80a) Lehman MM, “On Understanding Laws, Evolution and Conservation in the Large Program Life Cycle”, Journal of System and Software, vol 1, no. 3, 1980

(Zhan01) Zhang T, “Agent-based Interperability in Telecommunications Applications”, PhD Thesis, University of Berlin, 2001

(Mikk00) Mikkonen T; Lahde E; Niemi J; Siiskonen M, “Managing software evolution with the service concept”, in the Proceedings of International Symposium on Principles of Software Evolution, pp 46 – 50, 2000

(Kou01) Koutsoukos G, Gouveia J, Andrade L, Fiadeiro J L, “Managing Evolution in Telecommunications Systems”, IFIP Conference Proceedings, Vol. 198, 2001

(Mak04) Makoond B, Khaddaj S, Saltmarsh C, "Conversational Dynamics", Kingston University, February 2004

(Morr00) Morrison R, Balasubramaniam D, Greenwood R M, Kirby G N C, Mayes K, Munro D S, Warboys B C, “A Compliant Persistent Architecture, in Software Practice and Experience” vol. 30, no. 4, pp. 363 – 386, 2000

(Estu99) Estublier J, “Is a process formalism an Architecture Description Language?”, Dassault Système / LSR, International Process Technology Workshop (IPTW), Villard de Lans, France, September, 1999