![Improving the Quality of Requirements in
Middleware Requirements Specifications
Vidyasagar Uddagiri
Auto and Industrials Unit
TCS Limited
Hyderabad, India
vidyasagar.uddagiri@tcs.com
Lingachary Eswarachary
Auto and Industrials Unit
TCS Limited
Charlotte,NC USA
lingachary.eswarachary@tcs.com
Manigandan Jagadeesan
Auto and Industrials Unit
TCS Limited
Chennai, India
manigandan.j@tcs.com
Vishal Kharat
Auto and Industrials Unit
TCS Limited
Mumbai, India
vishal.kharat@tcs.com
Abstract—The importance of middleware in the success of
IT and enterprise systems cannot be over emphasized. Digital
Transformation is increasing this importance exponentially. De-
spite such importance, the topic of requirements engineering in
middleware has limited attention in the literature. The authors
have observed in their experience that the challenges faced during
the requirements elicitation for middleware projects result in
poor Quality of Requirements (QoR). It is thus important to
understand the characteristics, nature, structure and process of
middleware requirements as well as these challenges better to
be able to improve the QoR. Based on the issues faced due
to unavailability of adequate data and people’s time in many
middleware projects in over twenty five years, the authors and
their organization have developed and leveraged successfully a
structured methodology to elicit and document the requirements
for middleware with high quality by working around the limita-
tions faced in such projects.
Index Terms—Middleware, Quality of Requirements, Require-
ments Specifications, Interface Requirements Definition, Digital
Transformation
I. INTRODUCTION
Middleware though referenced since 1968, started getting
greater recognition beginning 1980s as a way of linking newer
applications to older legacy systems. Its importance increased
further with the advent of distributed systems and continues
to grow manifold due to Digital transformation.
According to an industry analyst [1], middleware related
work accounts for 50 percent of the time and cost of building
a digital platform. In the 75 years of cumulative experience
of the authors, there was never a single middleware project
without challenges in the requirements elicitation process.
Based on the authors’ experience, the impact of poor Quality
of Requirements (QoR) for middleware specifications result in
low project success attributes such as:
• About 50 percent of the defects and 80 percent of rework
in middleware projects being attributable to poor QoR
• Unsatisfied customers, increased elapsed time and higher
costs are the secondary effects of the poor QoR
The primary focus of middleware literature was on topics
like prescription or selection of middleware technologies or
solution patterns for specific problems with little focus about
requirements engineering [2,3]. This paper aims to fill this gap
in literature, about middleware requirements engineering, with
some practical insights towards improvement of QoR based on
the learning of authors from their experience on many projects.
II. UNDERSTANDING THE CONTEXT AND CHALLENGES IN
MIDDLEWARE REQUIREMENTS ELICITATION
In the absence of understanding about the nature and content
of middleware requirements from literature, it is essential
to get this understanding. The unit of work in any mid-
dleware project involves building or modifying an interface
between two systems or applications. The interface could be a
component of a Middleware Platform like Enterprise Service
Bus (ESB) or Integration Platform as a Service (IPaaS) or
a composite application integrating two or more indepen-
dent application building blocks. An interface functions as
a component to exchange data or information. The different
scenarios in which interfaces need to be built or modified are:
• Build/Modify an interface
• Build/Migrate/Upgrade a middle platform
• Retrofit an interface to make it compatible with infras-
tructure changes of the hosting middleware platform
The requirements specifications to build such interface is
referred as an Interface Requirements Document (IRD). It
consists of some functional and a lot of non-functional re-
quirements. Since the majority of the requirements are non-
functional in nature, there can be a presumption that these
can be extracted by method like mining the applications and
transaction logs and applying reverse engineering principles,
as in some application modernization projects where reverse
engineering tools are applied to extract the business rules.
However, the characteristics of the system elements and the
technical environment in which middleware operates, poses
limitations for such automation. Such characteristics and lim-
itations are:
• An interface may connect through several applications
or components. The information discovery breaks at the
application or component boundaries, requiring human
intervention to synthesize the different pieces of data
• Applications may not maintain transaction data
412
2020 IEEE 28th International Requirements Engineering Conference (RE)
2332-6441/20/$31.00 ©2020 IEEE
DOI 10.1109/RE48521.2020.00060](https://image.slidesharecdn.com/repaper212-publishedcopy-200908091346/85/Improving-the-Quality-of-Requirements-in-Middleware-Requirements-Specifications-1-320.jpg)
![Fig. 1. Interface Requirement Document - Structure and Sample Content
• ESB in traditional implementations were stateless ma-
chines and never store any transaction data or payload.
Logs may be available with limited information
• Based on the characteristics of connectors or adapters
used in an interface, transaction data may not clearly
identify source or target application
• In some middleware platforms, the data is in proprietary
format and sometimes the platform may not expose data
to enable reverse engineering
Apart from system and technical environment challenges
cited above, there are people challenges observed in every
project. While some are common to any software project, few
challenges are unique to middleware projects. Interface being
like a bridge between applications, is not considered as an
independent entity for ownership impacting the middleware
projects in the following ways:
• No specific resource person for information elicitation,
unlike an application or a business process
• Middleware work is not included in the work plans for
the organizational units that support projects on various
applications and business processes. So, getting cross
functional commitment from the resource persons is
always a challenge
• Middleware technical team is the best knowledge re-
source for requirements. However, this team in all organi-
zations is a small shared services team, already burdened
with huge backlog of work and conflicts with priorities
of different projects, so getting their attention and time
too is a challenge
III. NATURE AND CONTENT OF MIDDLEWARE
REQUIREMENTS SPECIFICATION
The IRD consists of functional and non-functional require-
ments. Figure 1 provides a schematic overview of the content
of the elements of the interface requirement document.
The different requirements specification attributes for an
IRD should cover the following dimensions:
• General information
• Functional requirements
• Business capabilities and Process dependencies
Fig. 2. Different interventions for requirements elicitation
• Business impact
• Volume, Scalability and Performance
• Special design considerations
• Connectivity and Scheduling
• Data persistence
• Privacy and Security
• Testing and Error handling
• Legal and Regulatory requirements
IV. BEST PRACTICES FROM INDUSTRY EXPERIENCE
Different kinds of interventions were used for requirements
elicitation. Figure 2 lists the different interventions and the
suitability for different project types. Other artifacts developed
to help improve the quality of information in an IRD include
• Technical guidelines for interfaces
• Checklist to verify that necessary attributes are docu-
mented in an IRD
• Multi-dimensional Viewpoints model for checking the
completeness of interfaces identified in the project scope
V. FUTURE WORK
Based on the lessons learnt from the past projects, work is
in progress to determine if middleware requirements can be
integrated into DevOps tool chain in a Digital Transformation
program that involves IPaaS implementation.
Using machine learning, it is possible to enhance the
automation of requirements elicitation, to minimize the impact
caused by limited availability of human stake holders and
the current methods of requirement elicitation. This is an
immediate research focus for the authors.
REFERENCES
[1] R.van der Muelen,“Use a hybrid integration approach to empower
digital transformation”. Gartner. Updated April 26, 2018. [Online].
Available at https://www.gartner.com/smarterwithgartner/use-a-hybrid-
integration-approach-to-empower-digital-transformation, Accessed on:
May 11, 2020.
[2] V. Issarny, M. Caporuscio and N. Georgantas, ”A Perspective on the
Future of Middleware-based Software Engineering,” Future of Software
Engineering (FOSE ’07), Minneapolis, MN, 2007, pp. 244-258, doi:
10.1109/FOSE.2007.2.
[3] W. Emmerich, ”Software engineering and middleware: a roadmap,”
in the Proceedings of the Conference on The Future of Software
Engineering (ICSE ’00), ACM New York, NY, 2000, pp. 117-129, doi:
https://DOI.org/10.1145/336512.336542.
413](https://image.slidesharecdn.com/repaper212-publishedcopy-200908091346/85/Improving-the-Quality-of-Requirements-in-Middleware-Requirements-Specifications-2-320.jpg)
Improving the Quality of Requirements in Middleware Requirements Specifications
- 1. Improving the Quality of Requirements in Middleware Requirements Specifications Vidyasagar Uddagiri Auto and Industrials Unit TCS Limited Hyderabad, India vidyasagar.uddagiri@tcs.com Lingachary Eswarachary Auto and Industrials Unit TCS Limited Charlotte,NC USA lingachary.eswarachary@tcs.com Manigandan Jagadeesan Auto and Industrials Unit TCS Limited Chennai, India manigandan.j@tcs.com Vishal Kharat Auto and Industrials Unit TCS Limited Mumbai, India vishal.kharat@tcs.com Abstract—The importance of middleware in the success of IT and enterprise systems cannot be over emphasized. Digital Transformation is increasing this importance exponentially. De- spite such importance, the topic of requirements engineering in middleware has limited attention in the literature. The authors have observed in their experience that the challenges faced during the requirements elicitation for middleware projects result in poor Quality of Requirements (QoR). It is thus important to understand the characteristics, nature, structure and process of middleware requirements as well as these challenges better to be able to improve the QoR. Based on the issues faced due to unavailability of adequate data and people’s time in many middleware projects in over twenty five years, the authors and their organization have developed and leveraged successfully a structured methodology to elicit and document the requirements for middleware with high quality by working around the limita- tions faced in such projects. Index Terms—Middleware, Quality of Requirements, Require- ments Specifications, Interface Requirements Definition, Digital Transformation I. INTRODUCTION Middleware though referenced since 1968, started getting greater recognition beginning 1980s as a way of linking newer applications to older legacy systems. Its importance increased further with the advent of distributed systems and continues to grow manifold due to Digital transformation. According to an industry analyst [1], middleware related work accounts for 50 percent of the time and cost of building a digital platform. In the 75 years of cumulative experience of the authors, there was never a single middleware project without challenges in the requirements elicitation process. Based on the authors’ experience, the impact of poor Quality of Requirements (QoR) for middleware specifications result in low project success attributes such as: • About 50 percent of the defects and 80 percent of rework in middleware projects being attributable to poor QoR • Unsatisfied customers, increased elapsed time and higher costs are the secondary effects of the poor QoR The primary focus of middleware literature was on topics like prescription or selection of middleware technologies or solution patterns for specific problems with little focus about requirements engineering [2,3]. This paper aims to fill this gap in literature, about middleware requirements engineering, with some practical insights towards improvement of QoR based on the learning of authors from their experience on many projects. II. UNDERSTANDING THE CONTEXT AND CHALLENGES IN MIDDLEWARE REQUIREMENTS ELICITATION In the absence of understanding about the nature and content of middleware requirements from literature, it is essential to get this understanding. The unit of work in any mid- dleware project involves building or modifying an interface between two systems or applications. The interface could be a component of a Middleware Platform like Enterprise Service Bus (ESB) or Integration Platform as a Service (IPaaS) or a composite application integrating two or more indepen- dent application building blocks. An interface functions as a component to exchange data or information. The different scenarios in which interfaces need to be built or modified are: • Build/Modify an interface • Build/Migrate/Upgrade a middle platform • Retrofit an interface to make it compatible with infras- tructure changes of the hosting middleware platform The requirements specifications to build such interface is referred as an Interface Requirements Document (IRD). It consists of some functional and a lot of non-functional re- quirements. Since the majority of the requirements are non- functional in nature, there can be a presumption that these can be extracted by method like mining the applications and transaction logs and applying reverse engineering principles, as in some application modernization projects where reverse engineering tools are applied to extract the business rules. However, the characteristics of the system elements and the technical environment in which middleware operates, poses limitations for such automation. Such characteristics and lim- itations are: • An interface may connect through several applications or components. The information discovery breaks at the application or component boundaries, requiring human intervention to synthesize the different pieces of data • Applications may not maintain transaction data 412 2020 IEEE 28th International Requirements Engineering Conference (RE) 2332-6441/20/$31.00 ©2020 IEEE DOI 10.1109/RE48521.2020.00060
- 2. Fig. 1. Interface Requirement Document - Structure and Sample Content • ESB in traditional implementations were stateless ma- chines and never store any transaction data or payload. Logs may be available with limited information • Based on the characteristics of connectors or adapters used in an interface, transaction data may not clearly identify source or target application • In some middleware platforms, the data is in proprietary format and sometimes the platform may not expose data to enable reverse engineering Apart from system and technical environment challenges cited above, there are people challenges observed in every project. While some are common to any software project, few challenges are unique to middleware projects. Interface being like a bridge between applications, is not considered as an independent entity for ownership impacting the middleware projects in the following ways: • No specific resource person for information elicitation, unlike an application or a business process • Middleware work is not included in the work plans for the organizational units that support projects on various applications and business processes. So, getting cross functional commitment from the resource persons is always a challenge • Middleware technical team is the best knowledge re- source for requirements. However, this team in all organi- zations is a small shared services team, already burdened with huge backlog of work and conflicts with priorities of different projects, so getting their attention and time too is a challenge III. NATURE AND CONTENT OF MIDDLEWARE REQUIREMENTS SPECIFICATION The IRD consists of functional and non-functional require- ments. Figure 1 provides a schematic overview of the content of the elements of the interface requirement document. The different requirements specification attributes for an IRD should cover the following dimensions: • General information • Functional requirements • Business capabilities and Process dependencies Fig. 2. Different interventions for requirements elicitation • Business impact • Volume, Scalability and Performance • Special design considerations • Connectivity and Scheduling • Data persistence • Privacy and Security • Testing and Error handling • Legal and Regulatory requirements IV. BEST PRACTICES FROM INDUSTRY EXPERIENCE Different kinds of interventions were used for requirements elicitation. Figure 2 lists the different interventions and the suitability for different project types. Other artifacts developed to help improve the quality of information in an IRD include • Technical guidelines for interfaces • Checklist to verify that necessary attributes are docu- mented in an IRD • Multi-dimensional Viewpoints model for checking the completeness of interfaces identified in the project scope V. FUTURE WORK Based on the lessons learnt from the past projects, work is in progress to determine if middleware requirements can be integrated into DevOps tool chain in a Digital Transformation program that involves IPaaS implementation. Using machine learning, it is possible to enhance the automation of requirements elicitation, to minimize the impact caused by limited availability of human stake holders and the current methods of requirement elicitation. This is an immediate research focus for the authors. REFERENCES [1] R.van der Muelen,“Use a hybrid integration approach to empower digital transformation”. Gartner. Updated April 26, 2018. [Online]. Available at https://www.gartner.com/smarterwithgartner/use-a-hybrid- integration-approach-to-empower-digital-transformation, Accessed on: May 11, 2020. [2] V. Issarny, M. Caporuscio and N. Georgantas, ”A Perspective on the Future of Middleware-based Software Engineering,” Future of Software Engineering (FOSE ’07), Minneapolis, MN, 2007, pp. 244-258, doi: 10.1109/FOSE.2007.2. [3] W. Emmerich, ”Software engineering and middleware: a roadmap,” in the Proceedings of the Conference on The Future of Software Engineering (ICSE ’00), ACM New York, NY, 2000, pp. 117-129, doi: https://DOI.org/10.1145/336512.336542. 413
- 3. VI. ANNEXURE Additional information to support the content in the main paper is provided for a deeper understanding of the audience. A sample IRD will be used to explain the concepts discussed in this paper along with explaining more about the utilities and tools used as best practices in various projects that have been described in the various sections of this Annexure. A. Principles for middleware interface requirements Before embarking on the requirements elicitation, it is essential to finalize the design principles for middleware interfaces. These principles can help in improving QoR elicited for IRD attributes Some common principles that are derived from authors’ experience in middleware projects are: • Develop reusable application programming interfaces (API) as preferred way of data integration, wherever possible • Only secure protocols to be used, avoids insecure trans- mission • Mandatory encryption for data elements that are subject to regulatory compliance like SOX, GDPR • No archival / persistence of business data at the middle- ware layer • Source / Target system teams should advise the middle- ware project team for any specific fault handling at the middleware layer apart from common exception handling within the platform • Source / Target system teams must perform data valida- tion as per agreed layout and stop processing in case of data validation failures • Source / Target system teams must have estimates of data volumes expected to be processed by an interface • No business data to be sent in email messages triggered by failures in interface processing • No changes should be accommodated in Middleware layer to overcome the performance issues in Source / Target systems A disciplined alignment with these design principles can help in optimizing the scope in being able to rationalize the number of interfaces necessary for the project. B. Attributes and Dimensions to be documented in an IRD An IRD is the full source of interface requirements infor- mation for a middleware developer. Figure 3 provides an overview of the various attributes that are necessary to be captured under each dimension of requirements specification as part of an IRD. All the dimensions and attributes depicted in Figure 3 along with the source system to target system mapping will help in preparing an IRD with high QoR. Figure 4 depicts a handy checklist for the requirements analyst and interface designer to ensure completeness of the requirements specifications in an IRD. Fig. 3. Requirement Dimensions and Attributes Fig. 4. Checklist for validating completeness of IRD C. Multi Dimensional ViewPoints Validation Model In many projects, interfaces are identified based on the business requirements view specified by the customer from their active memory. These requirements pertain to frequently and commonly used processes and miss out capturing special and infrequently used business processes that are dormant in memory. Moreover since the interfaces are not owned by any specific application or business process owner, such misses are more likely in middleware projects than in other software projects. To prevent such lapses, a multi dimensional viewpoints validation model is developed that can help prompt any such misses. This model is based on standard industry business process models and best practices brought in by system integrators like the authors’ company from their experience gained from other customers in the same business domain. Figure 5 provides an overview of this model. D. Requirements Capturing and Governance Tool In projects with large number of interfaces, it is a greater challenge to manually collect the attributes required for the IRD. An online tool was developed where information elicited from the user helps to derive the inference and prompts gaps in completion immediately and ensuring documenting missing attribute values quickly. Figure 6 provides a process view of the different functions built into the tool used for requirements capturing. 414
- 4. Fig. 5. Four Dimension ViewPoints Validation Model Fig. 6. Requirements Capturing and Governance Tool Process Flow Figure 7 and Figure 8 are screenshots from the tool used for requirements capturing. They show the perspective of source and target system requirement specification attributes respectively that are keyed in or chosen from the various drop down lists defined in the user interface of the tool. Fig. 7. Screenshot of User Interface to capture Source System attributes Fig. 8. Screenshot of User Interface to capture Target System attributes The complete information is consolidated and generate an IRD automatically. This tool also has workflows mechanism to manage approvals of the requirements as well as monitoring the progress of interface life cycle throughout the project, thus providing control to the stakeholders to intervene at the right point in time and improve responsiveness and address delays due to information unavailability or people availability in a better manner. E. Machine Learning based automation - future research focus area To help the requirements analyst, a chat-bot shall be de- signed. It will be capable of asking the questions based on the profile of a person driven by cognitive instruction coming from the learning model. Entire information will be captured categorically into the database. This database will be mined, explored by the rules and models built in the engine to derive the inferences for attribute values recommendations. Multiple versions of the IRD can be produced but the best document with highest accuracy will be generated as recommendation to undergo review and approval by customer. The Recommendation Engine model will be based on similar four dimensions like the ViewPoints validation model depicted in Figure 5. However since the data is analyzed by machine intelligence, the processing and cognitive ability is enhanced and will improve the ability to address the gaps much beyond capabilities of the current interventions and methods. 415