Successful deployment of Intelligent Transport Systems (ITS) applications requires information on their impacts, costs and benefits, technical functioning and other characteristics. However, the resources, time and data available for evaluation of ITS systems are usually limited. Evaluation methods used for ITS systems should also be able to accommodate changes that may occur during the evaluation study. A need for more agile evaluation methods has therefore been identified.
The study provides definitions for different agile characteristics of evaluation methods for ITS systems, measurable criteria for these characteristics, and a definition for an agile evaluation method. The agile characteristics found to be relevant for evaluation methods for ITS include flexibility, leanness, lightness and simplicity, responsiveness, learning, speed and sustainability.
Photo: Gudellaphoto/Mostphotos.com
The study results indicated that agile methods for evaluating the impacts of ITS have many similar characteristics to expert assessment, such as limited requirements for time and resources. However, they can be expected to be more transparent and possibly also more accurate than expert assessment without supporting documentation. Agile methods for evaluating the technical functioning of ITS systems, based on observation of system behaviour, analysis of reliability and statistical analysis of study results, can be used to verify the operation of an ITS system or subsystem in realistic operating conditions. They can be applied also in situations in which no extensive and up-to-date documentation on system requirements is produced or maintained. These include for example novel systems that are still under development and cases in which the agile development model is used.
The agile characteristics were defined on the basis of a literature study taking into account the Agile Manifesto, literature on measuring the agility of software development methods, and the experience obtained from seven case studies published as peer-reviewed articles. The descriptions of agile characteristics, criteria for measuring the agility of evaluation methods and the definition of an agile evaluation method were validated by applying them to seven case studies.
The study concluded that not all simple or simplistic evaluation methods are agile methods, as simplicity does not imply that the evaluation method has other essential characteristics of agility such as flexibility and leanness.
More validation will likely be needed for descriptions of the agile characteristics of evaluation methods, for the criteria for agility, and for the definition of an agile evaluation method. Further analyses should also be carried out on how to employ agile development, testing and systems engineering methods in the development of ITS systems in an optimum way, and how agile evaluation methods could be connected to them.
Further information:
Risto Öörni (2019): Agile Evaluation Methods for Intelligent Transport Systems – Definitions of Agile Characteristics and Assessment Criteria
Contact:
Risto Öörni
risto.oorni@vtt.fi
VTT, Technical Research Centre of Finland Ltd, Finland
Follow us: