Algorithms, Metrics, and Validation in Effort Estimation and Their Impact on DevOps Project Management: A Systematic Mapping Study
DOI:
https://doi.org/10.32870/recibe.v12i2.309Keywords:
Estimación del esfuerzo, gestión de proyectos, DevOps, alcance, costo, tiempoAbstract
In software development projects, effort estimation plays a critical role in determining project success or failure, as it is intrinsically linked to scope, cost, and schedule. Despite numerous efforts to improve estimation accuracy, most of the existing literature focuses on traditional software development projects, leaving a gap in the area of effort estimation for DevOps projects. This research focuses on identifying the algorithms/methods, metrics, and validation processes currently used for effort estimation. The study found a variety of techniques, including machine learning-based algorithms, software size estimation methods, checklists, agile estimation techniques, and estimation frameworks. These techniques evaluate estimation accuracy using a wide range of metrics, including Mean Magnitude of Relative Error (MMRE), Median Magnitude of Relative Error (MdMRE), Prediction at Level n (Pred(n)), Standardized Accuracy (SA), Mean Absolute Error (MAE), Squared Error, Mean Absolute Residual, Magnitude of Relative Error (MRE), Median Balanced Relative Error Bias, Relative Error, Absolute Error, Root Relative Squared Error (RRSE), Root Mean Squared Error (RMSE), Accuracy, Mean Squared Error (MSE), Coefficient of Variation, Mean, and Standard Deviation. The studies reviewed were validated through surveys, case studies, user stories, interviews, factors, and projects extracted from well-known datasets and repositories such as COCOMO81, NASA93, Maxwell, China, and ISBSG. The results of this research have the potential to contribute new approaches for improving the effort estimation process in DevOps projects, helping to fill a significant gap in the current literature on this topic.References
S. McConnell, Software Estimation: Demystifying the Black Art. Redmond, Wash, 2006.
The Standish Group, «CHAOS Report Beyond Infinity», The Standish Group, 2020. The Standish Group
K. Petersen, S. Vakkalanka, y L. Kuzniarz, «Guidelines for conducting systematic mapping studies in software
engineering: An update», Information and Software Technology, vol. 64, pp. 1-18, ago. 2015, doi:
1016/j.infsof.2015.03.007.
B. Kitchenham y S. Charters, «Guidelines for performing systematic literature reviews in software
engineering», Technical report, EBSE Technical Report EBSE-2007-01, 2007. Accedido: 26 de agosto de 2023. [En
línea]. Disponible en: https://docs.edtechhub.org/lib/EDAG684W
D. Budgen, M. Turner, P. Brereton, y B. Kitchenham, «Using Mapping Studies in Software Engineering»,
presentado en Annual Workshop of the Psychology of Programming Interest Group, 2008. Accedido: 26 de agosto de
[En línea]. Disponible en: https://www.semanticscholar.org/paper/Using-Mapping-Studies-in-Software-
Engineering-Budgen-Turner/f7644baad4c2c9bf0a7a2b1107209d2b9495aaff
S. Grimstad, M. Jørgensen, y K. Moløkken-Østvold, «Software effort estimation terminology: The tower of
Babel», Information and Software Technology, vol. 48, n.o 4, pp. 302-310, abr. 2006, doi: 10.1016/j.infsof.2005.04.004.
F. Pinciroli, «Strategies for agile software development based on technical and environmental complexity
factors», presentado en XXVIII Congreso Argentino de Ciencias de la Computación (CACIC) (La Rioja, 3 al 6 de
octubre de 2022), 2023. Accedido: 25 de agosto de 2023. [En línea]. Disponible en:
http://sedici.unlp.edu.ar/handle/10915/149423
Guía del PMBOK, «GUÍA DE LOS FUNDAMENTOS PARA LA DIRECCIÓN DE PROYECTOS (PMBOK)»,
Project Management Institute, Inc., 2017.
X. Yang, J. Liu, y D. Zhang, «A Comprehensive Taxonomy for Prediction Models in Software Engineering»,
Information, vol. 14, n.o 2, Art. n.o 2, feb. 2023, doi: 10.3390/info14020111.
B. W. Boehm, «Software Engineering Economics», IEEE Trans. Software Eng., vol. SE-10, n.o 1, pp. 4-21,
ene. 1984, doi: 10.1109/TSE.1984.5010193.
E. Mendes, Cost Estimation Techniques for Web Projects. IGI Global, 1d. C. Accedido: 26 de agosto de 2023.
[En línea]. Disponible en: https://www.igi-global.com/book/cost-estimation-techniques-web-projects/www.igiglobal.
com/book/cost-estimation-techniques-web-projects/207
M. Usman, J. Börstler, y K. Petersen, «An Effort Estimation Taxonomy for Agile Software Development»,
International Journal of Software Engineering and Knowledge Engineering, vol. 27, pp. 641-674, may 2017, doi:
1142/S0218194017500243.
A. Zakrani, A. Najm, y A. Marzak, «Support Vector Regression Based on Grid-Search Method for Agile
Software Effort Prediction», en 2018 IEEE 5th International Congress on Information Science and Technology (CiSt),
oct. 2018, pp. 1-6. doi: 10.1109/CIST.2018.8596370.
J. Angara, S. Prasad, y G. Sridevi, «Towards Benchmarking User Stories Estimation with COSMIC Function
Points-A Case Example of Participant Observation», International Journal of Electrical and Computer Engineering
(IJECE), vol. 8, n.o 5, Art. n.o 5, oct. 2018, doi: 10.11591/ijece.v8i5.pp3076-3083.
T. Hacaloğlu y O. Demirörs, Measurability of functional size in agile software projects: Multiple case studies
with COSMIC FSM. IEEE, 2019. doi: 10.1109/SEAA.2019.00041.
E. Scott y D. Pfahl, «Using developers’ features to estimate story points», en Proceedings of the 2018
International Conference on Software and System Process, Gothenburg Sweden: ACM, may 2018, pp. 106-110. doi:
1145/3202710.3203160.
T. T. Khuat y M. H. Le, «A Novel Hybrid ABC-PSO Algorithm for Effort Estimation of Software Projects Using
Agile Methodologies», Journal of Intelligent Systems, vol. 27, n.o 3, pp. 489-506, jul. 2018, doi: 10.1515/jisys-2016-
Ch. Prasada Rao, P. Siva Kumar, S. Rama Sree, y J. Devi, «An Agile Effort Estimation Based on Story Points
Using Machine Learning Techniques», en Proceedings of the Second International Conference on Computational
Intelligence and Informatics, V. Bhateja, J. M. R. S. Tavares, B. P. Rani, V. K. Prasad, y K. S. Raju, Eds., en Advances
in Intelligent Systems and Computing. Singapore: Springer, 2018, pp. 209-219. doi: 10.1007/978-981-10-8228-3_20.
T. J. Gandomani, H. Faraji, y M. Radnejad, «Planning Poker in cost estimation in Agile methods: Averaging
Vs. Consensus», en 2019 5th Conference on Knowledge Based Engineering and Innovation (KBEI), feb. 2019, pp.
-071. doi: 10.1109/KBEI.2019.8734960.
M. Choetkiertikul, H. K. Dam, T. Tran, T. Pham, A. Ghose, y T. Menzies, «A Deep Learning Model for Estimating
Story Points», IEEE Transactions on Software Engineering, vol. 45, n.o 7, pp. 637-656, jul. 2019, doi:
1109/TSE.2018.2792473.
F. Vera-Rivera, J. Barbosa-Mora, y M. Gaona, «Generación automática de la planificación de la entrega en
desarrollo de software agil, asignación de historias de usuario a los desarrolladores usando algoritmos genéticos»,
Aibi revista de investigación, administración e ingeniería, vol. 8, pp. 29-38, jun. 2020, doi: 10.15649/2346030X.735.
N. A. Bhaskaran, Dr. V. Jayaraj, y Professor, School of Computer Science Engineering & Applications,
Bharathidasan University Tiruchirappalli, India, «A Hybrid Effort Estimation Technique for Agile Software Development
(HEETAD)», IJEAT, vol. 9, n.o 1, pp. 1078-1087, oct. 2019, doi: 10.35940/ijeat.A9480.109119.
N. V. Prykhodko y S. B. Prykhodko, «A MULTIPLE NON-LINEAR REGRESSION MODEL TO ESTIMATE THE
AGILE TESTING EFFORTS FOR SMALL WEB PROJECTS», Radio Electronics, Computer Science, Control, n.o 2,
Art. n.o 2, may 2019, doi: 10.15588/1607-3274-2019-2-17.
E. Dantas, A. Costa, M. Vinicius, M. Perkusich, H. Almeida, y A. Perkusich, «An Effort Estimation Support Tool
for Agile Software Development: An Empirical Evaluation», jul. 2019, pp. 82-87. doi: 10.18293/SEKE2019-141.
B. Tanveer, A. M. Vollmer, S. Braun, y N. Ali, «An evaluation of effort estimation supported by change impact
analysis in agile software development», Journal of Software: Evolution and Process, vol. 31, abr. 2019, doi:
1002/smr.2165.
Rola y Kuchta, «Application of Fuzzy Sets to the Expert Estimation of Scrum-Based Projects», Symmetry, vol.
, p. 1032, ago. 2019, doi: 10.3390/sym11081032.
S. Bilgaiyan, S. Mishra, y M. N. Das, «Effort estimation in agile software development using experimental
validation of neural network models», International Journal of Information Technology, vol. 11, abr. 2018, doi:
1007/s41870-018-0131-2.
J. Angara, S. Prasad, y G. Sridevi, «DevOps Project Management Tools for Sprint Planning, Estimation and
Execution Maturity», Cybernetics and Information Technologies, vol. 20, n.o 2, pp. 79-92, may 2020, doi: 10.2478/cait-
-0018.
A. Sharma y N. Chaudhary, «Analysis of Software Effort Estimation Based on Story Point and Lines of Code
using Machine Learning», IJCDS, vol. 12, n.o 1, pp. 131-140, jul. 2022, doi: 10.12785/ijcds/1201012.
A. Kaushik y D. Tayal, «A Comparative Analysis on Effort Estimation for Agile and Non-agile Software Projects
Using DBN-ALO», Arabian Journal for Science and Engineering, vol. 45, nov. 2019, doi: 10.1007/s13369-019-04250-
M. Vyas y N. Hemrajani, «PREDICTING EFFORT OF AGILE SOFTWARE PROJECTS USING LINEAR
REGRESSION, RIDGE REGRESSION AND LOGISTIC REGRESSION», vol. 13, n.o 2, 2021.
C. V. Dave, «An Efficient Framework for Cost and Effort Estimation of Scrum Projects», IJRASET, vol. 9, n.o
, pp. 1478-1487, nov. 2021, doi: 10.22214/ijraset.2021.39030.
CEOLEVEL, «¿Estándares, metodologías o marcos de trabajo? ¿Sabes diferenciarlos?», CEOLEVEL, 9 de
noviembre de 2020. https://www.ceolevel.com/estandares-metodologias-o-marcos-de-trabajo-sabes-diferenciarlos
(accedido 26 de agosto de 2023).
A. Raslan, N. Darwish, y Cairo University, «An Enhanced Framework for Effort Estimation of Agile Projects»,
IJIES, vol. 11, n.o 3, pp. 205-214, jun. 2018, doi: 10.22266/ijies2018.0630.22.
D. D. R. Tripathi, «Evaluation of the Feasibility of Parametric Estimation in Devops Continuous Planning»,
International Journal for Research in Applied Science and Engineering Technology, vol. 9, n.o 9, p. 542, 2021,
Accedido: 26 de agosto de 2023. [En línea]. Disponible en:
https://www.academia.edu/52397035/Evaluation_of_the_Feasibility_of_Parametric_Estimation_in_Devops_Continuo
us_Planning
F. Alshammari, «Cost estimate in scrum project with the decision-based effort estimation technique», Soft
Computing, vol. 26, pp. 1-13, jul. 2022, doi: 10.1007/s00500-022-07352-w.
S. A. Butt et al., «A software-based cost estimation technique in scrum using a developer’s expertise»,
Advances in Engineering Software, vol. 171, p. 103159, sep. 2022, doi: 10.1016/j.advengsoft.2022.103159.
S. Abusaeed, S. U. R. Khan, y A. Mashkoor, «A Fuzzy AHP-based approach for prioritization of cost overhead
factors in agile software development», Applied Soft Computing, vol. 133, p. 109977, ene. 2023, doi:
1016/j.asoc.2022.109977.
A. Vetrò, R. Dürre, M. Conoscenti, D. M. Fernández, y M. Jørgensen, «Combining Data Analytics with Team
Feedback to Improve the Estimation Process in Agile Software Development», Foundations of Computing and
Decision Sciences, vol. 43, n.o 4, pp. 305-334, nov. 2018, doi: 10.1515/fcds-2018-0016.
M. Usman, K. Petersen, J. Börstler, y P. Santos Neto, «Developing and using checklists to improve software
effort estimation: A multi-case study», Journal of Systems and Software, vol. 146, pp. 286-309, dic. 2018, doi:
1016/j.jss.2018.09.054.
M. Adnan, M. Afzal, y K. Asif, «Ontology-Oriented Software Effort Estimation System for E-commerce
Applications Based on Extreme Programming and Scrum Methodologies», The Computer Journal, vol. 62, pp. 1605-
, nov. 2019, doi: 10.1093/comjnl/bxy141.
R. K. Mallidi y M. Sharma, «Study on Agile Story Point Estimation Techniques and Challenges», International
Journal of Computer Applications, vol. 174, n.o 13, pp. 9-14, ene. 2021, Accedido: 26 de agosto de 2023. [En línea].
Disponible en: https://www.ijcaonline.org/archives/volume174/number13/31736-2021921014
H. Premalatha y C. Srikrishna, «Effort Estimation in Agile Software Development using Evolutionary
CostSensitive Deep Belief Network», International Journal of Intelligent Engineering and Systems, vol. 12, pp. 261-
, abr. 2019, doi: 10.22266/ijies2019.0430.25.
L.-D. Radu, «Effort Prediction in Agile Software Development with Bayesian Networks»:, en Proceedings of
the 14th International Conference on Software Technologies, Prague, Czech Republic: SCITEPRESS - Science and
Technology Publications, 2019, pp. 238-245. doi: 10.5220/0007842802380245.
D. Meedeniya y H. Thennakoon, «Impact Factors and Best Practices to Improve Effort Estimation Strategies
and Practices in DevOps», ago. 2021. doi: 10.1145/3484399.3484401
A. Sharma and N. Chaudhary, "Linear Regression Model for Agile Software Development Effort Estimation," 2020
th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE), Jaipur, India, 2020,
pp. 1-4, doi: 10.1109/ICRAIE51050.2020.9358309.
B. Valenzuela, I. Alvarado, R. Santaolaya, H. Reyes. “Identification of methods, approaches, and factors in effort
estimation for DevOps projects: a systematic literature mapping”. 2023 Mexican International Conference on Computer
Science (ENC). Guanajuato, Guanajuato, México.
Downloads
Published
Versions
- 2026-06-21 (2)
- 2024-02-27 (1)
