Gestión de Materiales Experimentales y su Uso en Replicaciones de Experimentos: Un Mapeo Sistemático de la Literatura - Replication Management Tools and Materials for Use in Replication in Experimental Software Engineering: A Systematic Mapping Study
DOI:
https://doi.org/10.32870/recibe.v5i3.62Palabras clave:
Ingeniería del Software Experimental, Replicación de Experimentos, Material Experimental, Gestión del Material ExperimentalResumen
La Ingeniería de Software Experimental (ISE) aplica la experimentación para obtener conocimiento sobre productos, métodos, técnicas usados en el proceso de desarrollo software. Para validar el conocimiento es necesario repetir el experimento original tantas veces como sea necesario. Se requieren mucha información y materiales acerca del experimento para que un experimentador realice una replicación. Previa a la ejecución de la replicación del experimento, algunos o la totalidad de los materiales pueden requerir cambios produciendo nuevas o modificadas versiones de estos. Una vez ejecutada la replicación del experimento se espera que, parte o la totalidad de los materiales, se incorporen al material del experimento original. El incremento en el número de replicaciones del experimento original está directamente relacionado con el incremento de las versiones del material experimental, que ocasiona comúnmente confusión y desorden en su organización. En este trabajo, realizamos un mapeo sistemático de la literatura (MSL) para localizar trabajos sobre Gestión de Materiales Experimentales y su Uso en Replicaciones en Ingeniería de Software Experimental. Los resultados encontrados reflejan la escasa existencia de artículos sobre esta temática. Además, el análisis de los estudios encontrados ha descubierto que dichos artículos, en su mayoría, adolecen de problemas en la gestión de versiones de materiales experimentales para poder llevar a cabo un proceso de replicación. Estos datos proporcionan información de interés para iniciar la investigación de la adopción del paradigma de gestión de configuración de software en la gestión del material experimental en ISE.Abstract: Experimental Software Engineering (ESE) applies experimentation to get knowledge about products, methods and techniques used in the software development process. The original experiment should be repeated as many times as necessary to validate that knowledge. Lots of information and materials about experiment is required for an experimenter to perform a replication. Before the execution of the experimental replication, all or part of the materials may require changes producing new or modified versions of these. After experimental replication is executed, it is expected that all or part of the materials are incorporated in the material of the original experiment. The increase of the number of replications of the original experiment is directly related to the increase of the versions of the experimental material, generating commonly confusion and disorder in the organization. In this work, we conduct a mapping study to locate articles about Replication Management Tools and Materials for use in experiment replication in ESE. The results show the lack of articles in this subject. Furthermore, most of them suffers from problems in the version management of the experimental materials to carry out a replication process. These data provide interesting information to start a research of the adoption of the software configuration management paradigm in the experimental material management.Keywords: Experimental Software Engineering; Experiment Replication; Experimental Material; Experimental Material Management.Citas
Albayrak, Ã, & Carver, J. (2012). Investigation of individual factors impacting the effectiveness of requirements inspections: A replicated experiment., 1-26. doi:10.1007/s10664-012-9221-0
Basili, V., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sorumgard, S., & Zelkowitz, M. (1996). Packaging researcher experience to assist replication of experiments. Proc. of the ISERN Meeting 1996,
Basili, V. R., Shull, F., & Lanubile, F. (1999). Building knowledge through families of experiments.25(4), 456-473.
Beecham, S., Baddoo, N., Hall, T., Robinson, H., & Sharp, H. (2006). Protocol for a systematic literature review of motivation in software engineering. University of Hertfordshire,
Boehm, B., & Basili, V. (April 2001). The CEBASE framework for strategic software development and evolution.
da Silva, F. Q. B., França, A. C. C., Suassuna, M., de Sousa Mariz, L. M. R., Rossiley, I., de Miranda, R. C. G., . . . Espindola, E. (2013). Team building criteria in software projects: A mix-method replicated study. Information and Software Technology, 55(7), 1316-1340. doi:http://dx.doi.org/10.1016/j.infsof.2012.11.006
Daly, J., Brooks, A., Miller, J., Roper, M., & Wood, M. (1994). Verification of results in software maintenance through external replication. Software Maintenance, 1994. Proceedings., International Conference On, 50-57. doi:10.1109/ICSM.1994.336790
Dieste, O., & Padua, O. A. G. (2007). Developing search strategies for detecting relevant experiments for systematic reviews. Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium On, 215-224. doi:10.1109/ESEM.2007.19
Dieste, O., Fernandez, E., Martinez, R. G., & Juristo, N. (2011). Comparative analysis of meta-analysis methods: When to use which? Evaluation & Assessment in Software Engineering (EASE 2011), 15th Annual Conference On, 36-45. doi:10.1049/ic.2011.0005
Do, H., Elbaum, S., & Rothermel, G. (2005). Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering, 10(4), 405-435.
Dunsmore, A., Roper, M., & Wood, M. (2002). Further investigations into the development and evaluation of reading techniques for object-oriented code inspection. Software Engineering, 2002. ICSE 2002. Proceedings of the 24rd International Conference On, 47-57.
Espinosa, E. (2014). Gestión de configuración y línea de productos para mejorar el proceso experimental en ingeniería del software.
Gallardo, E. G. E. (2012). Using configuration management and product line software paradigms to support the experimentation process in software engineering. Research Challenges in Information Science (RCIS), 2012 Sixth International Conference On, 1-6. doi:10.1109/RCIS.2012.6240454
He, L., & Carver, J. (2006). PBR vs. checklist: A replication in the n-fold inspection context. Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, 95-104.
Hofmann, B., & Wulf, V. (2002). Building communities among software engineers: The VISEK approach to intra-and inter-organizational learning., 25-33.
Juristo, N., & Vegas, S. (2011). Design patterns in software maintenance: An experiment replication at UPM - experiences with the RESER'11 joint replication project. Replication in Empirical Software Engineering Research (RESER), 2011 Second International Workshop On, 7-14. doi:10.1109/RESER.2011.8
Kamsties, E., & Lott, C. M. (1995). An empirical evaluation of three defect-detection techniques. Proceedings of the Fifth European Software Engineering Conference,
Lott, C. M.Comparing reading and testing techniques. Retrieved from http://web.archive.org/web/20071026204353/http://www.chris-lott.org/work/exp/
Maldonado, J. C., Carver, J., Shull, F., Fabbri, S., Dória, E., Martimiano, L., . . . Basili, V. (2006). Perspective-based reading: A replicated experiment focused on individual reviewer effectiveness. Empirical Software Engineering, 11(1), 119-142.
Mendonça, M. G., Maldonado, J. C., de Oliveira, Maria Cristina Ferreira, Carver, J., Fabbri, C., Shull, F., . . . Basili, V. R. (2008). A framework for software engineering experimental replications. Engineering of Complex Computer Systems, 2008. ICECCS 2008. 13th IEEE International Conference On, 203-212.
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008). Systematic mapping studies in software engineering. 12th International Conference on Evaluation and Assessment in Software Engineering, , 17 1.
Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide Wiley-Blackwell.
Sabaliauskaite, G., Matsukawa, F., Kusumoto, S., & Inoue, K. (2003). Further investigations of reading techniques for object-oriented design inspection. Information and Software Technology, 45(9), 571-585.
Schmidt, S. (2009). Shall we really do it again? the powerful concept of replication is neglected in the social sciences.13(2), 90-100.
Shull, F. J., Carver, J. C., Vegas, S., & Juristo, N. (2008). The role of replications in empirical software engineering. Empirical Software Engineering, 13(2), 211-218.
Shull, F., Basili, V., Carver, J., Maldonado, J. C., Travassos, G. H., Mendoncca, M., & Fabbri, S. (2002). Replicating software engineering experiments: Addressing the tacit knowledge problem. ISESE '02: Proceedings of the 2002 International Symposium on Empirical Software Engineering, 7.
Shull, F., Lanubile, F., & Basili, V. R. (2000). Investigating reading techniques for object-oriented framework learning. Software Engineering, IEEE Transactions On, 26(11), 1101-1118. doi:10.1109/32.881720
Shull, F., Mendoncca, M. G., Basili, V., Carver, J., Maldonado, J. C., Fabbri, S., . . . Ferreira, M. C. (2004). Knowledge-sharing issues in experimental software engineering.9, 111-137(27).
Shull, F., Mendonça, M. G., Basili, V., Carver, J., Maldonado, J. C., Fabbri, S., . . . Ferreira, M. C. (2004). Knowledge-sharing issues in experimental software engineering. Empirical Software Engineering, 9(1-2), 111-137.
Sjøberg, D. I., Hannay, J. E., Hansen, O., Kampenes, V. B., Karahasanovic, A., Liborg, N., & Rekdal, A. C. (2005). A survey of controlled experiments in software engineering. Software Engineering, IEEE Transactions On, 31(9), 733-753.
Vanschoren, J., & Blockeel, H. (2009). Stand on the shoulders of giants. towards a portal for collaborative experimentation in data mining. Proceedings of the SoKD-09 International Workshop on Third Generation Data Mining at ECML PKDD 2009, , 1 88-99.
Vegas, S., Juristo, N., Moreno, A., Solari, M., & Letelier, P. (2006). Analysis of the influence of communication between researchers on experiment replication. Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, 28-37.
Zendler, A. (2001). A preliminary software engineering theory as investigated by published experiments. Empirical Software Engineering, 6(2), 161-180.
