Evaluating the Graph-based Visualization Technique: A Controlled Experiment
DOI:
https://doi.org/10.29019/enfoqueute.v8n1.134Keywords:
software visualization, controlled experiment, software comprehensionAbstract
Many researchers have highlighted the scarcity of empirical studies that systematically examine the advantages and disadvantages of the use of visualization techniques for software understanding activities. Such studies are crucial for gathering and analyzing objective and quantifiable evidence about the usefulness of proposed visualization techniques and tools, and ultimately, for guiding the research in software visualization. This paper presents a controlled experiment aimed at assessing the impact of a graph-based visualization technique on comprehension tasks. Six common comprehension tasks were performed by 20 undergraduate software engineering students. The completion time and the accuracy of the participants’ responses were measured. The results indicate that on one hand the use of the graph-based visualization increases the correctness (by 21.45% in average) but on the other hand it does not reduce the completion time in program comprehension tasks.
Downloads
Download data is not yet available.
References
Aracic, I., & Mezini, M. (2006). Flexible abstraction techniques for graph-based visualizations. In Eclipse Technology eXchange workshop (eTX) at ECOOP.
Caserta, P., & Zendra, O. (2011). Visualization of the static aspects of software: A survey. IEEE transactions on visualization and computer graphics, 17(7), 913-933.
Cornelissen, B., Zaidman, A., & Van Deursen, A. (2011). A controlled experiment for program comprehension through trace visualization.
Da Silva, F. Q., Suassuna, M., França, A. C. C., Grubb, A. M., Gouveia, T. B., Monteiro, C. V., & dos Santos, I. E. (2014). Replication of empirical studies in software engineering research: a systematic mapping study. Empirical Software Engineering, 19(3), 501-557.
Dąbrowski, R., Stencel, K., & Timoszuk, G. (2011). Software is a directed multigraph. In Software Architecture (pp. 360-369). Springer Berlin Heidelberg.
de Magalhães, C. V., da Silva, F. Q., Santos, R. E., & Suassuna, M. (2015). Investigations about replication of empirical studies in software engineering: A systematic mapping study. Information and Software Technology, 64, 76-101.
Easterbrook, S., Singer, J., Storey, M. A., & Damian, D. (2008). Selecting empirical methods for software engineering research. In Guide to advanced empirical software engineering (pp. 285-311). Springer London.
Fittkau, F., Finke, S., Hasselbring, W., & Waller, J. (2015, May). Comparing trace visualizations for program comprehension through controlled experiments. In Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension (pp. 266-276). IEEE Press.
Haitzer, T., & Zdun, U. (2013, July). Controlled experiment on the supportive effect of architectural component diagrams for design understanding of novice architects. In European Conference on Software Architecture (pp. 54-71). Springer Berlin Heidelberg.
Knodel, J., Muthig, D., & Naab, M. (2008). An experiment on the role of graphical elements in architecture visualization. Empirical Software Engineering, 13(6), 693-726.
Lungu, M., Lanza, M., & Gîrba, T. (2006, March). Package patterns for visual architecture recovery. In Conference on Software Maintenance and Reengineering (CSMR'06) (pp. 10-pp). IEEE.
Pacione, M. J., Roper, M., & Wood, M. (2004, November). A novel software visualisation model to support software comprehension. In Reverse Engineering, 2004. Proceedings. 11th Working Conference on (pp. 70-79). IEEE.
Quante, J. (2008, June). Do Dynamic Object Process Graphs Support Program Understanding? -A Controlled Experiment. In Program Comprehension, 2008. ICPC 2008. The 16th IEEE International Conference on (pp. 73-82). IEEE.
Sensalire, M., Ogao, P., & Telea, A. (2009, September). Evaluation of software visualization tools: Lessons learned. In Visualizing Software for Understanding and Analysis, 2009. VISSOFT 2009. 5th IEEE International Workshop on (pp. 19-26). IEEE.
Seriai, A., Benomar, O., Cerat, B., & Sahraoui, H. (2014, September). Validation of Software Visualization Tools: A Systematic Mapping Study. In Software Visualization (VISSOFT), 2014 Second IEEE Working Conference on (pp. 60-69). IEEE.
Shahin, M., Liang, P., & Babar, M. A. (2014). A systematic review of software architecture visualization techniques. Journal of Systems and Software, 94, 161-185.
Wettel, R., Lanza, M., & Robbes, R. (2010). Empirical validation of CodeCity: A controlled experiment. Tech Report 2010/05, University of Lugano.
Wettel, R., Lanza, M., & Robbes, R. (2011, May). Software systems as cities: a controlled experiment. In Proceedings of the 33rd International Conference on Software Engineering (pp. 551-560). ACM.J.
Caserta, P., & Zendra, O. (2011). Visualization of the static aspects of software: A survey. IEEE transactions on visualization and computer graphics, 17(7), 913-933.
Cornelissen, B., Zaidman, A., & Van Deursen, A. (2011). A controlled experiment for program comprehension through trace visualization.
Da Silva, F. Q., Suassuna, M., França, A. C. C., Grubb, A. M., Gouveia, T. B., Monteiro, C. V., & dos Santos, I. E. (2014). Replication of empirical studies in software engineering research: a systematic mapping study. Empirical Software Engineering, 19(3), 501-557.
Dąbrowski, R., Stencel, K., & Timoszuk, G. (2011). Software is a directed multigraph. In Software Architecture (pp. 360-369). Springer Berlin Heidelberg.
de Magalhães, C. V., da Silva, F. Q., Santos, R. E., & Suassuna, M. (2015). Investigations about replication of empirical studies in software engineering: A systematic mapping study. Information and Software Technology, 64, 76-101.
Easterbrook, S., Singer, J., Storey, M. A., & Damian, D. (2008). Selecting empirical methods for software engineering research. In Guide to advanced empirical software engineering (pp. 285-311). Springer London.
Fittkau, F., Finke, S., Hasselbring, W., & Waller, J. (2015, May). Comparing trace visualizations for program comprehension through controlled experiments. In Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension (pp. 266-276). IEEE Press.
Haitzer, T., & Zdun, U. (2013, July). Controlled experiment on the supportive effect of architectural component diagrams for design understanding of novice architects. In European Conference on Software Architecture (pp. 54-71). Springer Berlin Heidelberg.
Knodel, J., Muthig, D., & Naab, M. (2008). An experiment on the role of graphical elements in architecture visualization. Empirical Software Engineering, 13(6), 693-726.
Lungu, M., Lanza, M., & Gîrba, T. (2006, March). Package patterns for visual architecture recovery. In Conference on Software Maintenance and Reengineering (CSMR'06) (pp. 10-pp). IEEE.
Pacione, M. J., Roper, M., & Wood, M. (2004, November). A novel software visualisation model to support software comprehension. In Reverse Engineering, 2004. Proceedings. 11th Working Conference on (pp. 70-79). IEEE.
Quante, J. (2008, June). Do Dynamic Object Process Graphs Support Program Understanding? -A Controlled Experiment. In Program Comprehension, 2008. ICPC 2008. The 16th IEEE International Conference on (pp. 73-82). IEEE.
Sensalire, M., Ogao, P., & Telea, A. (2009, September). Evaluation of software visualization tools: Lessons learned. In Visualizing Software for Understanding and Analysis, 2009. VISSOFT 2009. 5th IEEE International Workshop on (pp. 19-26). IEEE.
Seriai, A., Benomar, O., Cerat, B., & Sahraoui, H. (2014, September). Validation of Software Visualization Tools: A Systematic Mapping Study. In Software Visualization (VISSOFT), 2014 Second IEEE Working Conference on (pp. 60-69). IEEE.
Shahin, M., Liang, P., & Babar, M. A. (2014). A systematic review of software architecture visualization techniques. Journal of Systems and Software, 94, 161-185.
Wettel, R., Lanza, M., & Robbes, R. (2010). Empirical validation of CodeCity: A controlled experiment. Tech Report 2010/05, University of Lugano.
Wettel, R., Lanza, M., & Robbes, R. (2011, May). Software systems as cities: a controlled experiment. In Proceedings of the 33rd International Conference on Software Engineering (pp. 551-560). ACM.J.
Published
2017-02-24
Issue
Section
Miscellaneous
License
The authors retain all copyrights ©.
- The authors retain their trademark and patent rights, as well as rights to any process or procedure described in the article.
- The authors retain the right to share, copy, distribute, perform, and publicly communicate the article published in Enfoque UTE (for example, post it in an institutional repository or publish it in a book), provided that acknowledgment of its initial publication in Enfoque UTE is given.
- The authors retain the right to publish their work at a later date, to use the article or any part of it (for example, a compilation of their work, lecture notes, a thesis, or for a book), provided that they indicate the source of publication (authors of the work, journal, volume, issue, and date).
How to Cite
Evaluating the Graph-based Visualization Technique: A Controlled Experiment. (2017). Enfoque UTE, 8(1), pp. 201-216. https://doi.org/10.29019/enfoqueute.v8n1.134
