[154] Kintis M., Malevris N. Using data flow patterns for equivalent mutant detection. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; 2014:196–205. doi:10.1109/ICSTW.2014.21.

[155] Kintis M., Malevris N. MEDIC: A static analysis framework for equivalent mutant identification. Inf. Softw. Technol. 2015;68:1–17. doi:10.1016/j.infsof.2015.07.009.

[156] Holling D., Banescu S., Probst M., Petrovska A., Pretschner A. Nequivack: assessing mutation score confidence. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:152–161. doi:10.1109/ICSTW.2016.29.

[157] Kintis M., Malevris N. Identifying more equivalent mutants via code similarity. In: 20th Asia-Pacific Software Engineering Conference, APSEC 2013, Ratchathewi, Bangkok, Thailand, December 2–5, 2013, vol. 1; 2013:180–188. doi:10.1109/APSEC.2013.34.

[158] Foster K.A. Error sensitive test cases analysis (ESTCA). IEEE Trans. Softw. Eng. 1980;6(3):258–264. doi:10.1109/TSE.1980.234487.

[159] Howden W.E. Weak mutation testing and completeness of test sets. IEEE Trans. Softw. Eng. 1982;8(4):371–379. doi:10.1109/TSE.1982.235571.

[160] Tai K. Predicate-based test generation for computer programs. In: Proceedings of the 15th International Conference on Software Engineering, Baltimore, MD, USA, May 17–21, 1993; 1993:267–276. http://portal.acm.org/citation.cfm?id=257572.257631.

[161] Tai K. Theory of fault-based predicate testing for computer programs. IEEE Trans. Softw. Eng. 1996;22(8):552–562. doi:10.1109/32.536956.

[162] M. Papadakis, Error detection methods in Java programs using the mutation method, (Masters thesis), Athens University of Economics and Business 2005.

[163] Kaminski G., Ammann P., Offutt J. Better predicate testing. In: Proceedings of the 6th International Workshop on Automation of Software Test, AST 2011, Waikiki, Honolulu, HI, USA, May 23–24, 2011; 2011:57–63. doi:10.1145/1982595.1982608.

[164] Just R., Kapfhammer G.M., Schweiggert F. Do redundant mutants affect the effectiveness and efficiency of mutation analysis? In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:720–725. doi:10.1109/ICST.2012.162.

[165] Just R., Schweiggert F. Higher accuracy and lower run time: efficient mutation analysis using non-redundant mutation operators. Softw. Test. Verif. Reliab. 2015;25(5–7):490–507. doi:10.1002/stvr.1561.

[166] Fernandes L., Ribeiro M., Carvalho L., Gheyi R., Mongiovi M., Santos A., Cavalcanti A., Ferrari F., Maldonado J.C. Avoiding useless mutants. In: Proceedings of the 16th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences; New York, NY, USA: ACM; 2017:978-1-4503-5524-7187–198. doi:10.1145/3136040.3136053.

[167] Laurent T., Ventresque A., Papadakis M., Henard C., Traon Y.L. Assessing and improving the mutation testing practice of PIT. CoRR. 2016. http://arxiv.org/abs/1601.02351.

[168] Lindström B., Marki A. On strong mutation and subsuming mutants. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:112–121. doi:10.1109/ICSTW.2016.28.

[169] Kurtz B., Ammann P., Offutt J. Static analysis of mutant subsumption. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICSTW.2015.7107454.

[170] DeMillo R.A., Offutt A.J. Constraint-based automatic test data generation. IEEE Trans. Softw. Eng. 1991;17(9):900–910. doi:10.1109/32.92910.

[171] Fraser G., Zeller A. Mutation-driven generation of unit tests and oracles. IEEE Trans. Softw. Eng. 2012;38(2):278–292. doi:10.1109/TSE.2011.93.

[172] Harman M., Jia Y., Langdon W.B. Strong higher order mutation-based test data generation. In: SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13th European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5–9, 2011; 2011:212–222. doi:10.1145/2025113.2025144.

[173] Anand S., Burke E.K., Chen T.Y., Clark J.A., Cohen M.B., Grieskamp W., Harman M., Harrold M.J., McMinn P. An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 2013;86(8):1978–2001. doi:10.1016/j.jss.2013.02.061.

[174] Wotawa F., Nica M., Aichernig B.K. Generating distinguishing tests using the Minion constraint solver. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010, Workshops Proceedings; 2010:325–330. doi:10.1109/ICSTW.2010.11.

[175] Nica S. On the improvement of the mutation score using distinguishing test cases. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21–25, 2011; 2011:423–426. doi:10.1109/ICST.2011.40.

[176] Papadakis M., Malevris N. An effective path selection strategy for mutation testing. In: 16th Asia-Pacific Software Engineering Conference, APSEC 2009, December 1–3, 2009, Batu Ferringhi, Penang, Malaysia; 2009:422–429. doi:10.1109/APSEC.2009.68.

[177] Papadakis M., Malevris N. Automatically performing weak mutation with the aid of symbolic execution, concolic testing and search-based testing. Softw. Q. J. 2011;19(4):691–723. doi:10.1007/s11219-011-9142-y.

[178] Anand S., Pasareanu C.S., Visser W. JPF-SE: a symbolic execution extension to Java PathFinder. In: Tools and Algorithms for the Construction and Analysis of Systems, 13th International Conference, TACAS 2007, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2007 Braga, Portugal, March 24–April 1, 2007; 2007:134–138. doi:10.1007/978-3-540-71209-1_12.

[179] Cadar C., Dunbar D., Engler D.R. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: 8th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2008, December 8–10, 2008, San Diego, CA, USA, Proceedings; 2008:209–224. http://www.usenix.org/events/osdi08/tech/full_papers/cadar/cadar.pdf.

[180] Riener H., Bloem R., Fey G. Test case generation from mutants using model checking techniques. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Berlin, Germany, March 21–25, 2011, Workshop Proceedings; 2011:388–397. doi:10.1109/ICSTW.2011.55.

[181] Papadakis M., Malevris N., Kallia M. Towards automating the generation of mutation tests. In: The 5th Workshop on Automation of Software Test, AST 2010, May 3–4, 2010, Cape Town, South Africa; 2010:111–118. doi:10.1145/1808266.1808283.

[182] Zhang L., Xie T., Zhang L., Tillmann N., de Halleux J., Mei H. Test generation via dynamic symbolic execution for mutation testing. In: 26th IEEE International Conference on Software Maintenance (ICSM 2010), September 12–18, 2010, Timisoara, Romania; 2010:1–10. doi:10.1109/ICSM.2010.5609672.

[183] Bardin S., Kosmatov N., Cheynier F. Efficient leveraging of symbolic execution to advanced coverage criteria. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014, March 31 2014–April 4, 2014, Cleveland, OH, USA; 2014:173–182. doi:10.1109/ICST.2014.30.

[184] Jamrozik K., Fraser G., Tillmann N., de Halleux J. Augmented dynamic symbolic execution. In: IEEE/ACM International Conference on Automated Software Engineering, ASE’12, Essen, Germany, September 3–7, 2012; 2012:254–257. doi:10.1145/2351676.2351716.

[185] Souza F.C.M., Papadakis M., Traon Y.L., Delamaro M.E. Strong mutation-based test data generation using hill climbing. In: Proceedings of the 9th International Workshop on Search-Based Software Testing, SBST@ICSE 2016, Austin, TX, USA, May 14–22, 2016; 2016:45–54. doi:10.1145/2897010.2897012.

[186] Ayari K., Bouktif S., Antoniol G. Automatic mutation test input data generation via ant colony. In: Genetic and Evolutionary Computation Conference, GECCO 2007, Proceedings, London, England, UK, July 7–11, 2007; 2007:1074–1081. doi:10.1145/1276958.1277172.

[187] Fraser G., Zeller A. Mutation-driven generation of unit tests and oracles. In: Proceedings of the Nineteenth International Symposium on Software Testing and Analysis, ISSTA 2010, Trento, Italy, July 12–16, 2010; 2010:147–158. doi:10.1145/1831708.1831728.

[188] Papadakis M., Malevris N. Automatic mutation based test data generation. In: 13th Annual Genetic and Evolutionary Computation Conference, GECCO 2011, Companion Material Proceedings, Dublin, Ireland, July 12–16, 2011; 2011:247–248. doi:10.1145/2001858.2001997.

[189] Papadakis M., Malevris N. Killing mutants effectively a search based approach. In: Knowledge-Based Software Engineering—Proceedings of the Tenth Conference on Knowledge-Based Software Engineering, JCKBSE 2012, Rodos, Greece, August 23–26, 2012; 2012:217–226. doi:10.3233/978-1-61499-094-9-217.

[190] Papadakis M., Malevris N. Searching and generating test inputs for mutation testing. SpringerPlus. 2193-18012013;2(1):121. doi:10.1186/2193-1801-2-121.

[191] Patrick M., Alexander R., Oriol M., Clark J.A. Using mutation analysis to evolve subdomains for random testing. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18–22, 2013; 2013:53–62. doi:10.1109/ICSTW.2013.14.

[192] Fraser G., Arcuri A. Achieving scalable mutation-based generation of whole test suites. Empir. Softw. Eng. 2015;20(3):783–812. doi:10.1007/s10664-013-9299-z.

[193] Shi A., Gyori A., Gligoric M., Zaytsev A., Marinov D. Balancing trade-offs in test-suite reduction. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16–22, 2014; 2014:246–256. doi:10.1145/2635868.2635921.

[194] Lou Y., Hao D., Zhang L. Mutation-based test-case prioritization in software evolution. In: 26th IEEE International Symposium on Software Reliability Engineering, ISSRE 2015, Gaithersbury, MD, USA, November 2–5, 2015; 2015:46–57. doi:10.1109/ISSRE.2015.7381798.

[195] Zhang L., Marinov D., Khurshid S. Faster mutation testing inspired by test prioritization and reduction. In: International Symposium on Software Testing and Analysis, ISSTA ’13, Lugano, Switzerland, July 15–20, 2013; 2013:235–245. doi:10.1145/2483760.2483782.

[196] Just R., Kapfhammer G.M., Schweiggert F. Using non-redundant mutation operators and test suite prioritization to achieve efficient and scalable mutation analysis. In: 23rd IEEE International Symposium on Software Reliability Engineering, ISSRE 2012, Dallas, TX, USA, November 27–30, 2012; 2012:11–20. doi:10.1109/ISSRE.2012.31.

[197] Zhu Q., Panichella A., Zaidman A. Speeding-up mutation testing via data compression and state infection. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW); 2017:103–109. doi:10.1109/ICSTW.2017.25.

[198] M.E. Delamaro, Proteum - a mutation analysis based testing environmen, (Masters thesis), University of São Paulo, Sao Paulo, Brazil, 1993.

[199] Kim S., Ma Y., Kwon Y.R. Combining weak and strong mutation for a noninterpretive Java mutation system. Softw. Test. Verif. Reliab. 2013;23(8):647–668. doi:10.1002/stvr.1480.

[200] Just R., Ernst M.D., Fraser G. Efficient mutation analysis by propagating and partitioning infected execution states. In: International Symposium on Software Testing and Analysis, ISSTA ’14, San Jose, CA, USA, July 21–26, 2014; 2014:315–326. doi:10.1145/2610384.2610388.

[201] Mateo P.R., Usaola M.P. Mutant execution cost reduction: through MUSIC (mutant schema improved with extra code). In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:664–672. doi:10.1109/ICST.2012.156.

[202] Mateo P.R., Usaola M.P. Reducing mutation costs through uncovered mutants. Softw. Test. Verif. Reliab. 2015;25(5–7):464–489. doi:10.1002/stvr.1534.

[203] King K.N., Offutt A.J. A Fortran language system for mutation-based software testing. Softw. Pract. Exper. 1991;21(7):685–718. doi:10.1002/spe.4380210704.

[204] Papadakis M., Delamaro M.E., Traon Y.L. Proteum/FL: a tool for localizing faults using mutation analysis. In: 13th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2013, Eindhoven, Netherlands, September 22–23, 2013; 2013:94–99. doi:10.1109/SCAM.2013.6648189.

[205] Schuler D., Zeller A. Javalanche: efficient mutation testing for Java. In: Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2009, Amsterdam, The Netherlands, August 24–28, 2009; 2009:297–298. doi:10.1145/1595696.1595750.

[206] Mateo P.R., Usaola M.P. Parallel mutation testing. Softw. Test. Verif. Reliab. 2013;23(4):315–350. doi:10.1002/stvr.1471.

[207] Gligoric M., Jagannath V., Luo Q., Marinov D. Efficient mutation testing of multithreaded code. Softw. Test. Verif. Reliab. 2013;23(5):375–403. doi:10.1002/stvr.1469.

[208] Gligoric M., Jagannath V., Marinov D. MuTMuT: efficient exploration for mutation testing of multithreaded code. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010; 2010:55–64. doi:10.1109/ICST.2010.33.

[209] Gong P., Zhao R., Li Z. Faster mutation-based fault localization with a novel mutation execution strategy. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICSTW.2015.7107448.

[210] Zhang L., Marinov D., Zhang L., Khurshid S. Regression mutation testing. In: International Symposium on Software Testing and Analysis, ISSTA 2012, Minneapolis, MN, USA, July 15–20, 2012; 2012:331–341. doi:10.1145/2338965.2336793.

[211] Wright C.J., Kapfhammer G.M., McMinn P. Efficient mutation analysis of relational database structure using mutant schemata and parallelisation. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18–22, 2013; 2013:63–72. doi:10.1109/ICSTW.2013.15.

[212] Zhou C., Frankl P.G. Inferential checking for mutants modifying satabase states. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21–25, 2011; 2011:259–268. doi:10.1109/ICST.2011.63.

[213] Patrick M., Craig A.P., Cunniffe N.J., Parry M., Gilligan C.A. Testing stochastic software using pseudo-oracles. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18–20, 2016; 2016:235–246. doi:10.1145/2931037.2931063.

[214] Rutherford M.J., Carzaniga A., Wolf A.L. Evaluating test suites and adequacy criteria using simulation-based models of distributed systems. IEEE Trans. Softw. Eng. 2008;34(4):452–470. doi:10.1109/TSE.2008.33.

[215] Grün B.J.M., Schuler D., Zeller A. The impact of equivalent mutants. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009, Workshops Proceedings; 2009:192–199. doi:10.1109/ICSTW.2009.37.

[216] Schuler D., Dallmeier V., Zeller A. Efficient mutation testing by checking invariant violations. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, ISSTA 2009, Chicago, IL, USA, July 19–23, 2009; 2009:69–80. doi:10.1145/1572272.1572282.

[217] Schuler D., Zeller A. (Un-)covering equivalent mutants. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010; 2010:45–54. doi:10.1109/ICST.2010.30.

[218] Schuler D., Zeller A. Covering and uncovering equivalent mutants. Softw. Test. Verif. Reliab. 2013;23(5):353–374. doi:10.1002/stvr.1473.

[219] Schwarz B., Schuler D., Zeller A. Breeding high-impact mutations. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Berlin, Germany, March 21–25, 2011, Workshop Proceedings; 2011:382–387. doi:10.1109/ICSTW.2011.56.

[220] Papadakis M., Traon Y.L. Mutation testing strategies using mutant classification. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing, SAC ’13, Coimbra, Portugal, March 18–22, 2013; 2013:1223–1229. doi:10.1145/2480362.2480592.

[221] Papadakis M., Delamaro M.E., Traon Y.L. Mitigating the effects of equivalent mutants with mutant classification strategies. Sci. Comput. Program. 2014;95:298–319. doi:10.1016/j.scico.2014.05.012.

[222] Kintis M., Papadakis M., Malevris N. Isolating first order equivalent mutants via second order mutation. In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:701–710. doi:10.1109/ICST.2012.160.

[223] Kintis M., Papadakis M., Malevris N. Employing second-order mutation for isolating first-order equivalent mutants. Softw. Test. Verif. Reliab. 2015;25(5–7):508–535. doi:10.1002/stvr.1529.

[224] Usaola M.P., Mateo P.R., Lamancha B.P. Reduction of test suites using mutation. In: Fundamental Approaches to Software Engineering–15th International Conference, FASE 2012, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2012, Tallinn, Estonia, March 24–April 1, 2012; 2012:425–438. doi:10.1007/978-3-642-28872-2_29.

[225] Hao D., Zhang L., Wu X., Mei H., Rothermel G. On-demand test suite reduction. In: 34th International Conference on Software Engineering, ICSE 2012, June 2–9, 2012, Zurich, Switzerland; 2012:738–748. doi:10.1109/ICSE.2012.6227144.

[226] Alipour M.A., Shi A., Gopinath R., Marinov D., Groce A. Evaluating non-adequate test-case reduction. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, Singapore, September 3–7, 2016; 2016:16–26. doi:10.1145/2970276.2970361.

[227] Nguyen D.C., Marchetto A., Tonella P. Change sensitivity based prioritization for audit testing of webservice compositions. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Berlin, Germany, March 21–25, 2011, Workshop Proceedings; 2011:357–365. doi:10.1109/ICSTW.2011.50.

[228] Tengeri D., Vidács L., Beszédes Á., Jász J., Balogh G., Vancsics B., Gyimóthy T. Relating code coverage, mutation score and test suite reducibility to defect density. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:174–179. doi:10.1109/ICSTW.2016.25.

[229] Kurtz B., Ammann P., Offutt J., Kurtz M. Are we there yet? How redundant and equivalent mutants affect determination of test completeness. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:142–151. doi:10.1109/ICSTW.2016.41.

[230] Fraser G., Zeller A. Generating parameterized unit tests. In: Proceedings of the 20th International Symposium on Software Testing and Analysis, ISSTA 2011, Toronto, ON, Canada, July 17–21, 2011; 2011:364–374. doi:10.1145/2001420.2001464.

[231] Knauth T., Fetzer C., Felber P. Assertion-driven development: assessing the quality of contracts using meta-mutations. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009, Workshops Proceedings; 2009:182–191. doi:10.1109/ICSTW.2009.40.

[232] Fraser G., Arcuri A. EvoSuite: automatic test suite generation for object-oriented software. In: SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13th European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5–9, 2011; 2011:416–419. doi:10.1145/2025113.2025179.

[233] Yoshida H., Tokumoto S., Prasad M.R., Ghosh I., Uehara T. FSX: fine-grained incremental unit test generation for C/C++ programs. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18–20, 2016; 2016:106–117. doi:10.1145/2931037.2931055.

[234] Yoshida H., Tokumoto S., Prasad M.R., Ghosh I., Uehara T. FSX: a tool for fine-grained incremental unit test generation for C/C++ programs. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, November 13–18, 2016; 2016:1052–1056. doi:10.1145/2950290.2983937.

[235] Mirshokraie S., Mesbah A., Pattabiraman K. PYTHIA: generating test cases with oracles for JavaScript applications. In: 2013 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013, Silicon Valley, CA, USA, November 11–15, 2013; 2013:610–615. doi:10.1109/ASE.2013.6693121.

[236] Mirshokraie S., Mesbah A., Pattabiraman K. JSEFT: automated javascript unit test generation. In: 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICST.2015.7102595.

[237] Staats M., Gay G., Heimdahl M.P.E. Automated oracle creation support, or: how I learned to stop worrying about fault propagation and love mutation testing. In: 34th International Conference on Software Engineering, ICSE 2012, June 2–9, 2012, Zurich, Switzerland; 2012:870–880. doi:10.1109/ICSE.2012.6227132.

[238] Gay G., Staats M., Whalen M.W., Heimdahl M.P.E. Automated oracle data selection support. IEEE Trans. Softw. Eng. 2015;41(11):1119–1137. doi:10.1109/TSE.2015.2436920.

[239] Jahangirova G., Clark D., Harman M., Tonella P. Test oracle assessment and improvement. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18–20, 2016; 2016:247–258. doi:10.1145/2931037.2931062.

[240] Barr E.T., Harman M., McMinn P., Shahbaz M., Yoo S. The oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 2015;41(5):507–525. doi:10.1109/TSE.2014.2372785.

[241] Papadakis M., Traon Y.L. Using mutants to locate “unknown” faults. In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:691–700. doi:10.1109/ICST.2012.159.

[242] Chekam T.T., Papadakis M., Traon Y.L. Assessing and comparing mutation-based fault localization techniques. CoRR. 2016. http://arxiv.org/abs/1607.05512.

[243] Moon S., Kim Y., Kim M., Yoo S. Ask the mutants: mutating faulty programs for fault localization. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014, March 31 2014–April 4, 2014, Cleveland, OH, USA; 2014:153–162. doi:10.1109/ICST.2014.28.

[244] Zhang L., Zhang L., Khurshid S. Injecting mechanical faults to localize developer faults for evolving software. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, Part of SPLASH 2013, Indianapolis, IN, USA, October 26–31, 2013; 2013:765–784. doi:10.1145/2509136.2509551.

[245] Hong S., Lee B., Kwak T., Jeon Y., Ko B., Kim Y., Kim M. Mutation-based fault localization for real-world multilingual programs (T). In: 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9–13, 2015; 2015:464–475. doi:10.1109/ASE.2015.14.

[246] Hong S., Kwak T., Lee B., Jeon Y., Ko B., Kim Y., Kim M. MUSEUM: debugging real-world multilingual programs using mutation analysis. Inf. Softw. Technol. 2017;82:80–95. doi:10.1016/j.infsof.2016.10.002.

[247] Murtaza S.S., Madhavji N.H., Gittens M., Li Z. Diagnosing new faults using mutants and prior faults. In: Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, Waikiki, Honolulu, HI, USA, May 21–28, 2011; 2011:960–963. doi:10.1145/1985793.1985959.

[248] Murtaza S.S., Hamou-Lhadj A., Madhavji N.H., Gittens M. An empirical study on the use of mutant traces for diagnosis of faults in deployed systems. J. Syst. Softw. 2014;90:29–44. doi:10.1016/j.jss.2013.11.1094.

[249] Musco V., Monperrus M., Preux P. Mutation-based graph inference for fault localization. In: 16th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2016, Raleigh, NC, USA, October 2–3, 2016; 2016:97–106. doi:10.1109/SCAM.2016.24.

[250] Debroy V., Wong W.E. Using mutation to automatically suggest fixes for faulty programs. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010; 2010:65–74. doi:10.1109/ICST.2010.66.

[251] Debroy V., Wong W.E. Combining mutation and fault localization for automated program debugging. J. Syst. Softw. 2014;90:45–60. doi:10.1016/j.jss.2013.10.042.

[252] Weimer W., Nguyen T., Le Goues C., Forrest S. Automatically finding patches using genetic programming. In: 31st International Conference on Software Engineering, ICSE 2009, May 16–24, 2009, Vancouver, Canada, Proceedings; 2009:364–374. doi:10.1109/ICSE.2009.5070536.

[253] Le Goues C., Dewey-Vogt M., Forrest S., Weimer W. A systematic study of automated program repair: fixing 55 out of 105 bugs for $8 each. In: 34th International Conference on Software Engineering, ICSE 2012, June 2–9, 2012, Zurich, Switzerland; 2012:3–13. doi:10.1109/ICSE.2012.6227211.

[254] Weimer W., Fry Z.P., Forrest S. Leveraging program equivalence for adaptive program repair: models and first results. In: 2013 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013, Silicon Valley, CA, USA, November 11–15, 2013; 2013:356–366. doi:10.1109/ASE.2013.6693094.

[255] Tan S.H., Roychoudhury A. relifix: automated repair of software regressions. In: 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16–24, 2015, vol. 1; 2015:471–482. doi:10.1109/ICSE.2015.65.

[256] Long F., Rinard M. Staged program repair with condition synthesis. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Bergamo, Italy, August 30–September 4, 2015; 2015:166–178. doi:10.1145/2786805.2786811.

[257] Kim D., Nam J., Song J., Kim S. Automatic patch generation learned from human-written patches. In: 35th International Conference on Software Engineering, ICSE ’13, San Francisco, CA, USA, May 18–26, 2013; 2013:802–811. doi:10.1109/ICSE.2013.6606626.

[258] Petke J., Haraldsson S., Harman M., Langdon W., White D., Woodward J. Genetic improvement of software: a comprehensive survey. IEEE Trans. Evol. Comput. 1089-778X2017;PP(99):1. doi:10.1109/TEVC.2017.2693219.

[259] Chandra S., Torlak E., Barman S., Bodík R. Angelic debugging. In: Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, Waikiki, Honolulu, HI, USA, May 21–28, 2011; 2011:121–130. doi:10.1145/1985793.1985811.

[260] Zhang J., Wang Z., Zhang L., Hao D., Zang L., Cheng S., Zhang L. Predictive mutation testing. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18–20, 2016; 2016:342–353. doi:10.1145/2931037.2931038.

[261] Rojas J.M., Fraser G. Code defenders: a mutation testing game. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:162–167. doi:10.1109/ICSTW.2016.43.

[262] Rojas J.M., White T.D., Clegg B.S., Fraser G. Code defenders: crowdsourcing effective tests and subtle mutants with a mutation testing game. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, Buenos Aires, Argentina, May 20–28, 2017; 2017:677–688.

[263] Harman M., Jia Y., Langdon W.B. A manifesto for higher order mutation testing. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010, Workshops Proceedings; 2010:80–89. doi:10.1109/ICSTW.2010.13.

[264] Jia Y., Harman M. Constructing subtle faults using higher order mutation testing. In: Eighth IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2008), September 28–29, 2008, Beijing, China; 2008:249–258. doi:10.1109/SCAM.2008.36.

[265] Harman M., Jia Y., Mateo P.R., Polo M. Angels and monsters: an empirical investigation of potential test effectiveness and efficiency improvement from strongly subsuming higher order mutation. In: ACM/IEEE International Conference on Automated Software Engineering, ASE ’14, Vasteras, Sweden, September 15–19, 2014; 2014:397–408. doi:10.1145/2642937.2643008.

[266] Omar E., Ghosh S., Whitley D. Constructing subtle higher order mutants for Java and AspectJ programs. In: IEEE 24th International Symposium on Software Reliability Engineering, ISSRE 2013, Pasadena, CA, USA, November 4–7, 2013; 2013:340–349. doi:10.1109/ISSRE.2013.6698887.

[267] Omar E., Ghosh S., Whitley D. Subtle higher order mutants. Inf. Softw. Technol. 2017;81:3–18. doi:10.1016/j.infsof.2016.01.016.

[268] Omar E., Ghosh S., Whitley D. Comparing search techniques for finding subtle higher order mutants. In: Genetic and Evolutionary Computation Conference, GECCO ’14, Vancouver, BC, Canada, July 12–16, 2014; 2014:1271–1278. doi:10.1145/2576768.2598286.

[269] Wu F., Harman M., Jia Y., Krinke J. HOMI: searching higher order mutants for software improvement. In: Search Based Software Engineering—8th International Symposium, SSBSE 2016, Raleigh, NC, USA, October 8–10, 2016, Proceedings; 2016:18–33. doi:10.1007/978-3-319-47106-8_2.

[270] Shin D., Yoo S., Bae D. Diversity-aware mutation adequacy criterion for improving fault detection capability. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:122–131. doi:10.1109/ICSTW.2016.37.

[271] Shin D., Yoo S., Bae D.H. A theoretical and empirical study of diversity-aware mutation adequacy criterion. IEEE Trans. Softw. Eng. 0098-55892017;PP(99):1. doi:10.1109/TSE.2017.2732347 1.

[272] Henard C., Papadakis M., Perrouin G., Klein J., Traon Y.L. Assessing software product line testing via model-based mutation: an application to similarity testing. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18–22, 2013; 2013:188–197. doi:10.1109/ICSTW.2013.30.

[273] Henard C., Papadakis M., Traon Y.L. MutaLog: a tool for mutating logic formulas. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; 2014:399–404. doi:10.1109/ICSTW.2014.54.

[274] Henard C., Papadakis M., Perrouin G., Klein J., Traon Y.L. Towards automated testing and fixing of re-engineered feature models. In: 35th International Conference on Software Engineering, ICSE ’13, San Francisco, CA, USA, May 18–26, 2013; 2013:1245–1248. doi:10.1109/ICSE.2013.6606689.

[275] Henard C., Papadakis M., Traon Y.L. Mutation-based generation of software product line test configurations. In: Search-Based Software Engineering—6th International Symposium, SSBSE 2014, Fortaleza, Brazil, August 26–29, 2014; 2014:92–106. doi:10.1007/978-3-319-09940-8_7.

[276] Filho R.A.M., Vergilio S.R. A mutation and multi-objective test data generation approach for feature testing of software product lines. In: 29th Brazilian Symposium on Software Engineering, SBES 2015, Belo Horizonte, MG, Brazil, September 21–26, 2015; 2015:21–30. doi:10.1109/SBES.2015.17.

[277] Filho R.A.M., Vergilio S.R. A multi-objective test data generation approach for mutation testing of feature models. J. Softw. Eng. R&D. 2016;4:4. doi:10.1186/s40411-016-0030-9.

[278] Arcaini P., Gargantini A., Vavassori P. Generating tests for detecting faults in feature models. In: 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICST.2015.7102591.

[279] Arcaini P., Gargantini A., Vavassori P. Automatic detection and removal of conformance faults in feature models. In: 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016, Chicago, IL, USA, April 11–15, 2016; 2016:102–112. doi:10.1109/ICST.2016.10.

[280] Trakhtenbrot M.B. Implementation-oriented mutation testing of statechart models. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010, Workshops Proceedings; 2010:120–125. doi:10.1109/ICSTW.2010.55.

[281] Aichernig B.K., Jöbstl E. Towards symbolic model-based mutation testing: pitfalls in expressing semantics as constraints. In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:752–757. doi:10.1109/ICST.2012.169.

[282] Aichernig B.K., Brandl H., Jöbstl E., Krenn W. Efficient mutation killers in action. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21–25, 2011; 2011:120–129. doi:10.1109/ICST.2011.57.

[283] Aichernig B.K., Brandl H., Jöbstl E., Krenn W., Schlick R., Tiran S. Killing strategies for model-based mutation testing. Softw. Test. Verif. Reliab. 2015;25(8):716–748. doi:10.1002/stvr.1522.

[284] Belli F., Beyazit M. Exploiting model morphology for event-based testing. IEEE Trans. Softw. Eng. 2015;41(2):113–134. doi:10.1109/TSE.2014.2360690.

[285] El-Fakih K., Kolomeez A., Prokopenko S., Yevtushenko N. Extended finite state machine based test derivation driven by user defined faults. In: First International Conference on Software Testing, Verification, and Validation, ICST 2008, Lillehammer, Norway, April 9–11, 2008; 2008:308–317. doi:10.1109/ICST.2008.16.

[286] Su T., Meng G., Chen Y., Wu K., Yang W., Yao Y., Pu G., Liu Y., Su Z. Guided, stochastic model-based GUI testing of Android apps. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering; New York, NY, USA: ACM; 2017:978-1-4503-5105-8245–256. doi:10.1145/3106237.3106298.

[287] Aichernig B.K., Marcovic S., Schumi R. Property-based testing with external test-case generators. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW); 2017:337–346. doi:10.1109/ICSTW.2017.62.

[288] Aichernig B.K., Lorber F. Towards generation of adaptive test cases from partial models of determinized timed automata. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–6. doi:10.1109/ICSTW.2015.7107409.

[289] Aichernig B.K., Lorber F., Nickovic D. Time for mutants—model-based mutation testing with timed automata. In: Tests and Proofs—7th International Conference, TAP 2013, Budapest, Hungary, June 16–20, 2013; 2013:20–38. doi:10.1007/978-3-642-38916-0_2.

[290] Larsen K.G., Lorber F., Nielsen B., Nyman U.M. Mutation-based test-case generation with ecdar. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW); 2017:319–328. doi:10.1109/ICSTW.2017.60.

[291] Zhou T., Sun H., Liu J., Chen X., Du D. Improving testing coverage for safety-critical system by mutated specification. In: 21st Asia-Pacific Software Engineering Conference, APSEC 2014, Jeju, South Korea, December 1–4, 2014, vol, 1: Research Papers; 2014:43–46. doi:10.1109/APSEC.2014.15.

[292] Adra S.F., McMinn P. Mutation operators for agent-based models. In: Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7–9, 2010, Workshops Proceedings; 2010:151–156. doi:10.1109/ICSTW.2010.9.

[293] Stephan M., Alalfi M.H., Stevenson A., Cordy J.R. Using mutation analysis for a model-clone detector comparison framework. In: 35th International Conference on Software Engineering, ICSE ’13, San Francisco, CA, USA, May 18–26, 2013; 2013:1261–1264. doi:10.1109/ICSE.2013.6606693.

[294] Roy C.K., Cordy J.R. A mutation/injection-based automatic framework for evaluating code clone detection tools. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, Colorado, USA, April 1–4, 2009, Workshops Proceedings; 2009:157–166. doi:10.1109/ICSTW.2009.18.

[295] Stephan M., Alalfi M.H., Cordy J.R. Towards a taxonomy for simulink model mutations. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; 2014:206–215. doi:10.1109/ICSTW.2014.17.

[296] Pill I., Rubil I., Wotawa F., Nica M. SIMULTATE: a toolset for fault injection and mutation testing of simulink models. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11–15, 2016; 2016:168–173. doi:10.1109/ICSTW.2016.21.

[297] Khan Y., Hassine J. Mutation operators for the Atlas Transformation Language. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18–22, 2013; 2013:43–52. doi:10.1109/ICSTW.2013.13.

[298] Troya J., Bergmayr A., Burgue no L., Wimmer M. Towards systematic mutations for and with ATL model transformations. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICSTW.2015.7107455.

[299] Tisi M., Jouault F., Fraternali P., Ceri S., Bézivin J. On the use of higher-order model transformations. In: Model Driven Architecture - Foundations and Applications: 5th European Conference, ECMDA-FA 2009, Enschede, The Netherlands, June 23–26, 2009; 18–33. doi:10.1007/978-3-642-02674-4_3. LNCS. 2009;vol. 5562.

[300] Aranega V., Mottu J., Etien A., Degueule T., Baudry B., Dekeyser J. Towards an automation of the mutation analysis dedicated to model transformation. Softw. Test. Verif. Reliab. 2015;25(5–7):653–683. doi:10.1002/stvr.1532.

[301] Bartel A., Baudry B., Munoz F., Klein J., Mouelhi T., Traon Y.L. Model driven mutation applied to adaptative systems testing. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011 Workshops Proceedings, March 21–March 25, 2011, Berlin, Germany; 2011:408–413. doi:10.1109/ICSTW.2011.24.

[302] Arcaini P., Gargantini A., Riccobene E. Using mutation to assess fault detection capability of model review. Softw. Test. Verif. Reliab. 2015;25(5–7):629–652. doi:10.1002/stvr.1530.

[303] Kaplan M., Klinger T., Paradkar A.M., Sinha A., Williams C., Yilmaz C. Less is more: a minimalistic approach to UML model-based conformance test generation. In: First International Conference on Software Testing, Verification, and Validation, ICST 2008, Lillehammer, Norway, April 9–11, 2008; 2008:82–91. doi:10.1109/ICST.2008.48.

[304] Fraser G., Wotawa F. Using model-checkers to generate and analyze property relevant test-cases. Softw. Q. J. 2008;16(2):161–183. doi:10.1007/s11219-007-9031-6.

[305] Trakhtenbrot M. Mutation patterns for temporal requirements of reactive systems. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW); 2017:116–121. doi:10.1109/ICSTW.2017.27.

[306] Sullivan A., Wang K., Zaeem R.N., Khurshid S. Automated rest generation and mutation testing for alloy. In: 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST); 2017:264–275. doi:10.1109/ICST.2017.31.

[307] Xu D., el Ariss O., Xu W., Wang L. Testing aspect-oriented programs with finite state machines. Softw. Test. Verif. Reliab. 2012;22(4):267–293. doi:10.1002/stvr.440.

[308] Lindström B., Andler S.F., Offutt J., Pettersson P., Sundmark D. Mutating aspect-oriented models to test cross-cutting concerns. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICSTW.2015.7107456.

[309] Elrakaiby Y., Mouelhi T., Traon Y.L. Testing obligation policy enforcement using mutation analysis. In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:673–680. doi:10.1109/ICST.2012.157.

[310] Mouelhi T., Fleurey F., Baudry B. A generic metamodel for security policies mutation. In: First International Conference on Software Testing Verification and Validation, ICST 2008, Lillehammer, Norway, April 9–11, 2008, Workshops Proceedings; 2008:278–286. doi:10.1109/ICSTW.2008.2.

[311] Bertolino A., Daoudagh S., Lonetti F., Marchetti E., Martinelli F., Mori P. Testing of PolPA-based usage control systems. Softw. Q. J. 2014;22(2):241–271. doi:10.1007/s11219-013-9216-0.

[312] Mouelhi T., Traon Y.L., Baudry B. Transforming and selecting functional test cases for security policy testing. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009; 2009:171–180. doi:10.1109/ICST.2009.49.

[313] Nguyen P.H., Papadakis M., Rubab I. Testing delegation policy enforcement via mutation analysis. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18–22, 2013; 2013:34–42. doi:10.1109/ICSTW.2013.12.

[314] Hwang J., Xie T., Kateb D.E., Mouelhi T., Traon Y.L. Selection of regression system tests for security policy evolution. In: IEEE/ACM International Conference on Automated Software Engineering, ASE’12, Essen, Germany, September 3–7, 2012; 2012:266–269. doi:10.1145/2351676.2351719.

[315] Dadeau F., Héam P., Kheddam R. Mutation-based test generation from security protocols in HLPSL. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21–25, 2011; 2011:240–248. doi:10.1109/ICST.2011.42.

[316] Dadeau F., Héam P., Kheddam R., Maatoug G., Rusinowitch M. Model-based mutation testing from security protocols in HLPSL. Softw. Test. Verif. Reliab. 2015;25(5–7):684–711. doi:10.1002/stvr.1531.

[317] Papadakis M., Henard C., Traon Y.L. Sampling program inputs with mutation analysis: going beyond combinatorial interaction testing. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014, March 31 2014–April 4, 2014, Cleveland, OH, USA; 2014:1–10. doi:10.1109/ICST.2014.11.

[318] Zhang Z., Wu T., Zhang J. Boundary value analysis in automatic white-box test generation. In: 26th IEEE International Symposium on Software Reliability Engineering, ISSRE 2015, Gaithersbury, MD, USA, November 2–5, 2015; 2015:239–249. doi:10.1109/ISSRE.2015.7381817.

[319] Patrick M., Jia Y. Kernel density adaptive random testing. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICSTW.2015.7107451.

[320] Patrick M., Jia Y. KD-ART: should we intensify or diversify tests to kill mutants?. Inf. Softw. Technol. 2017;81:36–51. doi:10.1016/j.infsof.2016.04.009.

[321] Galeotti J.P., Furia C.A., May E., Fraser G., Zeller A. Inferring loop invariants by mutation, dynamic analysis, and static checking. IEEE Trans. Softw. Eng. 2015;41(10):1019–1037. doi:10.1109/TSE.2015.2431688.

[322] Andrés C., Merayo M.G., Núñez M. Passive testing of stochastic timed systems. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009; 2009:71–80. doi:10.1109/ICST.2009.35.

[323] Andrés C., Merayo M.G., Molinero C. Advantages of mutation in passive testing: an empirical study. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009, Workshops Proceedings; 2009:230–239. doi:10.1109/ICSTW.2009.33.

[324] Andrés C., Merayo M.G., Núñez M. Formal passive testing of timed systems: theory and tools. Softw. Test. Verif. Reliab. 2012;22(6):365–405. doi:10.1002/stvr.1464.

[325] Pankumhang T., Rutherford M. Iterative instrumentation for code coverage in time-sensitive systems. In: 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICST.2015.7102594.

[326] Groce A., Ahmed I., Jensen C., McKenney P.E. How verified is my code? Falsification-driven verification (T). In: 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9–13, 2015; 2015:737–748. doi:10.1109/ASE.2015.40.

[327] Svajlenko J., Roy C.K., Duszynski S. ForkSim: generating software forks for evaluating cross-project similarity analysis tools. In: 13th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2013, Eindhoven, Netherlands, September 22–23, 2013; 2013:37–42. doi:10.1109/SCAM.2013.6648182.

[328] Bowes D., Hall T., Harman M., Jia Y., Sarro F., Wu F. Mutation-aware fault prediction. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18–20, 2016; 2016:330–341. doi:10.1145/2931037.2931039.

[329] Zhang J., Lou Y., Zhang L., Hao D., Zhang L., Mei H. Isomorphic regression testing: executing uncovered branches without test augmentation. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, November 13–18, 2016; 2016:883–894. doi:10.1145/2950290.2950313.

[330] Nardo D.D., Pastore F., Briand L.C. Generating complex and faulty test data through model-based mutation analysis. In: 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, April 13–17, 2015; 2015:1–10. doi:10.1109/ICST.2015.7102589.

[331] Arcaini P., Gargantini A., Riccobene E., Vavassori P. Rehabilitating equivalent mutants as static anomaly detectors in software artifacts. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; 2015:1–6. doi:10.1109/ICSTW.2015.7107452.

[332] Arcaini P., Gargantini A., Riccobene E., Vavassori P. A novel use of equivalent mutants for static anomaly detection in software artifacts. Inf. Softw. Technol. 2017;81:52–64. doi:10.1016/j.infsof.2016.01.019.

[333] Baudry B., Allier S., Monperrus M. Tailored source code transformations to synthesize computationally diverse program variants. In: International Symposium on Software Testing and Analysis, ISSTA 2014, San Jose, CA, USA, July 21–26, 2014; 2014:149–159. doi:10.1145/2610384.2610415.

[334] Lisper B., Lindström B., Potena P., Saadatmand M., Bohlin M. Targeted mutation: efficient mutation analysis for testing non-functional properties. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW); 2017:65–68. doi:10.1109/ICSTW.2017.18.

[335] A. Babu, mutatepy: A Mutation Testing Tool for C, http://members.femto-st.fr/pierre-cyrille-heam/mutatepy (accessed May 2017).

[336] Moore I. Jester and Pester. 2001. (accessed May 2017) http://jester.sourceforge.net/.

[337] Delamaro M.E., Maldonado J.C., Vincenzi A.M.R. Proteum/IM 2.0: an integrated mutation testing environment. In: Boston, MA: Springer US; 2001:978-1-4757-5939-691–101. doi:10.1007/978-1-4757-5939-6_17.

[338] Andrews J.H., Zhang Y. General test result checking with log file analysis. IEEE Trans. Softw. Eng. 2003;29(7):634–648. doi:10.1109/TSE.2003.1214327.

[339] Andrews J.H., Briand L.C., Labiche Y. Is mutation an appropriate tool for testing experiments? In: Roman G., Griswold W.G., Nuseibeh B., eds. 27th International Conference on Software Engineering (ICSE 2005), May 15–21, 2005, St. Louis, MO, USA; ACM; 2005:402–411. doi:10.1145/1062455.1062530.

[340] Ma Y., Offutt J., Kwon Y.R. MuJava: a mutation system for Java. In: Osterweil L.J., Rombach H.D., Soffa M.L., eds. 28th International Conference on Software Engineering (ICSE 2006), Shanghai, China, May 20–28, 2006; ACM; 2006:827–830. doi:10.1145/1134425.

[341] Offutt J., Ma Y., Kwon Y.R. An experimental mutation system for Java. ACM SIGSOFT Softw. Eng. Notes. 2004;29(5):1–4. doi:10.1145/1022494.1022537.

[342] Do H., Rothermel G. On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Trans. Softw. Eng. 2006;32(9):733–752. doi:10.1109/TSE.2006.92.

[343] Tuya J., Suarez-Cabal M.J.C. de la Riva. SQLMutation: a tool to generate mutants of SQL database queries. In: Second Workshop on Mutation Analysis (Mutation 2006—ISSRE Workshops 2006); 2006:1. doi:10.1109/MUTATION.2006.13.

[344] Jumble Testing Tool for Java. 2007. (accessed May 2017) http://jumble.sourceforge.net/.

[345] Feng X., Marr S., O’Callaghan T. ESTP: an experimental software testing platform. In: Testing: Academic Industrial Conference—Practice and Research Techniques (taic part 2008); 2008:59–63. doi:10.1109/TAIC-PART.2008.8.

[346] Tan R.P., Edwards S. Evaluating automated unit testing in Sulu. In: First International Conference on Software Testing, Verification, and Validation, ICST 2008, Lillehammer, Norway, April 9–11, 2008; IEEE Computer Society; 2008:62–71. doi:10.1109/ICST.2008.59.

[347] Jia Y., Harman M. MILU: a customizable, runtime-optimized higher order mutation testing tool for the full C language. In: Proceedings of the 3rd Testing: Academic and Industrial Conference Practice and Research Techniques (TAIC PART 2008); 2008:94–98 Windsor, UK.

[348] Rajan A., Whalen M.W., Staats M., Heimdahl M.P.E. Requirements coverage as an adequacy measure for conformance testing. In: Liu S., Maibaum T.S.E., Araki K., eds. Formal Methods and Software Engineering, 10th International Conference on Formal Engineering Methods, ICFEM 2008, Kitakyushu-City, Japan, October 27–31, 2008; Springer; 86–104. doi:10.1007/978-3-540-88194-0_8. Proceedings, Lecture Notes in Computer Science. 2008;vol. 5256.

[349] Weiglhofer M., Wotawa F. “On the fly” input output conformance verification. In: Proceedings of the IASTED International Conference on Software Engineering; Anaheim, CA, USA: ACTA Press; 2008:978-0-88986-716-1286–291.

[350] Zhou C., Frankl P.G. Mutation testing for Java database applications. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009; 2009:396–405. doi:10.1109/ICST.2009.43.

[351] Zhou C., Frankl P.G. JDAMA: Java database application mutation analyser. Softw. Test. Verif. Reliab. 2011;21(3):241–263. doi:10.1002/stvr.462.

[352] Delamare R., Baudry B., Ghosh S., Traon Y.L. A test-driven approach to developing pointcut descriptors in aspectJ. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009; 2009:376–385. doi:10.1109/ICST.2009.41.

[353] Delamare R., Baudry B., Traon Y.L. AjMutator: a tool for the mutation analysis of aspectJ pointcut descriptors. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009, Workshops Proceedings; 2009:200–204. doi:10.1109/ICSTW.2009.41.

[354] Domínguez-Jiménez J.J., Estero-Botaro A., Medina-Bulo I. A framework for mutant genetic generation for WS-BPEL. In: Nielsen M., Kucera A., Miltersen P.B., Palamidessi C., Tuma P., Valencia F.D., eds. SOFSEM 2009: Theory and Practice of Computer Science, 35th Conference on Current Trends in Theory and Practice of Computer Science, Spindleruv Mlýn, Czech Republic, January 24–30, 2009; Springer; 229–240. doi:10.1007/978-3-540-95891-8_23. Lecture Notes in Computer Science. 2009;5404 vol.

[355] Aydal E.G., Paige R.F., Utting M., Woodcock J. Putting formal specifications under the magnifying glass: model-based testing for validation. In: Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, CO, USA, April 1–4, 2009; IEEE Computer Society; 2009:131–140. doi:10.1109/ICST.2009.20.

[356] Dobolyi K., Weimer W. Harnessing web-based application similarities to aid in regression testing. In: ISSRE 2009, 20th International Symposium on Software Reliability Engineering, Mysuru, Karnataka, India, November 16–19, 2009; IEEE Computer Society; 2009:71–80. doi:10.1109/ISSRE.2009.18.

[357] Ellims M., Ince D., Petre M. The Csaw C mutation tool: initial results. In: Testing: Academic and Industrial Conference Practice and Research Techniques—MUTATION (TAICPART-MUTATION 2007); 2007:185–192. doi:10.1109/TAIC.PART.2007.28.

[358] Lakehal A., Parissis I. Structural coverage criteria for LUSTRE/SCADE programs. Softw. Test. Verif. Reliab. 2009;19(2):133–154. doi:10.1002/stvr.394.

[359] Durães J., Madeira H. Definition of software fault emulation operators: a field data study. In: 2003 International Conference on Dependable Systems and Networks (DSN 2003), June 22–25, 2003, San Francisco, CA, USA; IEEE Computer Society; 2003:105–114. doi:10.1109/DSN.2003.1209922.

[360] Madeyski L., Radyk N. Judy—a mutation testing tool for Java. IET Softw. 2010;4(1):32–42. doi:10.1049/iet-sen.2008.0038.

[361] Just R. The major mutation framework: efficient and scalable mutation analysis for Java. In: International Symposium on Software Testing and Analysis, ISSTA ’14, San Jose, CA, USA, July 21–26, 2014; 2014:433–436. doi:10.1145/2610384.2628053.

[362] Madiraju P., Namin A.S. Para(μ) - a partial and higher-order mutation tool with concurrency operators. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Berlin, Germany, March 21–25, 2011, Workshop Proceedings; 2011:351–356. doi:10.1109/ICSTW.2011.34.

[363] MuBPEL - A Mutation Testing Tool for WS-BPEL. 2011. (accessed May 2017) https://neptuno.uca.es/redmine/projects/sources-fm/wiki/MuBPEL/.

[364] Winbladh K., Ranganathan A. Evaluating test selection strategies for end-user specified flow-based applications. In: Alexander P., Pasareanu C.S., Hosking J.G., eds. 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), Lawrence, KS, USA, November 6–10, 2011; IEEE Computer Society; 2011:400–403. doi:10.1109/ASE.2011.6100083.

[365] Saifan A.A., Dingel J., Bradbury J.S., Posse E. Implementing and evaluating a runtime conformance checker for mobile agent systems. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21–25, 2011; IEEE Computer Society; 2011:269–278. doi:10.1109/ICST.2011.62.

[366] Just R., Kapfhammer G.M., Schweiggert F. Using conditional mutation to increase the efficiency of mutation analysis. In: Proceedings of the 6th International Workshop on Automation of Software Test, AST 2011, Waikiki, Honolulu, HI, USA, May 23–24, 2011; 2011:50–56. doi:10.1145/1982595.1982606.

[367] Dan H., Hierons R.M. SMT-C: a semantic mutation testing tools for C. In: Fifth IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, Canada, April 17–21, 2012; 2012:654–663. doi:10.1109/ICST.2012.155.

[368] Schirp M. Mutation Testing for Ruby. 2012. (accessed May 2017) https://github.com/mbj/mutant.

[369] Kusano M., Wang C. CCmutator: a mutation generator for concurrency constructs in multithreaded C/C++ applications. In: 2013 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013, Silicon Valley, CA, USA, November 11–15, 2013; 2013:722–725. doi:10.1109/ASE.2013.6693142.

[370] Bradbury J.S., Cordy J.R., Dingel J. Mutation operators for concurrent Java (J2SE 5.0). In: Second Workshop on Mutation Analysis (Mutation 2006–ISSRE Workshops 2006); 2006:11. doi:10.1109/MUTATION.2006.10.

[371] Kapfhammer G.M., McMinn P., Wright C.J. Search-based testing of relational schema integrity constraints across multiple database management systems. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013, Luxembourg, Luxembourg, March 18–22, 2013; IEEE Computer Society; 2013:31–40. doi:10.1109/ICST.2013.47.

[372] Bertolino A., Daoudagh S., Lonetti F., Marchetti E. XACMUT: XACML 2.0 mutants generator. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18–22, 2013; IEEE Computer Society; 2013:28–33. doi:10.1109/ICSTW.2013.11.

[373] Ye C., Jacobsen H. Whitening SOA testing via event exposure. IEEE Trans. Softw. Eng. 2013;39(10):1444–1465. doi:10.1109/TSE.2013.20.

[374] Derezińska A., Hałas K. Analysis of Mutation Operators for the Python Language. In: Cham: Springer International Publishing; 2014:978-3-319-07013-1155–164. doi:10.1007/978-3-319-07013-1_15.

[375] Omar E., Ghosh S., Whitley D. HOMAJ: a tool for higher order mutation testing in aspectJ and Java. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; 2014:165–170. doi:10.1109/ICSTW.2014.19.

[376] Mahajan S., Halfond W.G.J. Finding HTML presentation failures using image comparison techniques. In: Crnkovic I., Chechik M., Grünbacher P., eds. ACM/IEEE International Conference on Automated Software Engineering, ASE ’14, Vasteras, Sweden, September 15–19, 2014; ACM; 2014:91–96. doi:10.1145/2642937.2642966.

[377] El-Fakih K., Simão A., Jadoon N., Maldonado J.C. On studying the effectiveness of extended finite state machine based test selection criteria. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; IEEE Computer Society; 2014:222–229. doi:10.1109/ICSTW.2014.25.

[378] Maruchi K., Shin H., Sakai M. MC/DC-like structural coverage criteria for function block diagrams. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; IEEE Computer Society; 2014:253–259. doi:10.1109/ICSTW.2014.27.

[379] Walsh T.A., McMinn P., Kapfhammer G.M. Automatic detection of potential layout faults following changes to responsive web pages (N). In: Cohen M.B., Grunske L., Whalen M., eds. 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9–13, 2015; IEEE Computer Society; 2015:709–714. doi:10.1109/ASE.2015.31.

[380] Abreu R., Hofer B., Perez A., Wotawa F. Using constraints to diagnose faulty spreadsheets. Softw. Q. J. 2015;23(2):297–322. doi:10.1007/s11219-014-9236-4.

[381] Belli F., Beyazit M., Endo A.T., Mathur A.P.A. da Silva Simão. Fault domain-based testing in imperfect situations: a heuristic approach and case studies. Softw. Q. J. 2015;23(3):423–452. doi:10.1007/s11219-014-9242-6.

[382] Fabbri S.C.P.F., Delamaro M.E., Maldonado J.C., Masiero P.C. Mutation analysis testing for finite state machines. In: 5th International Symposium on Software Reliability Engineering, ISSRE 1994, Monterey, CA, USA, November 6–9, 1994; IEEE; 1994:220–229. doi:10.1109/ISSRE.1994.341378.

[383] Simao A., Petrenko A., Maldonado J.C. Comparing finite state machine test coverage criteria. IET Softw. 2009;3(2):91–105.

[384] Guan J., Offutt J. A model-based testing technique for component-based real-time embedded systems. In: Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13–17, 2015; IEEE Computer Society; 2015:1–10. doi:10.1109/ICSTW.2015.7107407.

[385] Ocariza Jr F.S., Pattabiraman K., Mesbah A. Detecting inconsistencies in JavaScript MVC applications. In: Bertolino A., Canfora G., Elbaum S.G., eds. 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16–24, 2015, vol. 1; IEEE Computer Society; 2015:325–335. doi:10.1109/ICSE.2015.52.

[386] Krenn W., Schlick R., Tiran S., Aichernig B., Jobstl E., Brandl H. MoMut::UML model-based mutation testing for UML. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST); 2015:1–8. doi:10.1109/ICST.2015.7102627 ISSN 2159-4848.

[387] Enoiu E.P., Causevic A., Sundmark D., Pettersson P. A controlled experiment in testing of safety-critical embedded software. In: 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016, Chicago, IL, USA, April 11–15, 2016; IEEE Computer Society; 2016:1–11. doi:10.1109/ICST.2016.15.

[388] Matinnejad R., Nejati S., Briand L.C., Bruckmann T. Automated test suite generation for time-continuous simulink models. In: Dillon L.K., Visser W., Williams L., eds. Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016; ACM; 2016:595–606. doi:10.1145/2884781.2884797.

[389] Henard C., Papadakis M., Harman M., Jia Y., Traon Y.L. Comparing white-box and black-box test prioritization. In: Dillon L.K., Visser W., Williams L., eds. Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016; ACM; 2016:523–534. doi:10.1145/2884781.2884791.

[390] Devroey X., Perrouin G., Schobbens P.Y., Heymans P. Poster: VIBeS, transition system mutation made easy. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering; 817–818. doi:10.1109/ICSE.2015.263. 0270-52572015;vol. 2.

[391] Parsai A., Murgia A., Demeyer S. LittleDarwin: a feature-rich and extensible mutation testing framework for large and complex Java systems. In: Cham: Springer International Publishing; 2017:978-3-319-68972-2148–163. doi:10.1007/978-3-319-68972-2_10.

[392] Rothermel G., Untch R.H., Chu C., Harrold M.J. Test case prioritization: an empirical study. In: 1999 International Conference on Software Maintenance, ICSM 1999, Oxford, England, UK, August 30–September 3, 1999; 1999:179–188. doi:10.1109/ICSM.1999.792604.

[393] Namin A.S., Kakarla S. The use of mutation in testing experiments and its sensitivity to external threats. In: Proceedings of the 20th International Symposium on Software Testing and Analysis, ISSTA 2011, Toronto, ON, Canada, July 17–21, 2011; 2011:342–352. doi:10.1145/2001420.2001461.

[394] Kintis M., Papadakis M., Papadopoulos A., Valvis E., Malevris N. Analysing and comparing the effectiveness of mutation testing tools: a manual study. In: 16th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2016, Raleigh, NC, USA, October 2–3, 2016; 2016:147–156. doi:10.1109/SCAM.2016.28.

[395] Binder R.V. Testing object-oriented systems: models, patterns, and tools. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc.; 1999.0-201-80938-9.

[396] Visser W. What makes killing a mutant hard. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, Singapore, September 3–7, 2016; 2016:39–44. doi:10.1145/2970276.2970345.

[397] Ammann P., Delamaro M.E., Offutt J. Establishing theoretical minimal sets of mutants. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014, March 31 2014–April 4, 2014, Cleveland, OH, USA; 2014:21–30. doi:10.1109/ICST.2014.13.

[398] Kurtz B., Ammann P., Delamaro M.E., Offutt J., Deng L. Mutant subsumption graphs. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; 2014:176–185. doi:10.1109/ICSTW.2014.20.

[399] Arcuri A., Briand L. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: ICSE; 2011:978-1-4503-0445-01–10. doi:10.1145/1985793.1985795.

[400] Harman M., McMinn P., de Souza J.T., Yoo S. Search based software engineering: techniques, taxonomy, tutorial. In: Berlin, Heidelberg: Springer; 2012:978-3-642-25231-01–59. doi:10.1007/978-3-642-25231-0_1.

[401] Delamaro M.E., Offutt J. Assessing the influence of multiple test case selection on mutation experiments. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31–April 4, 2014, Cleveland, OH, USA; 2014:171–175. doi:10.1109/ICSTW.2014.22.

[402] Kintis M., Papadakis M., Papadopoulos A., Valvis E., Malevris N., Le Traon Y. How effective mutation testing tools are? An empirical analysis of java mutation testing tools with manual analysis and real faults. Empir. Softw. Eng. 2017;doi:10.1007/s10664-017-9582-5 (accepted for publication).

[403] Gopinath R., Ahmed I., Alipour M.A., Jensen C., Groce A. Does choice of mutation tool matter?. Softw. Q. J. 1573-13672016;1–50. doi:10.1007/s11219-016-9317-7.

[404] Márki A., Lindström B. Mutation tools for Java. In: Proceedings of the Symposium on Applied Computing, SAC 2017, Marrakech, Morocco, April 3–7, 2017; 2017:1364–1415. doi:10.1145/3019612.3019825.

[405] Arcuri A., Briand L.C. Adaptive random testing: an illusion of effectiveness? In: Dwyer M.B., Tip F., eds. Proceedings of the 20th International Symposium on Software Testing and Analysis, ISSTA 2011, Toronto, ON, Canada, July 17–21, 2011; ACM; 2011:265–275. doi:10.1145/2001420.2001452.

u06-02-9780128151211

Mike Papadakis is a research scientist at the Interdisciplinary Centre for Security, Reliability and Trust (SnT) at the University of Luxembourg. He received a PhD diploma in Computer Science from the Athens University of Economics and Business. His research interests include software testing, static analysis, prediction modelling, mutation analysis, and search-based software engineering.

u06-03-9780128151211

Marinos Kintis is a research associate at the Interdisciplinary Centre for Security, Reliability and Trust (SnT) at the University of Luxembourg. He received the PhD degree from the Department of Informatics of the Athens University of Economics and Business in 2016. The main topic of his dissertation was the introduction of effective techniques to ameliorate the adverse effects of the Equivalent Mutant Problem when testing software with Mutation. His main research interests include software testing, mutation testing, and program analysis. He was awarded a Best Paper Award at the 16th International Working Conference on Source Code Analysis and Manipulation (SCAM 2016) and co-organised the 13th International Workshop on Mutation Analysis (MUTATION 2018).

u06-04-9780128151211

Jie Zhang is a final-year PhD candidate at the School of Electronics Engineering and Computer Science, Peking University, P.R. China, supervised by Lu Zhang. She is also a research associate in CREST, UCL, supervised by Earl Barr and Mark Harman. She has won the 2016 Fellowship at Microsoft Research Asia, the Top-ten Research Excellence Award of EECS, Peking University, the Lee Wai Wing Scholarship at Peking University, the 2015 National Scholarship, and so on. She served on the program committees of Mutation 2017 and Mutation 2018. Her major research interests are software testing, program analysis, end-user programming, and API mining.

u06-05-9780128151211

Yue Jia is a lecturer in the Department of Computer Science at University College London. His research interests cover mutation testing, app store analysis, and search-based software engineering. Dr. Jia is director of MaJiCKe, an automated test data generation start up and also co-founder of Appredict, an app store analytics company, spun out from UCL's UCLappA group.

u06-06-9780128151211

Yves Le Traon is professor at University of Luxembourg where he leads the SERVAL (SEcurity, Reasoning and VALidation) research team. His research interests within the group include (1) innovative testing and debugging techniques, (2) Android apps security and reliability using static code analysis, machine learning techniques and, (3) model-driven engineering with a focus on IoT and CPS. His reputation in the domain of software testing is acknowledged by the community. He has been General Chair of major conferences in the domain, such as the 2013 IEEE International Conference on Software Testing, Verification and Validation (ICST), and Program Chair of the 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS). He serves at the editorial boards of several, internationally known journals (STVR, SoSym, IEEE Transactions on Reliability) and is author of more than 140 publications in international peer-reviewed conferences and journals.

u06-07-9780128151211

Mark Harman is currently an engineering manager at Facebook and a professor of Software Engineering in the Department of Computer Science at University College London, where he directed the CREST centre for 10 years (2006–2017) and was Head of Software Systems Engineering (2012–2017). He is widely known for work on source code analysis, software testing, app store analysis, and Search Based Software Engineering (SBSE), a field he co-founded and which has grown rapidly to include over 1600 authors spread over more than 40 countries. His SBSE and testing work has been used by many organisations including Daimler, Ericsson, Google, Huawei, Microsoft, and Visa. Prof. Harman is a co-founder (and was co-director) of Appredict, an app store analytics company, spun out from UCL's UCLappA group, and was the chief scientific advisor to Majicke, an automated test data generation start up. In February 2017, he and the other two co-founders of Majicke (Yue Jia and Ke Mao) moved to Facebook, London, in order to develop their research and technology as part of Facebook.

a Subsumption is not guaranteed but it is probable to happen [32].

b Test prioritization techniques are typically assessed based on Average Percentage Faults Detected (APFD) or Normalized APFD (NAPFD) [392].

c The figure does not include tools that were used in fewer than four publications.

d The difference between the notions of minimal and disjoint mutants is that minimal mutants is the smallest possible representative set of mutants while the disjoint ones is a set that has no redundancies (perhaps not the minimal one) [33,39].

e Two methods can have a similar number of mutants killed on a project (overall number), but quite different numbers, of mutants killed, on the individual units of the project.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.133.156.251