Highlights from ESEC/FSE 2018 – Day 2

Image above: AI buffet from Tim Menzie’s slides. I guess Deep Learning is the caloric apple pie I’m not supposed to have, unless I’m willing to pay for the calories. Alas, I really like apple pie.

Continuing my post of yesterday, today I would like to highlight the following points from ESEC/FSE 2018:

• Obviously, my favorite part of the day was my own presentation on what I’ve called Experiment-Oriented Computing (EOC)! Essentially, what I did was to call attention to the many experimentation aspects that exists dispersed in our software systems. How come people have not yet realized this? We should recognize these similarities in order to properly formulate both general problems and general computational solutions to experimentation matters. I provide numerous examples, from which an architecture and various supporting principles can be abstracted. I’d be happy to receive any feedback, both positive and negative! (See “The Case for Experiment-Oriented Computing”)
• Tim Menzies, who stepped in as the keynote (the original one could not come), argued in favor of combining Artificial Intelligence with Software Engineering (SE). That is because AI software is still software and therefore can benefit from SE practices. Indeed, refactoring the problem of interest can actually produce superior Machine Learning (ML) models. Conversely, by now it is clear that ML techniques improve SE tasks (e.g., configuration optimization, effort estimation). He gave some provocative examples, including one in which a “smart” k-means use defeats “brute force” Deep Learning. (See http://tiny.cc/fse18)
• More work on reusable automated UI testing using Machine Learning. Now can someone please tell our customers that this is what they should be doing? I remain shocked that many big companies still do mostly manual testing even with so many technologies available! (See “AppFlow: Using Machine Learning to Synthesize Robust, Reusable UI Tests”)
• Variational Execution is an alternative to Symbolic Execution, which can be useful when variables have a finite and small number of possible values. Example applications include automatic program repair and mutation testing. (See “Beyond Testing Configurable Systems: Applying Variational Execution to Automatic Program Repair and Higher Order Mutation Testing”)
• Formal proofs (e.g., Coq or HOL proofs) actually contain a lot of linguistic patterns. This means that NLP techniques, traditional code completion, programming abstractions, perhaps writing classes, and really anything else related to informal languages are all quite relevant. Similarly, it is possible to mine error handling specifications from programs. By the way, all this reminded me of an insightful (but not well-known) book on Philosophy of Science called Patterns of Discovery. (See “On the Naturalness of Proofs” and “Mining Semantic Loop Idioms”)
• You can test Deep Learning models using Fuzzing techniques. “Neuron coverage” is as an analogous of “code coverage”, which I find quite comforting.  (See “DLFuzz: Differential Fuzzing Testing of Deep Learning Systems”)
• Roboticists, I was told by Afsoon Afzal (yesterday, but forgot to mention before), perform no proper functional testing using simulation. The problem seems to be that it is very hard to describe explicitly the precise properties one is interested in, since the physical world is so messy. No good specification language exists. So simulators are used to perform manual, instead of automated, testing. Well, I for one do not believe it is not possible to design an appropriate specification language. My guess is no one cared enough, although mistakes are very expensive. What are they thinking? (See “Quality Assurance Automation in Autonomous Systems”)

The proceedings are in the (payed) ACM DL, but you can probably find pre-prints of some of these papers in the authors’ sites.