This article was prompted by responses to a tweet I wrote in response to an analysis of the research area of papers submitted to the 2022 ICSE conference, the flagship conference for software engineering research. These led me to reflect on software engineering research in general. I’m sorry to say that I think that we, as a community, have not really delivered very much that’s substantive in over 40 years of research.
Fundamentally, I think there are 3 related root causes of this situation:
- Short-termism.
- Reductionist thinking
- Competition rather than cooperation
Short-termism is now endemic in software engineering (and indeed in all computer science) research. This is partly a consequence of the ‘publish or perish’ culture that started in the 1970s and that is now endemic in most countries. Researchers have to keep writing papers to remain credible - and who can blame them? There is a focus on short-term projects, publication of interim results, an unwillingness to invest in producing robust and reusable demonstrator systems and a tendency to jump on whatever bandwagon is fashionable at a particular moment in time (e.g. formal methods in the 1980s, machine learning now).
This short-termism has the direct consequence that much software engineering research is lacking ambition. Researchers who are mindful of their success metrics are reluctant to tackle long-term difficult projects. In the UK, in the early 2000s, there was a proposal to identify ‘Grand Challenges in Computing Science’. Some interesting projects, such as a verifying compiler, were proposed - none came to fruition.
Short-termism has been exacerbated by the policy of many funding agencies, such as the European Commission, that academic research should be collaborative with industry. Industry, quite understandably, does not see its role as a long-term research funder and focuses on short or, at best, medium term research. Unfortunately, for both good and bad reasons, this research has a low priority for many companies. My experience over 30 years is that it is often under-funded, cancelled at short notice, and inadequately staffed.
The work on small scale, short-term projects can be justified (perhaps) in a reductionist world where it can be argued that these can form the building blocks for some larger system. However, the reality is that reductionist approaches simply do not scale to the construction of large-scale, complex IT systems. I’ve written about this before and I won’t go into it here but, put simply, reductionist approaches rely on relatively simple and understandable relationships between the parts of the system. Large-scale software systems simply aren’t like that.
Far too many researchers still do not understand this and believe that we only need to find some magic method and the problems of software engineering will be solved. The address research ‘problems without the faintest idea of how their work might scale. Consequently, as far as software engineering practice is concerned, much of what they do is practically useless.
Underlying these issues is the human problem that we have a fragmented community that has failed to act together on many occasions and has been more interested in competition than collaboration. The reality of a competitive environment is that those who are already well-funded are in the best position to gain more funding. It is very difficult for new entrants to get funding for innovative ideas so, if they are to make any progress at all, they have to propose incremental research that’s already established. They must live off crumbs from the rich man’s table.
As a community, senior members (and in this I include myself) have failed to create an inclusive environment where new ideas from outside of the traditional elite are supported. We do not act together when dealing with policy makers and funding agencies and, with rare exceptions, have not spoken out against the futile short-termism that has hampered the development of our field.
Is there hope for the future? Frankly, I am not optimistic as change will rely on senior members who already have tenure behaving altruistically, giving up on short-term research and stepping back from competition to allow younger researchers to flourish. Individual efforts are important (I like to think that I tried in the last few years of my career) but in reality change can only come about when a majority of the community wishes for this to happen.
My background
I was involved in research in software engineering or related disciplines from about 1981 until I retired in 2014. From 1984 until 2014, all of my research involved working with industry in the UK and across Europe. I was privileged to work with a wide range of people from different industries and countries and I learned a great deal from these experiences.
This article reflects my experience of UK and European research. I think that the situation differs in detail in other countries but all suffer from the same fundamental problems.