What constitutes a truly excellent research publication in Digital Archaeology?
This question arises in the context that, like every other subject area in the UK, we’re under pressure to prepare for the next round of the Research Excellence Framework (REF), the periodic review of research quality across UK universities which is anticipated to take place in 2021. The results of these reviews affect institutional and subject-based rankings and also feed into the calculation of the annual research block grant (Research Excellence Grant) from central government which here in Scotland was just under £232,000,000 for 2017-18. So money and reputation are at stake: small wonder University administrators across the country are turning their minds to interim reviews and internal assessments in anxious anticipation.
A substantial proportion of the overall review outcome is based on academic publications which are assessed by peer review and graded using the (in)famous ratings of 4*, 3*, 2*, 1* and unclassified (only 3* and 4* actually ‘count’ in the final calculations; the lesser classifications can actually count against a unit). As part of our own institutional internal review, we’ve been asked to identify examples of 4* research outputs in our field (by others than ourselves, naturally!) as a means of helping us to ensure our own outputs are potentially rateable as 4*. What is 4*, you ask? We’re told “you’ll know it when you see it” and are directed to the REF 2014 definitions where 4* is defined as world-leading quality in terms of originality, significance, and rigour. By way of comparison, 3* is internationally excellent in the same terms, while 2* is internationally recognised – perish the thought that we may be splitting hairs here. And the less said about 1* the better, though many might consider being of national excellence to be no bad thing in itself.
So, to rephrase the original question in REF terms, what constitutes a 4* publication in Digital Archaeology?
Helpfully, the last REF provided some glosses for “originality, significance, and rigour”. Originality is refined in terms of innovation: for instance, it is suggested work which engages with new and/or complex problems, develops innovative research methods, and/or advances theory. Significance is seen in terms of “the development of the intellectual agenda of the field”, while rigour relates to “the intellectual precision, robustness and appropriateness of the concepts, analyses, theories and methodologies deployed”. On that basis, it might reasonably be assumed that the work is widely cited, reviewed (positively, presumably!), represents a large amount of work, is probably in a journal with a high Impact Factor, and whose academic impact goes beyond the immediate field.
Assuming we’re any the wiser about evaluating 4* quality, where are the examples of 4* outputs in Digital Archaeology? Maybe it’s just me, but I’m struggling to come up with examples for our review meeting. That’s not to say that there isn’t a good deal of innovative, enlightening, and robust (choose your own adjectives from those above!) Digital Archaeology out there – there is. But is it good enough?
In a paper presented in the ‘Everything wrong with …’ session organised by Stephani Crabtree, Philip Iris and Doug Rocks-Macqueen at the Computer Applications in Archaeology conference in Atlanta earlier this year, I argued that we are in the midst of a largely unrecognised crisis for Digital Archaeology. The relentless change, novelty, and innovation in hardware and software means there’s always something new on the horizon, always something becoming redundant, and so the game moves on. We constantly change the terms of our engagement and in the process duck the implications of our actions. We are largely satisfied with justifying our digital methodologies within their immediate application context, less concerned with their mediation in the production of archaeological knowledge. As a consequence, it’s easy to perceive of Digital Archaeology as under-theorised, subordinate and under-valued: a practice-based field using borrowed technologies, and correspondingly providing little more than a technical service to Archaeology as a whole (Huggett 2015, 80).
We don’t need to lose our emphasis on the practical applications of digital tools; it is perhaps more a case of changing the environment in which these are developed and applied. We lack a clear-cut, unambiguously and unashamedly theoretical and philosophical approach to accompany, support, and contextualise the practical. In this regard, we might revisit Marcos Llobera’s proposals for a new Archaeological Information Science, which arose out of his somewhat frustrated critique of Digital Archaeology and its relationship with the broader discipline. Many of his criticisms of our publications still stand (Llobera 2011, 216):
- Most do not make reference to any coherent body of theory and/or archaeology at large
- They reflect the possibilities and limitations of the software, but not necessarily the requirements and aspirations of archaeologists
- They focus on narrow issues of implementation without actively engaging in the improvement of the tools
- There are no clear well-defined research lines or paradigms
- They are not part of any well-established curriculum within archaeology
Or, to put it another way, we’re too particularistic, we’re too technology-bound, we behave as little more than end-users, all too often our tools are in search of an application, and we lack critical technical competencies.
Perhaps by addressing these challenges, we may begin to make Digital Archaeology count – to ensure that what we create and build has value, not solely in the instrumental sense, but as a discrete sub discipline which, through its engagement with the digital as an object of critical study, is capable of considered, knowledgeable, and deliberate transformation. We need to position ourselves at the forefront of key agendas in Archaeology and the broader humanities: openness and public engagement, future-oriented thinking, interdisciplinarity, big data, industry links, artificial intelligence, knowledge creation and organisation, visualisation, and so on. We need to lose our cringe: Digital Archaeology can be, and should be, so much more than it is now and has been for the past twenty years, and we should collectively have the vision and drive to make it so. Only then can we really put ourselves in a position to be able to claim to produce world-leading research excellence in Digital Archaeology.
References
Huggett, J. 2015 ‘Challenging Digital Archaeology’, Open Archaeology 1, 79-85. http://eprints.gla.ac.uk/104046/
Llobera, M. 2011. ‘Archaeological Visualisation: Towards an Archaeological Information Science (AISc)’, Journal of Archaeological Method and Theory 18: 193-223. http://dx.doi.org/10.1007/s10816-010-9098-4
