Audititis  a rampant contagion

Philip Tagg’s homepage

audititis n., a.k.a. ‘audit culture’: rampant contagion characterised by chronic proliferation of the irrational Unidimensional Quantification Syndrome (UQS) and resulting in severe impairment of human endeavour.

I originally wrote this page when I was still working at the University of Liverpool (UK). I hereby certify that the audititis epidemic described here was the main reason for my leaving the UK and for moving to Montréal in 2002.

Update, 9 years on (January 2011). I'm retired and back in the UK. The silly saga of managerial mayhem continues unabated here. Three recent books document the disgraceful destruction of dynamic research and, yes, of real efficiency, productivity and even profitability in our universities through the mindless application of short-term accountancy fetishes for mind-bogglingly unreal concepts of "efficiency", "productivity" and "profitability" (link).


Why, as a university teacher, I think it is wrong to comply with managerial edicts to submit a ‘Transparency Review’ by Philip Tagg (November 2001)

Explanation

Each academic year each university teacher in the UK has to submit a ‘Transparency Review’. It is probably the least onerous but one of the more patently ridiculous of the many bureaucratic audit tasks thrust upon us. The categories under which our work is classified in the review are so inadequate that if they had been presented to me in a second-year undergraduate mini-dissertation, I would have failed the student responsible for producing such an unproblematised and woolly taxonomy.

Extract from letter (7 October 2001) to those instructing me to submit a ‘Transparency Review’.

Before re-presenting my objections, let me say that I am aware that the ‘Transparency Review’ (hereinafter ‘TR’) is the brainchild of neither my Head of Department, nor of my university, but of the UK government, more specifically of HEFCE.[1] I also understand that HEFCE has instructed universities to instruct heads of department to instruct us to comply with the TR. I am, so to speak, lowest down in this chain of command. If I object I must do so upwards through the chain of command. I appreciate that those on the middle rungs of the ladder (please excuse the mixed metaphor) are in the difficult position of having to explain to those at the top why those below do not comply with the edicts issued from above. However, being at the bottom is, as I am sure you realise, not an enviable position either.

My first objection to the TR is simple, namely that it is a waste of time which could be productively spent developing education and research. University teachers are already obliged to spend enough time with new teaching programmes, committe meetings and internal bureaucracy without having further paper-squandering tasks foisted upon us. Indeed, here in the UK we are the laughing stock of university teachers in such countries as Canada whose subjection to managerial quantification mania is considerably less than ours.

My second objection concerns labour relations and the fact that the TR can be easily converted into a time-and-motion study which ought only to be implemented if employees are in agreement with its aims and procedures. My union — AUT (Association of University Teachers) — has yet to agree conclusively on whether or not the TR is an acceptable procedure. I definitely think it is not.

My third and most important objection to the TR is ethical. I have yet to meet a single colleague in this university or elsewhere in the UK who thinks the TR is of any use at all. All colleagues I have spoken to reiterate my first two objections and view the TR (along with other comparable exercises) as a tiresome game they have to play, an irritating hoop they feel they have to jump through, for fear of incurring the wrath and possible financial retribution of management (HEFCE via our university ‘superiors’). These colleagues fill in their forms in a few minutes, ensuring that their figures add up to 100%, but they have little or no clue to what, in real terms, those figures actually refer. After all, the criteria according to which disparate activities are to be included under which categories are anything but transparent, nor do the review’s categories correspond to the real everyday life of an acadmic since most of our activities simultaneously cover several of the categories available in varying proportions. Since most colleagues just want to get rid of the irritant as quickly as possible they fill everything in just to feed the managerial machine, i.e. for no good reason.

As if the dubious value of staff input at this stage were not enough, the figures (which, incidentally, do not even approximate the real amount of time we spend on various activities but express percentages so gross as to be misleading in proportion to an unspecified working week) are then totted up and collated by universities and delivered to HEFCE which is then supposed to make decisions about apportioning funds to universities on such flimsy grounds. Many colleagues are aware of these radical shortcomings in the TR and some even know that HEFCE is likely to abandon it within the next few years in response to its obvious inadequacy and inaccuracy. On top of all this, the return rate is in steady decline and teaching staff are demoralised and irritated by yet another bureaucratic burden which shows them no trust in their efforts to offer the best teaching and to carry out the best research they can.

A senior professor at one of the ‘old’ universities told me (Sunday 7 October, 2001) that he regarded, for the kind of reasons just presented, the TR and other exercises to be ‘a complete waste of time’. Admitting that he ‘went through the motions’ of complying with the TR in his department ‘just to get management of their backs’, he was well aware of the review’s fraudulent nature. He further apologised to me for being ‘a chicken’ and wished me ‘all power’ in fighting for the right not to exacerbate the fraud by adding one more dubiously gathered and shoddily processed set of statistics to an already flawed information-gathering exercise.

There are many colleagues, such as the professor just mentioned, not to mention newly appointed colleagues, colleagues with young children, colleagues in financial straits, etc. who are unable, through fear of managerial reprisal, to take a stand against the type of bureaucratic game typified by the TR. However, acquiescence on the part of those of us able to fight the absurdity of such exercises as the TR is, I feel, ethically unacceptable. It would be dishonest of me to contribute to so dishonest an exercise. Am I seriously being instructed to take part in something that is both methodologically flawed and statistically fraudulent?

Postscript

Yes, I was being ordered to contribute to a methodologically flawed and statistically fraudulent procedure, but not in those terms, of course. I was called to a meeting with a very senior academic-cum-manager who issued no explicit orders, but who by coercion and implication made it abundantly clear that it was presumptuous and insubordinate of me not to comply. I was not allowed to laugh at the emperor's new clothes. This saga continues in the first part of my Open Letter to the AUT concerning the right of members to disobey, on grounds of conscience, managerial edicts to submit to audit after audit.

Endnotes (letter)

1. Higher Education Funding Council of England.


Conscientious Objections to Audit

Background document to proposal submitted to the Association of University Teachers (AUT), February 2002,
by Philip Tagg

v.2, Montréal, March 2006

• Summary  • Personal experience   • Critiques of audit  
Perversion of principles  (fairness, objectivity, prudence
| quality, assurance, transparency, accountabilty
| innovation, cynicism and hypocrisy)  
Conclusions  • References  • Endnotes  • Glossary

Summary

After a brief personal account of encounters with ‘audit culture’ I present a summary of some critical views of the phenomenon. These views are provided by highly respected authorities in the humanities and social sciences, as well as in accountancy, business studies, management and economics. I then confront audit’s semantic hijacking and falsification of basic principles such as fairness, objectivity, prudence, accountability and transparency, explaining how audit also perverts the course of research and impedes innovation. A short account of how audit causes cynicism and hypopcrisy leads to the conclusion that collective opposition to rigid neo-managerialist accountability is a prerequisite for the improvement of learning, teaching, research and management in UK universities.

Recent personal experience

In November 2001 I sent a letter to my head of department, explaining why I would not take part in the ‘transparency’ review imposed on us by management. I gave three main reasons.

  1. The exercise is methodologically flawed: if a second-year student had conducted a survey using categories as nebulous as those of the ‘transparency’ review, I would have asked him/her to resubmit.
  2. Incorrectly categorised data is either misleading or unreliable. If such data is used to inform managerial decisions, those decisions will be flawed and have adverse effects on our work.
  3. It is one of my duties to encourage good intellectual practice. If I take part in activities which do not constitute such practice, I open myself to accusations of hypocrisy. I do not think any university teacher should be forced to act hypocritically.

The reaction to my letter was to summon me to a meeting with my head of department and a university pro-vice-chancellor. The latter suggested that my views were elitist, implying that it was presumptuous to refuse to submit to audits of ‘accountability’. I assured the pro-vice-chancellor that I had (and have) no objections whatsoever to being held responsible for what I do in my job, provided that what I am regarded as responsible for, to whom I am held accountable, and for what ends any resulting ‘account’ is used, do not fundamentally conflict with the efforts I have made throughout my career to improve education and research in my area of competence.[1] I also tried to point out that I failed to see how standing up for good intellectual practice could be construed as incompatible with my responsibilities as a university teacher. Having assured myself by the end of the meeting that my objections would be passed on, I agreed, by way of compromise, to submit my ‘transparency’ review. I kept my part of the bargain but never learnt of any reciprocal action on the part of management.

Several colleagues told me that the ‘transparency’ review was not really that important, that it was itself under review, that I only needed spend a few minutes filling it in, and that it was ‘not worth making a fuss about’. These comments still puzzle me because, if the review was such an insignificant bureaucratic chore, why had it then been necessary to convene a meeting of three senior members of academic staff so urgently? Why had such pressure been exerted on me to comply? I can only conclude that the formality of compliance, i.e. being seen to obey managerial edicts, was considered more important than trying to improve education and research in my field of studies, more important than doing my real job, more important than implementing the good intellectual practice which, I feel, the ‘transparency’ review contradicts.

After that meeting I received the following four items of audit-related news: [1] the Institute of Popular Music (where I worked) was shortly to be subjected to an IPR exercise; [2] the external assessor in that review committee was someone without experience of our subject area; [3] I was instructed to submit yet another self-audit (a ‘portfolio of activities’) whose purpose is unclear; [4] the most recent (2002) Music RAE committee awarded top marks (5*) to no other departments than those whose representatives constituted the committee awarding those marks! Two more issues troubled me: [5] I was depressed and felt brow-beaten after the meeting which had included an element of disciplinary coercion that I found unwarranted; [6] I had on reflection begun to interpret the pro-vice-chancellor’s notion of ‘accountability’ as disingenuous. I was therefore determined to find out what lies beneath the confusion and coercion of the audit zealots. After the meeting I discussed ‘audititis’ with many colleagues, all of whom have encouraged me to persevere in my opposition to audit obsession. I even started studying literature on ‘audit culture’. What I discovered was both disturbing and illuminating.

Critiques of audit

The first lesson I learnt from literature about audit in the UK is that I am far from alone in expressing strong objections to the audit culture of the ‘new managerialism’ imposed on our universities. On the contrary, focus groups in an independent study funded by the Economic and Social Research Council (grant no. R000237661) were reported as agreeing that ‘the UK higher education system [is] now highly managerial and bureaucratic, with declining trust and discretion. Higher workloads and long hours, finance-driven decisions, remote senior management teams and greater pressure for internal and external accountability’ were mentioned as prime causes for concern (UUK 2001). The report concludes that ‘university staff not occupying management roles’ felt that universities were ‘awash with managerialism’ and that ‘New Managerialism’ was ‘rife’. If, as the report states, the majority of us are so strongly opposed to audit culture, it makes no sense to treat individuals who take a stand against it as isolated or marginalised trouble-makers who need to be herded like wayward sheep into the neo-managerialist fold.

If it is the duty of our trade union to represent the interests of the majority of its members, then the majority’s demonstrable opposition to audit should be part of union policy. The fact that it is not is both illogical and unethical. It also means that, if we are serious about our work as educators and researchers under current conditions, we have to resort to individual measures to try and counteract the most adverse effects of audit. For example, staff at one UK department of English have found it necessary to set up an English Subject Centre because:

‘We live in a culture that appears to be obsessed with audit. The RAE, QAA assessment, continuation audit, institutional reviews’... ‘mean that we are constantly having to defend what we do and how we do it. We drown in the paperwork that this causes and block the system with frenzied email attachments. Our natural eloquence becomes polluted with the rhetoric of officialese, and the filing cabinet fills to overflowing’... (Simons 2001).

One of the English Subject Centre’s main aims is to:

‘re-focus attention on the central business of educating our students’ and to ‘return academics to the genuine excitement of teaching, one of the reasons why most of us went into this business in the first place’ (ibid.).

It cannot be right that colleagues have to take additional initiatives, involving additional work, just to ensure that the prime focus of our activities is catered for. However, such additional work is clearly necessary under the current neo-managerialist regime. University teachers have to either create alternatives outside the current system, as above, or to spend valuable time in efforts to expose the destructiveness of audit, all with a view to creating a more constructive and productive work environment. For example, I should be getting on with my research, not writing a document criticising what ought not to exist in the first place. I bet colleagues like Gombrich, Strathern and McIntyre, who I refer to next, would all say more or less the same thing.

Richard Gombrich, Professor of Oriental Languages at Oxford, is particularly harsh in his criticism of university audits. He argues that higher education policy in the UK over the last twenty years has been an ‘unmitigated catastrophe’, constituting ‘the murder of a profession’ (Gombrich 2000). Drawing on official statistics and enumerating hair-raising examples of the ‘sadistic party game’ in which all sorts of performance indicators are manipulated to the short-term economic advantage of the institution under review (op. cit., app. 3), he presents convincing evidence that audits pervert and destroy what they are proclaimed to promote. One of Gombrich’s conclusions is that UK university standards of education and research have been severely damaged by the imposition of audit culture.

Marilyn Strathern (FBA, OBE), Professor of Social Anthropology at Cambridge,[2] is less polemical in her critique. She highlights fallacies of audit culture, including the notion that it is somehow possible to evaluate the results of social processes, such as education and research, without having to deal with the processes themselves (Strathern 2000a). While the outcome of social activities, she explains, is in reality never certain with regard to either form, content or time of ‘delivery’, the form in which audit-culture outcome has to be described ‘is known in advance’. It is in other words retroactive, a virtual construction which ‘works backwards [.. .] from the categories by which accountability (say) can be ascertained to the evidence for it’ instead of vice versa. ‘Audit produces its own effect’, Strathern continues, ‘insofar as the report on outcome/output takes a form it itself creates.’ Another of her conclusions is that audit avoids real issues of professional accomplishment — for example, the reputation of publications over decades, the success of former students in their thirties or forties (real outcome) — ‘by bypassing descriptive observation, or rather by restricting the output (results) of observation to data suitable for constructing indicators (virtual outcome).’ Such mechanisms are of course easily manipulated in the sort of way Gombrich (2000) characterises as ‘sadistic party games’.[3] Another of Strathern’s conclusions is expressed in the maxim ‘when a measure becomes a target, it ceases to be a good measure’ (Strathern 2000a).[4]

Strathern’s maxim is interpreted by Michael McIntyre, Professor of Applied Mathematics and Theoretical Physics at Cambridge (McIntyre 2001a, b), as a short version of Goodhart’s Law,[5] whose two main tenets state:

‘As soon as the government attempts to regulate any particular set of financial assets, these become unreliable as indicators of economic trends’... [because] ‘financial institutions can ... easily devise new types of financial assets.’

‘Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.’

It is worth noting that Professor Charles Goodhart (FBA) was for several years Chief Adviser to the Bank of England. His ‘law’ testifies to the fact that criticism of unidimensional (linear/bipolar/numerical/quantifiable) auditing is just as strong in the world of finance as in academe. This observation is adequately substantiated in statements made by business experts taking part in the BBC Radio 4 programme ‘Rough Counting’ (Dilnot 1999). These experts include:

  • Peter Miller, Professor of Accounting, London School of Economics;
  • Colette Bowe, Executive Chairperson of Save and Prosper, and former head of the Personal Investment Authority;
  • Clare Spottiswoode, Member of the Management Team of PA Consulting, London, and former Regulator of the UK Gas Industry;
  • Teresa Graham, Partner in Accountancy at Baker and Tilly, London, and member of the UK Better Regulation Task Force.
  • John Kay (FBA), Professor of Economics at Oxford, formerly Director of the Said Business School, founding Executive Director of London Economics, and Director of Halifax plc.

One recurrent point made by these experts is that good business practice depends largely on relatively informal arrangements because they are much more in tune with the highly complex realities of social and economic relations at the basis of doing business. In the same programme, John Kay (FBA), qualified by the ‘Speakers for Business’ website as ‘the UK’s leading thinker on business and economics’,[6] exemplifies this widely held view as follows.

‘General Motors employs ten times as many people as Toyota, although it produces very few more cars. Now that is partly because Toyota is more productive than General Motors but it isn’t that more productive, because the way Toyota functions is as essentially the centre of a network of supplying firms with whom it has long-term but relatively informal relationships: it depends on trust to sustain these relationships. In turn, these relationships give [Toyota] an openness with information and a flexibility of response which the General Motors structure [lacks]. General Motors felt it had to own and control and monitor everything that was important to the production of its cars. [They] actually achieved less flexibility, less speed of response and less core reliability in what went into the manufacture of its automobiles. The fact that people operate by reference to whole set of tacit rules that aren’t written down, couldn’t be written down, actually gives these kinds of economic structures real competitive advantages in the modern market place. And actually, the most important things that enforce contracts in business are not actually legal contracts, they are the needs and expectations that people have so that they will go on doing business with each other’ (Dilnot 1999).[7]

In other words, many highly respected, successful and experienced members of the business and management community — which, ironically, universities are supposed to emulate — advocate the importance of informal arrangements, of trust, and the necessity of interacting with the real world, with all its complexity of loose ends, unpredictability, multidimensional parameters, processes, organisations, systems and scenarios. Even the brave new world of marketing now encourages the use of ‘multi-attribute utilities’ and ‘multi-attribute attitude modelling’ rather than the old-style collection of unilinear, easily quantifiable numerical data.[8] In fact, the emergence of Complexity Studies (e.g. Elliott and Kiel 1997; Hoskin 1996; Morel and Ramanujam 1999), which attempt to confront the reality of all those ‘loose ends’, and which address that reality in the management of public institutions (e.g. Blackman 2001), seems to have resulted from the realisation that rigid (‘robust’) quantifiability is counterproductive when assessing the success of procedures in the sphere of economics, business and management.[9]

So far it should be clear that:

  • The vast majority of university teachers strongly oppose audit culture and neo-managerialism. I am just one of those many teachers.
  • Opposition to audit culture is also voiced by distinguished scholars and by figures at the cutting edge of accountancy and business economics.
  • University management lags far behind leaders in the business world when it comes to understanding the inefficiency and destructive power of overregulation by audit.

Why, then, does audit still prevail?

Audit: the perversion of basic principles

Fairness, objectivity and prudence

The three principles behind the zeal to audit everything are, according to McIntyre (2001a), fairness, objectivity and prudence. On the face of it, nothing could be more laudable. The problem is that audit perverts each of these principles.

Fairness is taken to mean that everyone must be treated in the same way, with the result that the noble principle of equality between humans is replaced by the aberration that humans are all the same. Differences in personality, expertise and modus operandi are essential to any good team or organisation, and meaningful assessment of the individual’s or organisation’s real value must consider each individual or organisation differently. Millions of years of life on earth show that survival depends on diversity.

Audit culture’s notion of objectivity rests on the erroneous assumption that numerical measures constitute the only reliable means of assessing anything — ‘what can be counted counts’ and nothing else, so to speak. The SPR, the RAE, league tables, etc. all exhibit an almost metaphysical belief in the quantifiability of everything. ‘Unmeasurables — such as professional ideals and ethics, ambition to do one’s best, eagerness to understand something in depth, willingness to share information freely, etc.,’ writes McIntyre (2001a), — ‘can’t be counted and [therefore] can’t be rewarded...’ Nor can interest, curiosity, enthusiasm, the ability to cooperate, intellectual generosity, artistic independence, originality, innovation and many other unquantifiable qualities essential to good education and research.

According to normal parlance, prudence denotes qualities of caution, circumspection and careful judgement. In audit culture, however, prudence is restricted to the almost paranoid notion that no-one can be trusted or, rather, that they can only be trusted if audited. Self-auditing may have become more common in recent years but it is still clear that we cannot be trusted to audit ourselves because our self-audits are audited by others. Prudence in the sense of circumspection and caution vis-à-vis the dubious practices of neo-managerialist auditing is of course not on the audit agenda. However, if we are not to be trusted, why should we trust our auditors? The recent demise of Enron and its accountants, the even more recent discovery of large-scale embezzlement at the Allied Irish Bank, not to mention, on a smaller scale, the self-aggrandising results of the most recent RAE for Music (see p. 1), all provide incontrovertible evidence that auditors need auditing and that audit procedures are both unreliable and open to severe abuse. Indeed, audit’s prudence principle, if applied systematically (see Fairness, above), would demand that auditors be audited by auditors who should be audited by more auditors, and so on, in an endless chain of audits and meta-audits. It is clear that audit culture’s notion of prudence is not only restricted and misleading but also, if applied properly, never-ending and, consequently, an inordinate waste of time, money and energy.

Quality assurance, transparency and accountability

Other recurrent buzzwords in audit culture are ‘quality assurance’, ‘transparency’ and ‘accountability’. The problem with these three phrases d’honneur is similar to that with audit notions of fairness, objectivity and prudence: the real sense of the words is hijacked, restricted and perverted.

The main problem with the quality of ‘quality assurance’ is obvious: audit can only express quantity because the only acceptable measurements of ‘outcome’ have to be countable (‘only what is counted counts’ again), i.e. expressed as quantity so that audited bodies can be ranked in league tables.[10] Consequently, the assurance of ‘quality assurance’ is also invalid since the ‘assurance’ is expressed as a quantity, not as quality, and because a quantity is by definition incapable of expressing any realistic notion of one single quality, let alone a combination of qualities. It is surely the latter which any ‘assurance’ would have to refer to in determining the success of any educational enterprise.

Audit also obfuscates transparency and accountability, as well as assurance, in two other ways. The obfuscations are revealed when you try to answer the following simple questions.

  • To whom should our actions be made transparent?
  • To whom are we accountable in our work?
  • Who needs assuring that we are doing the right thing?

Every educator I know would answer that we are primarily responsible to our students, indirectly also to their parents and to the taxpayer, since it is they who pay for the education we provide. It is in the obvious interests of all three parties that we prepare students to contribute meaningfully to our society and culture in the future. Since none of us is able to predict the future, we have to rely on accumulated experience, on extrapolation and on other forms of intelligent guesswork to prepare students for life in a world whose pace of change in cultural, economic and technological terms can hardly be expected to slow down, let alone remain locked in the immediate past, which is exactly what the audit model and its intrinsic retroactivity seems to expect by equating targets with measurements. The first obfuscation is therefore that audit irrationally expects the expected — the already known — for our students’ future because, as Strathern demonstrates, audit’s outcome (the ‘targets’ to be met) must be both established and known in advance. Such mechanisms are of little value to students facing a future whose only certainty is that it cannot be established or known in advance. We can only work with probabilities within that uncertainty, not with rigidly regulated unidimensional quantifications.

The second obfuscation concerns audit management’s hijacking of the primary relationship between teacher and student, a relationship as old, and as important for society, as that between doctor and patient. Despite increasing neo-managerialist bureaucracy in the medical profession, the most important external intervention regarding transparency and accountability between doctors and patients is reserved for cases of severe malpractice. Similarly, there would be a public outcry if marriage, another age-old social institution, were subjected to the rigid auditing with which we university teachers are expected to comply. Of course, like doctors or airline pilots, we teachers need control mechanisms to minimise breach of trust, but the vast majority of us joined the profession because we are committed to making the world a better place for coming generations, or because we are simply dedicated to a particular field of learning. Consequently, questions of transparency, accountability and assurance in student-teacher relationships should, as a general rule, be a matter for open negotiation between the two parties involved: external arbitration should only be introduced when the relationship suffers a breakdown. In addition, general guidelines can be very useful, for example, clear channels through which students and teachers can express opinions or complaints about their work together with a view to improvement and renewal, ways in which unresolved conflicts can be arbitrated externally, etc. Audit, however, does not work in that way. Under the disingenuous pretence of mediating aspects of transparency and accountability between teacher and student, audit acts as a parasite on that time-honoured primary relationship, so that the real transparency and accountability that should, and usually does, exist between the two parties is siphoned off to provide fuel for arguments about the ‘need’ for neo-managerialist ‘quality control’, i.e. for mechanisms which, as explained above, are both fundamentally flawed and detrimental to the primary aims of education and research.

One final argument that is often put forward on behalf of audit is that its tight network of ‘audit trails’ will ‘catch the bad apples’. A recurrent buzz-adjective in such argumentation is ‘robust’. The last time I heard it used seriously in public was in early February (2002) when the director of the Allied Irish Bank was interviewed on Channel 4 News about the bank’s failure to discover the disappearance of billions of its dollars through embezzlement by one of its middle-management employees. According to the director, nothing was wrong with the bank’s auditing because, as he put it, ‘we have robust mechanisms in place’ to root out that sort of thing. Pressed further by the interviewer, the director explained that, unfortunately, some employees knew the mechanisms inside out and how to use them to their own advantage. The Enron saga illustrates similarly ‘robust’ accountability mechanisms in an even more dramatic fashion (Chatterjee 2001). The point here is that the more regulations there are, the easier it is to cheat the system because each regulation provides yet another cover behind which the clever ‘bad apple’ who ‘knows the rules of the game’ can hide and manipulate. In a more open system based on trust, on the other hand, the peer pressure of not wanting to ‘let the side down’ has, in my experience, been far more effective a deterrent than rigid prescriptive regulation.

Audit: anti-innovative, cynical and hypocritical

I have left the three most serious objections until last: that audit impedes innovation, perverts research and fosters professional hypocrisy.

Audit obstructs innovation because its targets have to be known in advance (Strathern 2000a). Innovation cannot be audited because it involves, by definition, unknowns, experimentation and the development of previously untried (combinations of) materials, methods or ideas. Audit offers no reward to those who strive for the innovation necessary to keep abreast or in front of social, cultural, technological, intellectual or ideological change; it does, however, provide countless incentives for those who meet prescribed targets based on what is already known and established. To suggest that current research assessment exercises can redress the balance and meaningfully assess the value of innovation in our work as academics is absurd. Let me give two examples from my own experience.

In 1981 there was no official incentive for me to co-found the International Association for the Study of Popular Music (IASPM), just a lot of encouragement from institutionally isolated colleagues.[11] Where previously no academic forum for exchange of information and ideas in our subject area existed, we now have an association organising around 600 scholars in some 40 nations. Today (2002), holding office in IASPM even qualifies for official UK audit brownie points, but setting it up did not and I lost both time and money in the process! The same goes for the Mass Media Music Scholars’ Press, a not-for-profit corporation which I set up in 2000 and whose aim is to facilitate the publication of substantial musicological work about music in the modern media.[12] I am sure that, in a few years, MMMSP publications will earn official research audit brownie points; but I am even more certain that I will have lost a lot of time, money and brownie point opportunities by creating something whose main purpose was to disseminate a particular type of knowledge, but which auditeers can later hijack when they feel compelled to quantify ‘research publication output’ in my field of studies. True, I may personally lose out in monetary and short-term careerist prospects, but I gain something much more valuable by pursuing the unauditable: friendship, collegiality, solidarity, openness, honesty, intellectual generosity, a rich exchange of ideas, even some respect, gratitude, encouragement and trust. All these qualities are essential to good practice in scholarship but way outside the orbit of any audit or research assessment exercise. The same qualities also rhyme well with the trust and ‘informal arrangements’ highlighted by John Kay in relation to successful business practice (see above).

Audit perverts research in other ways. For example, it is obvious to the point of tautology that a short, popular textbook drawing on your own research must by definition draw on research that you have actually completed, otherwise you would be disseminating information whose reliability was questionable. However, that is exactly what the RAE forces scholars to do. Instead of spending five years or more completing your research, your institution presses you to produce a monograph or two to fill the quota for the highest possible rating within the audit’s predetermined time frame. Thorough, long-term research suffers in this way, as do those who have to read monographs drawing on such uncompleted research.

Perhaps the most serious accusation to level against audit is that it fosters a spirit of self-deprecatory cynicism and hypocrisy. Most colleagues find audit exercises ludicrous but nevertheless go through the motions of complying with their imperatives lest the wrath of management be incurred. We see straight through the emperor’s clothes, so to speak, but feel obliged to stifle our laughter and to hide our sense of ridicule when the emperor’s minions approach us ‘donkeys’ with their managerial sticks and carrots. Repeated viewing of the emperor in his see-through clothing — transparency in the true sense of the word — leads to understandable cynicism towards the system that forces us to carry out the ridiculous chores of audit. A recurrent quip from colleagues is that we’re forced, like circus animals, ‘to jump through hoops’. The more circus tricks we perform, the more we demean ourselves. After all, the whole idea of forcing someone to carry out a pointless task is to demean that person.[13] Moreover, audit gives the virtual and formal pride of place, neglecting the real content or effects of what we do in our job. It’s like having to recite seven-times-seven Ave Marias instead of giving alms to the poor or carrying out other charitable works.

Most heads of department with whom I have spoken are in private just as critical of audit as the rest of us. However, since they are responsible for the welfare of those under them as well as directly accountable to management, they often consider it wisest to be seen (by management) to be taking the audit seriously, even if they think it is ludicrous, destructive and an utter waste of time and money. Adopting the posture of compliance as a strategy is confusing and alienating for those of us who look to our seniors for a lead in pursuing the true vocational aims of our profession, but the strategy is more alienating for the head of department who is forced to act hypocritically, often knowingly so, in order to secure what he/she may regard as the lesser of two evils and to keep management off their backs. It is tragic that so many talented scholars in the UK have to waste so much time on audits and reviews, time they could have spent on the work for which they are renowned. It is heart-breaking to see respected colleagues forced to advocate something they oppose, just to appease the audit monster, all in the vain hope that acquiescence will somehow make our lives easier.[14]

My final objection to audit is that it flies in the face of almost everything I believe in as a teacher and researcher. For thirty-five years I have tried to prepare my students for the realities they will have to face when they complete their studies. Those who do well after university are mostly those who can ‘think on their feet’ and make intelligent decisions in their field of work. To help them succeed, I have to encourage all sorts of artistic and intellectual faculties which audit can never measure: critical thinking, independent thinking, lateral thinking, rational deduction, intellectual and artistic integrity, honesty and openness, etc. If I am serious about encouraging these qualities in my students, then I ought to practise what I preach. It would be unreasonable to expect respect from students and colleagues if I participate in actions diametrically opposed to the ideals of education and human respect for which I stand. I have lived by these well-tried principles for many years and am not prepared to abandon them.

Conclusions

It should be clear from this document that:

  • the overwhelming majority of AUT members object strongly to the imposition of neo-managerialist audits of our activities;
  • audit is demonstrably flawed and totally inadequate as a means of assessing the value of university teaching and research;
  • audit perverts the meaning of words denoting laudable principles of human behaviour;
  • audit prevents innovation, perverts research and fosters dishonesty, cynicism and hypocrisy.

It is for all the reasons presented in this document that I find it impossible to participate with a clear conscience in university audits. The only problem is that the AUT does not appear to support us if we should, on grounds of conscience, disobey orders to comply with audit instructions. If that is the case, I think the Union’s policy should change. If a prime function of the AUT is to look after its members’ interests then it would seem more than appropriate for the Union to offer strong support to any member who, on account of conscientious objection, refuses to carry out management’s auditing orders.

Through my local AUT representative I will be submitting a motion to the next conference to ensure that we are properly protected if we harbour conscientious objections to audit.[15] My hope is that a strong stance on this issue wil raise awareness and the willingness to combat the scourge of audititis, so that teaching and research can once again become the real focus of our endeavours. Inertia and acquiescence will, in my opinion, only aggravate what is already a very serious problem.


References


Endnotes

1. Much of my professional life has been devoted to the reform and improvement of music education and research at university level. See CV at tagg.org/ptcv.html. See also (by P Tagg): [1] (1966) ‘Popular music as a possible medium in secondary school education’; dissertation for Certificate in Education, University of Manchester [tagg.org/articles/mcr1966.html]. [2] (1982) ‘Music Teacher Training Problems and Popular Music Research’, pape r delivered at 1st International Conference on Popular Music Studies, Amsterdam, 1981; in D Horn and P Tagg (eds): Popular Music Perspectives, 1. Göteborg and Exeter: IASPM: 232-242. [tagg.org/articles/pmpsamus.html]. [3] (1983) ‘Why IASPM ? Which Tasks?’, paper delivered at 2nd IASPM International Conference, Reggio Emilia. In D Horn (ed): Popular Music Perspectives, 2. Göteborg, Exeter (1985): IASPM: 501-507 [tagg.org/articles/iasptask83.html]. [4] (1985) ‘Address on the State of the Association’, paper delivered at 3rd IASPM International Conference, Montréal [tagg.org/articles/montreal.html]. [5] (1987) Report from fourth IASPM International Conference, Accra (Ghana) [tagg.org/articles/ghanarpt .html]. [6] (1995) ‘Analysing music in the media. An epistemological mess’, keynote speech delivered at 8th IASPM International Conference, Glasgow. In T Hautamäki and H. Järviluoma (eds): Music on Show: Issues of Performance ,. Tampere (1998): Department of Folk Tradition: 319-329 [tagg.org/articles/glasg95.html]. [7] (1997) Closing speech, 9th IASPM International Conference, Kanazawa, Japan [tagg.org/articles/kanazawa.html]. [8] (1998) ‘The Göteborg conne ction: lessons in the history and politics of popular music education and research’, in Popular Music, 17/2: 219-242 [tagg.org/articles/gbgcnnct.html]. [9] (1999) ‘Popular Music Studies - bridge or barrier?’ Paper delivered at music education conference, Bolzano. In G Tonini (ed): Musica come ponte tra i popoli / Die Musik als Brücke zwischen den Völkern (2001): Libreria Musicale Italiana: 128-142 [tagg.org/articles/bolz9811.html]. [10] (2001a) ‘High and Low, Co ol and Uncool, Music and Knowledge — Conceptual falsifications and the institutionalisation of popular music studies’, keynote speech, IASPM UK Conference, Guildford [tagg.org/articles/iaspmuk2000.html]; shorter version published in Bulgarian Musicology, 2/2001: 9-18 [tagg.org/articles/sofia2000.html]. [11] (2001b) ‘Twenty Years After’, keynote speech, Founder’s Event, 11th International IASPM Conference, Turku, July 2001 [tagg.org/articles/turku2001.html].

2. Strathern was awarded her OBE for services to Social Anthropology. She was also trustee of the Museums and Galleries on Merseyside.

3. See also comments on the illusion of ‘robust mechanisms’.

4. The effects of audititis on local government have been devastating. According to Tsoukas (1994: 4, cited in Strathern, 2000a), new regulations in 1993 meant that local authorities in the UK had to publish no fewer than 152 indicators of output. ‘The idea was, he reports, to make councils’ performance transparent and thus to give them an incentive to improve their services. So even though elderly people might want a deep freeze and microwave rather than food delivered by hom e helps, if the number of home helps is the indicator for helping the elderly with their meals then an authority could only improve its recognised performance of help by providing the elderly with the very service they wanted less of, namely more home he lps. Local authorities would be aiming for high scores, and the language of indicators would take over the language of service.’

5. C Goodhart: Monetary Theory and Practice, p.96; quoted in McIntyre (2001b, citing Pears Cyclopedia, 1990-1, pp. G 27, G 31). McIntyre (loc. cit.) also reminds us that ‘Goodhart’s law is a sociological analogue of Heisenberg’s uncertainty principle in quantum mechanics. Measuring a system usually disturbs it. The more precise the measurement, and the shorter its timescale, the greater the energy of the disturbance and the greater the unpredictability of the outcome’.

6. http://www.sfb.co.uk/speakers/johnkay/ (visited 2002-02-19). The site offers the following CV: ‘John Kay began his academic career when he was elected a fellow of St John’s College, Oxford at the age of 21, a position which he still holds. As research director and director of the Institute for Fiscal Studies he established it as one of Britain’s most respected think tanks. Since then he has been a professor at the London Business School and the University of Oxford. He was the first director of Oxford University’s Said Business School. In 1986 he founded London Economics, a consulting business, of which he was executive chairman until 1996. During this period it grew into Britain’s largest independent eco nomic consultancy with a turnover of £10m and offices in London, Boston and Melbourne. He has been a director of Halifax plc and remains a director of several investment companies. He is the first (and still the only) Professor of Management to rece ive the academic distinction of Fellowship of the British Academy. In 1999 he resigned his position at Oxford and sold his interest in London Economics.’ The site also informs visitors that his average speaking price is £5,001 - 7,500. I can only assume that some businesses must find his ideas worth that sort of money!

7. Kay was then asked by the programme leader: ‘So does that mean that in the extreme, the absurd extreme, a contract that is expected to persist indefinitely doesn’t really need to have very much in it at all, because if it is going to go on forever, then you just jolly well have to work it out?’. Kay replied: ‘Yes, I mean the best example of that we all know would be something like a marriage contract, where you don’t need to write it down. The point at which you actually have to refer to the rules and the contract, then the relationship is no longer one which is worth, is going to profitable for either party to sustain.‘

8. Thanks to my brother, Dr Stephen Tagg, Lecturer in Marketing at the University of Strathclyde (Glasgow), for this observation. For further information, see the countless marketing theory websites listed when asking your search engine to find documents relating to the concepts ‘multi-attribute’ and ‘marketing’.

9. From colleagues in Human Geography I understand that Complexity Studies originally involved in the Business School of the University of Santa Fe. The inadequacy of audit culture in the business world is exemplified most clearly by John Kay (see the General Motors v. Toyota example, above). s

10. The analogy with games is significant. Football league tables rank teams primarily according to their number of wins, draws and defeats, secondarily on goal scoring average. The number of goals is in other words an arbitrary measurement of each team’s position in the league tables, whether or not the team plays attractive football or more defensively. Unlike education or health care, football is a game, and games are closed systems with internal rules that are agreed on by all wh o participate. Education and research are not games: they are real, open-ended practices whose general aims and procedures must be agreed upon, but whose internal rules must undergo almost constant change if the activities are to remain meaningful.

11. See www.iaspm.net for details of this organisation. For discussion of IASPM policies, see footnote 1.

12. See www.mmmsp.com for further information, including the MMMSP’s background and aims.

13. The soldier who steps out of line may be ordered to either clean out the latrines (a demeaning but not pointless task) or to dig a hole and fill it in again ( a demeaning task because it is pointless). In either case, being demeaned is the main dynamic of the punishment. The only ‘offence’ we are indicted of and then sentenced for in advance is that, as university teachers, we are on principle not to be trusted. Of course, no evidence is presented to substantiate such a ludicrous and prejudicial procedure.

14. One old friend and colleague, currently head of department at an old university, apologised to me for being a chicken, in the sense that he believed he ought to oppose the imposition of audits on his department but was afraid to do so (personal phone conversation, November 2001). I am, for obvious reasons, unable to reveal the identity of that head of department.

15. If conscientious objection to participation in military action is still regarded as a human right in the UK, it ought to be possible for academics to refuse to participate in anti-educational and anti-intellectual acts of auditing. Like conscientious objectors to military warfare, who are expeted to do menial work in the fire service, hospitals, etc., conscientious objectors to audit destructiveness could be assigned appropriately menial non-audit-related tasks, for example photocopying, running errands, doing archival work, etc., to compensate for the time we would otherwise have to waste on audits.


Glossary for non-UK university staff

The ‘Transparency Review’ is a crude time-and-motion study form UK university teachers must fill in every semester stating what percentage of their time is devoted to which category of activity, the categories being nebulous and totally incapable of reflecting what we actually do. The usual ruse is to fill in figures which you think management will interpret in the least negative way when apportioning funds. As long as the figures add up to 100%, no-one seems to care. What the 100% represents in real terms is also apparently irrelevant: 40 or 140 hours a week, it’s all the same! There’s no point in working better or more, just as long as you fill in the form!

Pro-vice-chancellor: senior member of academic staff, usually a professor, with management duties below those of the Vice-Chancellor (Rektor).

The IPR, or Internal Periodic Review, basically checks that department under review can produce the desired paper trail of bureaucratic procedures laid down by management and that members of staff can talk about those procedures in the desired manner, preferably using the magic vocabulary of audit officalese.

The RAE, or Research Assessment Exercise, is one of the UK university world’s most notorious audit procedures. A committee of experts in a particular subject area (e.g. Music) is appointed nationally to apportion marks on a scale from 1 (low) to 5* (high) for the quality of research at every UK university department in that same subject area. The crude ratings they give then determine how much funding the UK government gives each university for that subject over a period of several years. The committee’s experts are themselves academic staff at UK universities and have a vested interest in acquiring funds for their own departments. RAE committees are not themselves subject to any audit. Can you believe it?!

The QAA is the UK’s ‘Quality Assurance Agency’, whose main aim is, according to its own propaganda, ‘to promote public confidence that quality of provision and standards of awards in higher education are being safeguarded and enhanced.’ ‘Enhance’ — my arse! Whatever happened to ‘improve’? A visit to the QAA website guarantees hours of trendily pompous officialese and intellectually depressing reading.