Promotion and Tenure Guidelines Addendum: Rationale for Redefined Criteria
New Criteria for New Media
Version 2.2, January 2007
Authors: Joline Blais, Jon Ippolito, and Owen Smith in collaboration with Steve Evans and Nate Stormer.
ABSTRACT: An argument for redefining promotion and tenure criteria for faculty in new media departments of today's universities.
Introduction
Recognition and achievement in the field of new media must be measured by standards as high as but different from those in established artistic or scientific disciplines. As the reports from the American Council of Learned Societies[1], the Modern Language Association[2], and the University of Maine[3] recommend, promotion and tenure guidelines must be revised to encourage the creative and innovative use of technology if universities are to remain competitive in the 21st century.
The following points summarize some of the key areas in which new media research departs from traditional academic scholarship, with the aim of providing a rationale for specific criteria for promotion and tenure detailed elsewhere.
New form and content
The differences between traditional and new media excellence lie in both form and content. The hard-copy format of traditional review documentation, such as photocopies or slides, is insufficient for evaluating new media work; screenshots do little justice to electronic projects based on innovative interactive or participatory design. As the MLA puts it, "evaluative bodies should review faculty members' work in the medium in which it was produced. For example, Web-based projects should be viewed online, not in printed form."[4]
Further complicating the evaluation of new media achievements is the fact that they are often interdisciplinary, as reflected by the current University of Maine New Media faculty, whose backgrounds range from engineering to computer science to fine art to photojournalism to literature.
For example, while art professors typically divide clearly into critical (Art History) and creative (Studio Art) faculties, new media's brief history often requires its practitioners to develop a critical context for their own creative work. This is why the majority of pre-eminent new media critics are also artists.[5] It is also why new media research spans numerous genres, from critical essays to political activism to community-building to software design.
Limitations of academic journals
These differences may require evaluators of new media artist-researchers to look beyond the usual standards applicable in other disciplines. As noted by a 2003 National Academies report:
Because the field of [Information Technology and Creative Practices] is young and dynamic, ITCP production is hard to evaluate. Traditional review panels...may be hampered by their members' ties to single disciplines and the absence of a time-tested consensus about what constitutes good work in ITCP and why. [6]
Ironically, the National Academies study found that the highest benchmark for success in traditional academic departments, publication in peer-reviewed journals, is less relevant to success in new media--and empirically less an accurate measure of stature in the field--than more supple or timely forms of intellectual exposition:
The gold standard for academia--and the criterion most easily understood by parties outside a given subdiscipline--is the so-called archival journal (often published by scholarly or professional societies) that involves considerable editorial selection plus prepublication review and revision, which function as a screening system for quality. But the long lead time for such publications poses problems for subdisciplines in which timeliness--quickly getting an idea into the field--matters.[7]
Leonardo magazine (MIT Press) is currently the only print magazine universally recognized as a peer-reviewed journal about new media. There are currently a handful of networked peer-reviewed journals devoted to new media, such as Leonardo's Electronic Almanac (Cambridge), Fibreculture (Sydney), and First Monday (Chicago).
Yet the field's most prominent print publishers and research archivists[8] have acknowledged a 15-25 year lag and limited exposure that makes print publications far less relevant for new media research. Although promising new paradigms for distributed publication are on the horizon, at the time of writing these systems are only in the planning stage.[9] Finally, as the MLA warns, participation in electronic scholarship should not place extra demands on a researcher[10]; an accomplishment in new media research should substitute for a print article or monograph, not merely supplement them.
Alternative Recognition Measures
Given the accessibility and timeliness required for new media research, the following measures of recognition should be prioritized in the evaluation of new media research candidates:
1. Invited / edited publications
Invitations to publish in edited electronic journals or printed magazines and books should be recognized as the kind of peer influence that in other fields would be signaled by acceptance in peer-reviewed journals.
2. Live conferences
The 2003 National Academies study concludes that conferences on new media, both face-to-face and virtual, offer a more useful and in some cases more prestigious venue for exposition than academic journals:
[The sluggishness of journal publications] is offset somewhat by a flourishing array of conferences and other forums, in both virtual and real space, that provide a sense of community and an outlet as well as feedback[11]....The prestige associated with presentations at major conferences actually makes some of them more selective than journals.[12]
New forms of conference archiving--such as archived Webcasts--add value and exposure to the research presented at conferences.
3. Citations
Citations are a valuable and versatile measure of peer influence because they may come from or point to a variety of genres, from Web sites to databases to books in print. Examples include citations in:
a. Electronic archives and recognition networks, such as the publicly accessible databases maintained by the Daniel Langlois Foundation (Montreal), the V2 organization (Rotterdam), the Database of Virtual Art (Berlin), and the Media Art Net database (Karlsruhe).
b. Books, printed journals, and newspapers. These are easier to find now, thanks to Google Scholar, Google Print, and Amazon's "look inside the book" feature.
c. Syllabi and other pedagogical contexts. Google searches on .edu domains and citations of the author's work in syllabi from outside universities can measure the academic currency of an individual researcher or her ideas. In the sciences, readings or projects cited on a syllabus are likely to be popular textbooks, but in an emerging field like new media, such recognition is a more valid marker of relevance.
4. Download / visitor counts
Downloads and other traffic-related statistics represent a measure of influence that has gained importance in the online community recently. As a 2005 open access study[13] concludes:
Whereas the significance of citation impact is well established, access of research literature via the Web provides a new metric for measuring the impact of articles – Web download impact.
Download impact is useful for at least two reasons:
(1) The portion of download variance that is correlated with citation counts provides an early-days estimate of probable citation impact that can begin to be tracked from the instant an article is made Open Access and that already attains its maximum predictive power after 6 months.
(2) The portion of download variance that is uncorrelated with citation counts provides a second, partly independent estimate of the impact of an article, sensitive to another form of research usage that is not reflected in citations (Kurtz 2004).
5. Impact in online discussions
Email discussion lists are the proving grounds of new media discourse. They vary greatly in tone and substance, but even the least moderated of such lists can subject their authors to rigorous--and at times withering--scrutiny.[14] Measures such as the number of list subscribers, geographic scope, the presence or absence of moderation, and the number of replies triggered by a given contribution can give a sense of the importance of each discussion list.[15]
6. Impact in the real world
While magazine columns and newspaper editorials may have little standing in traditional academic subjects, one of the strengths of new media are their relevance to a daily life that is increasingly inflected by the relentless proliferation of technologies. Even counting Google search returns on the author's name or statistically improbable phrases can be a measure of real-world impact[16].
By privileging new media research with direct effect on local or global communities, the university can remain relevant in an age where much research takes place outside the ivory tower.
8. Net-native recognition metrics
Peer-evaluated online communities may invent their own measures of member evaluation, in which case they may be relevant to a researcher who participates in those communities. Examples of such self-policing communities include Slashdot, The Pool, Open Theory, and the Distributed Learning Project.
The MLA pins the responsibility for learning these new metrics on reviewers rather than the reviewed.[17] Given the mutability of such metrics, however, promotion and tenure candidates may be called upon to explain and give context to these metrics for their reviewers. Again, efforts to educate a scholar's colleagues about new media should be considered part of that scholar's research, not supplemental to it.
9. Reference letters
Letters of recommendation from outside referees are an important compensation for the irrelevance of traditional recognition venues. Nevertheless, it is insufficient merely to solicit such letters from professors tenured in new media at other universities, since so few exist.
More valuable is to use the measures outlined in this document to identify pre-eminent figures in new media, or to require new media promotion and tenure candidates to identify such figures and supply evidence that they qualify according to the criteria above.
[1] The ACLS recommends "policies for tenure and promotion that recognize and reward digital scholarship and scholarly communication; recognition should be given not only to scholarship that uses the humanities and social science cyberinfrastructure but also to scholarship that contributes to its design, construction, and growth....
We might expect younger colleagues to use new technologies with greater fluency and ease, but with tenure at stake, they will also be more risk-averse....
Senior scholars now have both the opportunity and the responsibility to take certain risks, first among which is to condone risk taking in their junior colleagues and their graduate students, making sure that such endeavors are appropriately rewarded."
"Our Cultural Commonwealth," report by the ACLS Commission on Cyberinfrastructure for the Humanities and Social Sciences, 29 July 2006,
[http://www.acls.org/cyberinfrastructure/cyber.htm[, accessed January 2, 2007.
[2] "Departments and institutions should recognize the legitimacy of scholarship produced in new media, whether by individuals or in collaboration, and create procedures for evaluating these forms of scholarship." December 2006 report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion,
[http://www.mla.org/tenure_promotion] accessed January 2, 2007.
[3] "The Commission encourages each department on campus, as well as the University as a whole, to examine promotion and tenure criteria to recognize and reward innovative uses of technology in teaching, research and service....the University needs to consider the criteria and standards used in the promotion and tenure process.
The Commission encourages each department and the University as a whole to consider whether faculty efforts in this area are recognized, valued, and/or encouraged." November 2003 report of the University of Maine Commission on Information Technologies, accessed at
[http://www.umaine.edu/documents/CIT.pdf] on May 2, 2004.
[4] MLA Committee on Information Technology. "Guidelines for Evaluating Work with Digital Media in the Modern Languages." 20 May 2000. ADE Bulletin 132 (2002): 94–95. 82, mirrored at
[http://www.mla.org/guidelines_evaluation_digital], accessed 2 January, 2007.
[5] A brief sampling of new media theorist-practitioners includes Simon Biggs (Cambridge University), Matthew Fuller (Piet Zwart Institute, Rotterdam), Mary Flanagan (Hunter), Alexander Galloway (NYU), Kenneth Goldberg (Berkeley), Eduardo Kac (Art Institute of Chicago), Natalie Jeremijenko (UCSD), Raphael Lozano-Hemmer (Karlstad University, Sweden), Lev Manovich (UCSD), Randall Packer (American University), Richard Rinehart (Berkeley), and Jeffrey Shaw (ZKM).
[6] National Research Council, Beyond Productivity: Information Technology, Innovation, and Creativity (Washington, DC: The National Academies Press, 2003), pp. 8-9.
[7] National Research Council, op. cit., p. 188.
[8] These estimates are from MIT's Roger Malina (Director of Leonardo magazine) and the Daniel Langlois Foundation's Alain Depocas (Director of the Centre for Documentation + Research), and are mirrored at
[http://cordova.asap.um.maine.edu/wiki/index.php/Standards_of_Recognition]
[9] The Interarchive project is a possible model for distributed publication; see
[http://newmedia.umaine.edu/interarchive]
[10] "Change in favor of a more capacious conception of scholarship, which we strongly endorse, should not mean ever-wider demands on faculty members, most especially those coming up for tenure and promotion." MLA Task Force on Evaluating Scholarship for Tenure and Promotion, op. cit., p. 21.
[11] National Research Council, op. cit., pp. 8-9.
[12] National Research Council, op. cit., p. 188.
[13] Tim Brody and Stevan Harnad, "Earlier Web Usage Statistics as Predictors of Later Citation Impact",
[http://eprints.ecs.soton.ac.uk/10647/], accessed 5 March 2005.
[14] This recent
[http://www.nettime.org/Lists-Archives/nettime-l-0504/msg00051.html]
rejoinder by Morlock Elloi on the
> If you have any past publications that might help me understand your point of view, I would gladly read them.
While I understand that in paidspeakerworld the weight of the argument is computed as (volume of publications) x (number of speeches), on nettime and elsewhere closer to reality arguments stand for themselves.
[15] Electronic and email texts also have a currency acknowledged by leading institutions in the field. As of December 21, 2005, one of the premiere bibliographic indices in new media, the Langlois Foundation's CR+D database, included the following indexation for "Jon Ippolito":
* Author of 10 documents
* Subject of 48 documents
* Participant to 21 events
* Organizer of 2 events
Of the 10 documents by the author indexed, 1 is from an email list and 2 are parts of Web sites. In the case of artist and critic Alexander Galloway, the relevance of his online texts is even more striking: although by 2005 he was the author of several journal articles and an important book from MIT Press, the two documents that represented his writing in the CR+D database were both from email lists.
[16] A statistically significant number of Google returns, eg > 30, may be a necessary but insufficient condition for confirming global impact.
[17] "In evaluating scholarship for tenure and promotion, committees and administrators must take responsibility for becoming fully aware both of the mechanisms of oversight and assessment that already govern the production of a great deal of digital scholarship and of the well-established role of new media in humanities research.
It is of course convenient when electronic scholarly editing and writing are clearly analogous to their print counterparts. But when new media make new forms of scholarship possible, those forms can be assessed with the same rigor used to judge scholarly quality in print media. We must have the flexibility to ensure that as new sources and instruments for knowing develop, the meaning of scholarship can expand and remain relevant to our changing times." MLA Task Force on Evaluating Scholarship for Tenure and Promotion, op. cit., p. 46.
[http://newmedia.umaine.edu/interarchive/new_criteria_for_new_media.html]
No comments:
Post a Comment