Digital Writing Research: Technologies, Methodologies, and Ethical Issues

Title: Digital Writing Research

Sub-title: Technologies, Methodologies, and Ethical Issues

Editor(s): Heidi A. McKee and Danielle DeVoss

Publish Date: August 2007

Pages: 476

Format: Cloth

This volume focuses on how writing technologies, specifically digital technologies, affect research--shaping the questions asked, the sites studies, the methodologies used, ethical issues, conclusions, and the actions taken by scholars, researchers, and teachers.

In this collection, the authors analyze methodologies, technologies, and ethical approaches for researching digital writing and writers working in digital contexts. Although many of the chapters provide examples drawn from studies conducted or reviewed by the authors, the focus of each chapter is on articulating particular methodologies and ethical approaches for conducting digital writing research. Because of the increasing digitalization of writing in educational, institutional, and social contexts, all composition researchers need to consider methodological and ethical approaches to digital writing research

The core questions addressed in the chapters include:

• How have researchers adapted methodologies for digital writing research?
• What methods are used by researchers studying sign systems beyond the textual?
• How is a particular writing technology being researched by computers and writing scholars?
• What constitutes appropriate human subject research in online environments?
• How are computerized technologies, particularly global technologies, raising new ethical issues related to privacy, individual rights, and representation?
• How have computers and digital spaces changed collaboration among researchers and participants?
• How have electronic journals and other methods of publishing writing research influenced our research directions and the distribution of research findings?

Given the continually evolving state of technology and human interactions with and through technological affordances, what preparation do future researchers need?This volume will provide experienced researchers with the means to reflect on various aspects of their research and offer researchers new to composition studies or to computers and writing research an introduction to possible approaches and related methodological and ethical issues.

Contents:

Introduction. RESEARCHING DIGITAL COMMUNITIES: REVIEW, TRIANGULATION, AND ETHICAL RESEARCH REPORTS. Digital Spaces, Online Environments, and Human Participant Research: Interfacing with Institutional Review Boards, Will Banks and Michelle Eble. Through the Eyes of Researchers, Rhetors and Audiences: Triangulating Data from the Digital Writing Situation, Kevin DePew. Playing Scavenger and Gazer with Scientific Discourse: Opportunities and Ethics for Online Research, Michelle Sidler. RESEARCHING GLOBAL CITIZENS AND TRANSNATIONAL INSTITUTIONS. Ethos and Research Positionality in Studies of Virtual Communities, Fil Sapienza. Researching (with) the Postnational 'Other': Ethics, Methodologies, and Qualitative Studies of Digital Literacy, Iswari Pandey. Researching Hybrid Literacies: Methodological Explorations of 'Ethnography' and the Practices of the Cybertariat, Beatrice Smith. RESEARCHING THE ACTIVITY OF WRITING: TIME-USE DIARIES, MOBILE TECHNOLOGIES, AND VIDEO SCREEN CAPTURE. Studying the Mediated Action of Composing with Time-Use Diaries, William Hart-Davidson. Mobile Technologies and a Phenomenology of Literacy, Joanne Addison. Capturing the Activity of Digital Writing: Using, Analyzing, and Supplementing Video Screen Capture, Cheryl Geisler and Shaun Slattery. RESEARCHING DIGITAL TEXTS AND MULTIMODAL SPACES. Coding Digital Texts and Multimedia, Stuart Blythe. Composition Meets Visual Communication: New Research Questions: Susan Hilligoss and Sean Williams. An Ecofeminist Methodology: Studying the Ecological Dimensions of the Digital Environments, Julia Romberger. Riding the Wave: Articulating a Critical Methodology for Web Research Practices, Amy Kimme Hea. Multimedia Research: Difficult Questions with Indefinite Answers, Janice McIntire-Strasburg. RESEARCHING THE RESEARCH PROCESS AND RESEARCH REPORTS. Whose Research is it, Anyway? The Multifaceted Methods for Multimodal Texts: Alternate Approaches to Citation Analysis for Electronic Sources, Colleen Reilly and Doug Eyman. Messy Contexts: Research as a Rhetorical Situation, Rebecca Rickly.

Combined Bibliography / Author Index / Subject Index

[http://tinyurl.com/28x4mn]

Thomson Scientific Launches New Citation Impact Forum

Thomson Scientific [has] announced the launch of its Citation Impact Forum [http://www.scientific.thomson.com/citationimpactforum], an online forum promoting scholarly discussion about all facets of citation-based research evaluation—from Thomson Scientific’s own Journal Impact Factor to emerging citation metrics, such as the h-index.

The company hopes that by promoting an understanding and discussion of scholarly metrics, those in the industry will be encouraged to use metrics more effectively.

The Citation Impact Forum will feature interviews with and commentary from industry leaders— bibliometricians, researchers, and publishers—about scholarly evaluation. By becoming a registered member of the forum, users have the added ability to join invitation-only forums where they can participate in discussions by sharing their own thoughts with other members. Registered members can also recommend future forum discussion topics.

Thomson Scientific said it will also use the forum to address—or to clarify—inaccuracies and common misinterpretations circulating among the scientific community printed in journals, blogs, and other forums.

Information Today

[http://newsbreaks.infotoday.com/wndReader.asp?ArticleId=40584]

Userscripts for the Life Sciences

Userscripts for the Life Sciences
Egon L Willighagen , Noel M O'Boyle , Harini Gopalakrishnan , Dazhi Jiao , Rajarshi Guha , Christoph Steinbeck and David J Wild

BMC Bioinformatics 2007, 8:487doi:10.1186/1471-2105-8-487
Published: 21 December 2007

Abstract (provisional)

Background
The web has seen an explosion of chemistry and biology related resources in the last 15 years: thousands of scientific journals, databases, wikis, blogs and resources are available with a wide variety of types of information. There is a huge need to aggregate and organise this information. However, the sheer number of resources makes it unrealistic to link them all in a centralised manner. Instead, search engines to find information in those resources flourish, and formal languages like Resource Description Framework and Web Ontology Language are increasingly used to allow linking of resources. A recent development is the use of userscripts to change the appearance of web pages, by on-the-fly modification of the web content. This pens possibilities to aggregate information and computational results from different web resources into the web page of one of those resources.

Results
Several userscripts are presented that enrich biology and chemistry related web resources by incorporating or linking to other computational or data sources on the web. The scripts make use of Greasemonkey-like plugins for web browsers and are written in JavaScript. Information from third-party resources are extracted using open Application Programming Interfaces, while common Universal Resource Locator schemes are used to make deep links to related information in that external resource. The userscripts presented here use a variety of techniques and resources, and show the potential of such scripts.

Conclusions
This paper discusses a number of userscripts that aggregate information from two or more web resources. Examples are shown that enrich web pages with information from other resources, and show how information from web pages can be used to link to, search, and process information in other resources. Due to the nature of userscripts, scientists are able to select those scripts they find useful on a daily basis, as the scripts run directly in their own web browser rather than on the web server. This flexibility allows the scientists to tune the features of web resources to optimise their productivity.

[http://baoilleach.blogspot.com/2008/01/enhancing-web-for-scientists.html]

Blog
[http://baoilleach.blogspot.com/2008/01/enhancing-web-for-scientists.html]

Establishing a Research Agenda for Scholarly Communication. 5

5. Value and Value Metrics in Scholarly Communications

Determining and measuring the effectiveness of and the value that is derived from scholarly communication is challenging and often subjective. John Houghton’s work on scholarly communication in Australia seeks to measure the economic and social returns to public sector investment in research and development and how those might rise with open access to published research findings. Analyses of citations in the published literature to the published literature lead to metrics such as the h-index, Eigenfactor, and the heavily-used impact factor. Extant measures may suffer from being tightly coupled to traditional processes while also inhibiting the application of other measures of value.

In the new digital environment, activities other than traditional or formal publication should be valued in the reward structure for scholarship. To this end, the Modern Language Association provides an example of examining current standards and emerging trends in publication requirements for tenure and promotion.

Illustrative Challenges
Citation analysis relies on a 50 year old assumption that the number of citations represents value, but in today’s environment this assumption is limiting. Other metrics could reflect the scholarly significance of new discoveries as they are developed and communicated. Effective metrics must be based on resources and practices that truly advance scholarly research. For example, it could be argued that journal articles have become totems to accrue and count for tenure and promotion but are not unique in their ability to advance scholarship and may be losing some effectiveness for this purpose.

"Open notebook science" and “open data” are examples of new research and communication practices that might be advancing scholarly research as much or more than what is possible through scholarly publication. The relationship between the reward system and indicators of the progress of knowledge is more tenuous. What resources and practices truly advance scholarly research? Even where robust indicators of the progress of knowledge exist, their relationship with the current reward system may be tenuous. How can the value and impact of communication practices be assessed and documented? How could these assessments be assimilated into the reward system?

Libraries should adopt a stronger role that more directly advances scholarly research beyond satisfying tenure and promotion practices. A starting point is work reported by King and Harley regarding formal vs. informal scholarly communications. Given that scholars are finding new ways to register, seek comment, refine, evaluate, and certify their work, how can those processes be tracked, recorded, measured, and reported as part of the value-chain in scholarship?

Should informal communications be captured and preserved by libraries, and if so, how? A useful analogy is to consider that presentations, preprints and letters and other informal communications are the conversations of science, while publications are the minutes. In some disciplines, journals are becoming less important to scholars than their professional meetings and informal networks where their accomplishments are recognized. How can librarians better characterize and, measure the contributions of these informal communications, and thereby make wise decisions about organized access to them?

Libraries also need to determine the value of their own services as contributions to the communication of scholarship. Which services, such as institutional repositories, should be evaluated, and what tools and measures exist for this purpose?

Research Possibilities
• Identify and evaluate the range of metrics currently used to measure the value and impact of scholarly publishing. There is citation analysis and its derivatives; but there are other measures being developed, including those that combine usage/readership and citation, such as those by the MESUR project at Los Alamos National Laboratory, and the UKSG Serials Group. A literature review to collect these efforts would provide a central reference point.
• Explore additional measurements that incorporate new kinds of indicators of value and covering a broader range of communication activities. New measures should address increased research efficiency and productivity, variations between disciplines, advancement of the process of research.

Other metrics may:
° characterize the value of Open Data and Open Notebook Science (disseminating source data, research methods, and negative experimental or clinical results) to advancing research and knowledge.
° correspond to technology transfer or other uses of new knowledge beyond generating further research, for instance, number of views, number of patents.
° show how informal communications are advancing the process of research

• Explore the relative value, importance, and significance of traditional journal and book publication compared to newer, informal forms of scholarly communication for a sample of representative scholars. This could build on studies by the CIC and Estabrook that indicated that in the humanities there is some acceptance of digital publications and new forms, while the scholarly monograph was still the standard for promotion and tenure.

[http://acrl.ala.org/scresearchagenda/index.php?title=Theme_5:_Value_and_Value_Metrics_of_Scholarly_Communications]

Establishing a Research Agenda for Scholarly Communication: A Call for Community Engagement

Establishing a Research Agenda for Scholarly Communication: A Call for Community Engagement

By ACRL Scholarly Communications Committee

Executive Summary:
The system of scholarly communication – which allows research results and scholarship to be registered, evaluated for quality, disseminated, and preserved – is rapidly evolving. Academic libraries and their parent institutions are adopting strategies, making plans, and taking action to respond to the changing environment and to influence its development. Believing that meaningful research can inform and assist the entire academic community in influencing and managing this evolution, the Association of College and Research Libraries (ACRL) convened an invitational meeting on July 21, 2007, to collectively brainstorm the evidence needed to inform strategic planning for scholarly communication programs.

Influencing a system as complex and dynamic as scholarly communication requires broad and deep understanding. The issues for investigation that emerged at the meeting range from cyberinfrastructure to changing academic organizational models to public policy. This report thematically summarizes and synthesizes the meeting’s rich discussion, framing eight essential research challenges and opportunities. We invite those engaged in creating, supporting, and distributing scholarship to comment and extend the issues and possible research initiatives. Without substantive comment from librarians and their partners, the goal of outlining a community research agenda cannot be considered complete.

Introduction

Themes and Research Opportunities:
Participants identified eight themes characterizing the changes transforming scholarly communications. Developing a deeper understanding of these challenges through research can enable academic stakeholders to influence and construct scholarly communication systems that optimally support the academic enterprise and the communities it serves.

Theme 1: The Impact and Implications of Cyberinfrastructure

Theme 2: Changing Organizational Models

Theme 3: How Scholars Work

Theme 4: Authorship and Scholarly Publishing

Theme 5: Value and Value Metrics of Scholarly Communications

Theme 6: Adoption of Successful Innovations

Theme 7: Preservation of Critical Materials

Theme 8: Public Policy and Legal Matters

Conclusion and Invitation

Appendix A: Attendee List

Appendix B: ACRL Scholarly Communications Committee Roster 2007-2008

Bibliography

Coverage and Commentary

[http://acrl.ala.org/scresearchagenda/index.php?title=Main_Page]

PDF Version Podcast Interview Press Release

Released November 2007

Show Me The Data! Show Me The Data! Show Me The Data!

Show Me The Data
Mike Rossner, Heather Van Epps, and Emma Hill
The Journal of Cell Biology, Vol. 179, No. 6, 1091-1092

The integrity of data, and transparency about their acquisition, are vital to science. The impact factor data that are gathered and sold by Thomson Scientific (formerly the Institute of Scientific Information, or ISI) have a strong influence on the scientific community, affecting decisions on where to publish, whom to promote or hire, the success of grant applications, and even salary bonuses. Yet, members of the community seem to have little understanding of how impact factors are determined, and, to our knowledge, no one has independently audited the underlying data to validate their reliability.

Calculations and negotiations
The impact factor for a journal in a particular year is declared to be a measure of the average number of times a paper published in the previous two years was cited during the year in question. For example, the 2006 impact factor is the average number of times a paper published in 2004 or 2005 was cited in 2006. There are, however, some quirks about impact factor calculations that have been pointed out by others, but which we think are worth reiterating here:

[snip]

3. Some publishers negotiate with Thomson Scientific to change these designations in their favor . The specifics of these negotiations are not available to the public, but one can't help but wonder what has occurred when a journal experiences a sudden jump in impact factor. For example, Current Biology had an impact factor of 7.00 in 2002 and 11.91 in 2003. [snip]

4. Citations to retracted articles are counted in the impact factor calculation. [snip]

5. Because the impact factor calculation is a mean, it can be badly skewed by a "blockbuster" paper. [snip]

When we asked Thomson Scientific if they would consider providing a median calculation in addition to the mean they already publish, they replied, "It's an interesting suggestion...The median...would typically be much lower than the mean. There are other statistical measures to describe the nature of the citation frequency distribution skewness, but the median is probably not the right choice." Perhaps so, but it can't hurt to provide the community with measures other than the mean, which, by Thomson Scientific's own admission, is a poor reflection of the average number of citations gleaned by most papers.

6. There are ways of playing the impact factor game, known very well by all journal editors, but played by only some of them. For example, review articles typically garner many citations, as do genome or other "data-heavy" articles [snip]. When asked if they would be willing to provide a calculation for primary research papers only, Thomson Scientific did not respond.

Integrity
As journal editors, data integrity means that data presented to the public accurately reflect what was actually observed. [snip]

Thomson Scientific makes its data for individual journals available for purchase. With the aim of dissecting the data to determine which topics were being highly cited and which were not, we decided to buy the data for our three journals (The Journal of Experimental Medicine, The Journal of Cell Biology, and The Journal of General Physiology) and for some of our direct competitor journals. Our intention was not to question the integrity of their data.

When we examined the data in the Thomson Scientific database, two things quickly became evident: first, there were numerous incorrect article-type designations. Many articles that we consider "front matter" were included in the denominator. This was true for all the journals we examined. Second, the numbers did not add up. The total number of citations for each journal was substantially fewer than the number published on the Thomson Scientific, Journal Citation Reports (JCR) website ... . The difference in citation numbers was as high as 19% for a given journal, and the impact factor rankings of several journals were affected when the calculation was done using the purchased data [snip]

Your database or mine?
When queried about the discrepancy, Thomson Scientific explained that they have two separate databases—one for their "Research Group" and one used for the published impact factors (the JCR). [snip]

When we requested the database used to calculate the published impact factors (i.e., including the erroneous records), Thomson Scientific sent us a second database. But these data still did not match the published impact factor data. This database appeared to have been assembled in an ad hoc manner to create a facsimile of the published data that might appease us. It did not.

Opaque data
It became clear that Thomson Scientific could not or (for some as yet unexplained reason) would not sell us the data used to calculate their published impact factor. If an author is unable to produce original data to verify a figure in one of our papers, we revoke the acceptance of the paper. We hope this account will convince some scientists and funding organizations to revoke their acceptance of impact factors as an accurate representation of the quality—or impact—of a paper published in a given journal.

Just as scientists would not accept the findings in a scientific paper without seeing the primary data, so should they not rely on Thomson Scientific's impact factor, which is based on hidden data. As more publication and citation data become available to the public through services like PubMed, PubMed Central, and Google Scholar®, we hope that people will begin to develop their own metrics for assessing scientific quality rather than rely on an ill-defined and manifestly unscientific number.

[snip]

[ http://www.jcb.org/cgi/content/full/179/6/1091]

The Insight Journal: An Open Access, Dynamic Publishing Environment

The Insight Journal is an Open Access on-line publication covering the domain of medical image processing. The unique characteristics of the Insight Journal include:

• Open-access to articles, data, code, and reviews : All submissions are available for free. The Insight Journal advances the idea that the results of scientific research should be made available to the public. Open access enables others to more fully understand the research and to more easily build upon the research. The field of medical image analysis will more rapidly advance.


• Open peer-review that invites discussion between reviewers and authors: The review process is made public and performed online. Every reader is a reviewer and may post comments. Reviewers' comments are posted with the submissions. Authors may respond to reviewers' comments. Authors may submit revisions to the papers in response to reviewers' comments. [snip]


• Emphasis on reproducible science via automated code compilation and testing: Code submitted to the IJ is verified by an automatic system. Authors are encouraged to post their papers along with the source code, the input data, and the expected output data needed to replicate the results presented in the paper. [snip]


• Support for continuous revision of articles, code, and reviews: The IJ is a dynamic publication environment. All submissions can be continually reviewed, tested, revised, and extended. [snip]


• The IJ is volunteer supported: We need your help in the form of submissions, reviews, and ideas! The IJ is a novel concept in the field of medical image analysis, and we need your time, intelligence, and insight to make it work.

[http://www.insight-journal.org/index.htm]