MESUR For Measure: MEtrics from Scholarly Usage of Resources

MESUR: MEtrics from Scholarly Usage of Resources

[http://www.mesur.org/]

The project's major objective is enriching the toolkit used for the assessment of the impact of scholarly communication items, and hence of scholars, with metrics that derive from usage data. The project has created a semantic model of the scholarly communication process, and an associated large-scale semantic store that relates a range of bibliographic, citation and usage data obtained from a variety of sources.

After mapping the structure of the scholarly community on the basis of the established reference data set, MESUR will conduct an investigation into the definition and validation of a range of usage-based metrics. The defined metrics will be cross-validated, resulting in the formulation of guidelines and recommendations.
MESUR Database

The MESUR data base now contains 1B usage events (2002-2007) obtained from 6 significant publishers, 4 large institutional consortia and 4 significant aggregators! The collected usage data spans more than 100,000 serials (including newspapers, magazines, etc.) and is related to journal citation data that spans about 10,000 journals and nearly 10 years (1996-2006). In addition we have obtained significant publisher-provided COUNTER usage reports that span nearly 2000 institutions worldwide.

The data is being ingested into a combination of relational and semantic web databases, the latter of which is now estimated to result in nearly 10 billion semantic statements (triples). MESUR is now producing large-scale, longitudinal maps of the scholarly community and a survey of more than 60 different metrics of scholarly impact.

Quick Facts

Funding: The Andrew W. Mellon Foundation / Timeline: October 2006 - October 2008

Principal investigator: Johan Bollen / Institution: Los Alamos National Laboratory (LANL) / Team: Digital Library Research & Prototyping Team of the LANL Research Library

People: Johan Bollen is the Principal Investigator; Herbert Van de Sompel serves as an architectural consultant; and Aric Hagberg of the LANL Mathematical Modeling and Analysis group serves as modeling consultants.

Marko A. Rodriguez, a recent PhD graduate at the University of California Santa Cruz and now LANL post-doc at the LANL Center for Non-Linear Science, has supported the project's research and development. Ryan Chute of the LANL Research Library is now the project’s main developer and database manager.

Source

[http://www.mesur.org/]

Documentation

Overview Papers / Lectures and Slides / MESUR Timeline / Metrics / MESUR Official Summary

[http://www.mesur.org/Documentation.html]

MESUR Publications

Articles / Lectures and Slides

[http://www.mesur.org/Publications.html]

Contact

[http://www.mesur.org/Contact.html]

Related

[http://scholarship20.blogspot.com/2008/12/final-impact-what-factors-really-matter.html]

See Also

A Principal Component Analysis of 39 Scientific Impact Measures

[http://scholarship20.blogspot.com/2009/06/principal-component-analysis-of-39.html]

Final Impact: What Factors Really Matter?

Columbia University Libraries. Information Services. Scholarly Communications Program / Fostering Innovation in Scholarly Communication /

Research Without Borders: The Changing World of Scholarly Communication

Final Impact: What Factors Really Matter?

October 30 2008 / 3:00 pm to 5:00 pm EDT/

Columbia University Medical Center / Hammer Health Sciences Center / 701 W. 168th Street / Room 401

A panel discussion on the debate about the best way to rank the importance and influence of scholarly publications.

Panelists:

• Marian Hollingsworth, director of Publisher Relations at Thomson Reuters and former assistant director of the National Federation of Abstracting and Information Services [Start: 4:15]


• Jevin West, an Achievement Awards for College Scientists Fellow at the University of Washington's Biology Department and head developer for Eigenfactor.org [Start: 20:10]


• Johan Bollen, a staff researcher at Los Alamos National Laboratory and the principal investigator of the MESUR project [Start: 61:45]

Columbia University Librarian Jim Neal introduces the talk [Start: 00:00]

A/V Available at

[http://scholcomm.columbia.edu/content/final-impact-what-factors-really-matter] [Duration: 106:02 minutes]

>>PDF/PPT Slides NOW Available [01-05-09]<<

Source

[http://scholcomm.columbia.edu/?q=content/multimedia]

Also Available As A Podcast

[http://deimos3.apple.com/WebObjects/Core.woa/Browse/columbia.edu.1786272348]

Related

"The Scholarly Communication Program is pleased to present a speaker series for the 2008-09 academic year on today's pivotal issues in scholarly communication. Six events will explore how scholars and researchers can take advantage of new and powerful ways of creating, sharing, reusing, and preserving knowledge."

[http://scholcomm.columbia.edu/content/events]

Research Without Borders: The Changing World of Scholarly Communication Programs Also Available As A Podcast

[http://deimos3.apple.com/WebObjects/Core.woa/Browse/columbia.edu.1786272348]

Research Without Borders: The Changing World of Scholarly Communication Facebook Global Group

[http://www.facebook.com/group.php?gid=26968639503]

CHE: Bringing Tenure Into The Digital Age: Q&A With Christine L. Borgman

Bringing Tenure Into the Digital Age

New tools for analyzing information are arriving every day, but that doesn’t mean scholars who use them well are being rewarded, says Christine L. Borgman, a professor of information studies at the University of California at Los Angeles. She contends that the new “scholarly information infrastructure” must be shaped with collaborative, interdisciplinary research.

Q. In your recent book, “Scholarship in the Digital Age,” you contend that the tenure system needs to reward people for contributions to collaborative digital projects instead of recognizing only those who publish books and articles. Why?

A. Data is becoming a first-class object. In the days of completely paper publication, the article or book was the end of the line. And once the book was in libraries, the data were often thrown away or allowed to deteriorate.

Now we’re in a massive shift. Data become resources. They are no longer just a byproduct of research. And that changes the nature of publishing, how we think about what we do, and how we educate our graduate students. The accumulation of that data should be considered a scholarly act as well as the publication that comes out of it.

[snip]

Q. Do you have any tips for the young scholar who feels deluged and overwhelmed?

A. Look for good data that have already been generated and are available. It’s the old saw about how an hour in the library can keep you from spending 60 hours in the lab. It’s similar in research nowadays. Finding good data that someone else has done, that you can build upon, is time well spent. [And] find partners that complement your expertise.

Q. What is your prescription when it comes to building infrastructure that makes all this information available?

A. We need a new conversation. We need to determine what we should be building, instead of just figuring if we build it, they will come. We’ve spent a lot of money on the technology without asking a lot of questions about the nature of scholarship.

When we do ask those questions, we will come up against entrenched interests, like the way we publish and get tenure. So we need to consider the policies and incentives for the reward system and for the use and reuse of information. These will need to change. —Lisa Guernsey

[http://chronicle.com/wiredcampus/article/3511/bringing-tenure-into-the-digital-age]

Workshop: Making The Web Work For Science

Making the Web Work for Science:

The Impact of e-Science and the Cyber-Infrastructure

A One-Day Workshop Co-sponsored by CENDI and NFAIS and Hosted by FLICC

Library of Congress, 101 Independence Ave, SE, Washington, DC 20540

Mumford Room / December 8, 2008 / 9:00am - 4:30pm

AGENDA (11-4-08)

8:30am - 9:00am: Registration/Coffee

9:00am - 9:15am: Welcome / Opening Remarks Roberta Shaffer, Director of FLICC, Library of Congress

9:15am - 10:00am: Making the Web Work for Science: The Current Landscape

The opening keynote will provide an overview of how the Web is currently being utilized for the advancement of science and scholarly communication. Roberta Shaffer will introduce Dr. Christine Borgman, Professor & Presidential Chair in Information Studies, University of California, Los Angeles, and author of Scholarship in the Digital Age: Information, Infrastructure, and the Internet.

10:00am - 10:15am: Break and Networking Opportunity

10:15am - 11:45 pm: Making the Web Work for Science: The Content Providers’ Perspective

This session will focus on how innovative content providers, including Federal STI program leaders, librarians, and publishers are leveraging current Web technologies in order to maximize global access to and use of scientific and scholarly information. The use of Web 2.0 features such as Wiki’s, RSS feeds and blogs will be discussed as will plans for the future.

The panel participants are Dr. Walter Warnick, Director, Office of Scientific and Technical Information, Department of Energy; Dr. Sayeed Choudhury, Johns Hopkins University; and Howard Ratner, Executive Vice President and Chief Technology Officer, Nature Publishing Group. Karen Spence, DOE/OSTI, will moderate.

11:45am - 12:45pm: Lunch

12:45pm - 2:00pm: Making the Web Work for Science: What Scientists Really Need!

In this session, two practicing scientists will discuss their use of conventional and Web-based information tools for scientific research, what works and what does not, and what they believe the information community needs to provide in to maximize the full potential of the Web as an effective and essential resource for scientific discovery.

The panel participants are Dr. Antony Williams, Founder, ChemSpider; and Dr. Alberto Conti, Astrophysicist, Space Telescope Science Institute. Jill O’Neill, NFAIS, will moderate.

2:00pm - 3:30pm: Making the Web Work for Science: Challenges to Implementation

In this session, three experts will discuss the technological, legal and cultural challenges that all organizations must overcome – libraries, publishing institutions, scientific laboratories, etc. - so that each can utilize the full potential of the Internet and the Web met in the fulfillment of their common mission – to build the world’s knowledgebase through enabling research and managing the flow of scholarly communication.

The participants are Dr. Michael R. Nelson, Visiting Professor at Georgetown University; Fred Haber, Vice President and General Counsel, Copyright Clearance Center; Dr. Michael Nielsen, Physicist and Science Writer, Perimeter Institute for Theoretical Physics (Canada). Bonnie C. Carroll, Executive Director of the CENDI Secretariat, will moderate.

3:30pm - 3:45pm: Break

3:45pm - 4:30pm: Making the Web Work for Science: What the Future Holds

This final keynote will explore the future promise of the Web and the various ways in which the cyber-infrastructure can ultimately re-engineer not only how scientific research is conducted, but also how the resultant information is communicated, shared, verified, and built upon as scientists and scholars around the globe increasingly collaborate in building the world’s knowledgebase of scientific and scholarly information.

Ellen Herbst, NTIS Director, will introduce Dr. Christopher Greer, recently of the National Science Foundation’s Cyber-Infrastructure Office, and currently the Director of Networking and Information Technology Research and Development (NITRD) of the National Coordination Office.

4:30pm: Adjournment

PDF Version Available At

[http://cendievents.infointl.com/nfais_cendi_120808/docs/Agenda.pdf]

Source

[http://cendievents.infointl.com/nfais_cendi_120808/docs/agenda.html]

General Information / Registration / Etc.

There is a two-fee structure for this workshop to allow the sponsors’ and host’s members an opportunity to attend at a reduced cost. CENDI, NFAIS, and FLICC members will be charged $65.00; all others have a registration fee of $95.00.

[http://cendievents.infointl.com/nfais_cendi_120808/]

Defrosting The Digital Library: Bibliographic Tools For The Next Generation Web

Hull, D., S. R. Pettifer, and D. B. Kell / October 2008 / Defrosting The Digital Library: Bibliographic Tools For The Next Generation Web / PLoS Comput Biol 4 (10), e1000204+

Introduction
Digital Libraries, DOIs, and URIs
Problems Using Digital Libraries
Some Tools for Defrosting Libraries
A Future with Warmer Libraries
Recommendations
Conclusions

Extensive Bibliography (210 Items)

Many scientists now manage the bulk of their bibliographic information electronically, thereby organizing their publications and citation material from digital libraries. However, a library has been described as “thought in cold storage,” and unfortunately many digital libraries can be cold, impersonal, isolated, and inaccessible places.

In this Review, we discuss the current chilly state of digital libraries for the computational biologist, including PubMed, IEEE Xplore, the ACM digital library, ISI Web of Knowledge, Scopus, Citeseer, arXiv, DBLP, and Google Scholar. We illustrate the current process of using these libraries with a typical workflow, and highlight problems with managing data and metadata using URIs.

We then examine a range of new applications such as Zotero, Mendeley, Mekentosj Papers, MyNCBI, CiteULike, Connotea, and HubMed that exploit the Web to make these digital libraries more personal, sociable, integrated, and accessible places.

We conclude with how these applications may begin to help achieve a digital defrost, and discuss some of the issues that will help or hinder this in terms of making libraries on the Web warmer places in the future, becoming resources that are considerably more useful to both humans and machines.

Source and Full Text

[http://dx.doi.org/10.1371/journal.pcbi.1000204]

Current Biomedical Publication System: A Distorted View of the Reality of Scientific Data?

Why Current Publication Practices May Distort Science

Young NS, Ioannidis JPA, Al-Ubaydli O

PLoS Medicine Vol. 5, No. 10, e201 / October 7 2008

[doi:10.1371/journal.pmed.0050201]

Summary

The current system of publication in biomedical research provides a distorted view of the reality of scientific data that are generated in the laboratory and clinic. This system can be studied by applying principles from the field of economics. The “winner's curse,” a more general statement of publication bias, suggests that the small proportion of results chosen for publication are unrepresentative of scientists' repeated samplings of the real world.

The self-correcting mechanism in science is retarded by the extreme imbalance between the abundance of supply (the output of basic science laboratories and clinical investigations) and the increasingly limited venues for publication (journals with sufficiently high impact). This system would be expected intrinsically to lead to the misallocation of resources. The scarcity of available outlets is artificial, based on the costs of printing in an electronic age and a belief that selectivity is equivalent to quality.

Science is subject to great uncertainty: we cannot be confident now which efforts will ultimately yield worthwhile achievements. However, the current system abdicates to a small number of intermediates an authoritative prescience to anticipate a highly unpredictable future. In considering society's expectations and our own goals as scientists, we believe that there is a moral imperative to reconsider how scientific data are judged and disseminated.

Full Text Available At:

[http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0050201]

Box 1. Potential Competing or Complementary Options and Solutions for Scientific Publication

• Accept the current system as having evolved to be the optimal solution to complex and competing problems.
• Promote rapid, digital publication of all articles that contain no flaws, irrespective of perceived “importance”.
• Adopt preferred publication of negative over positive results; require very demanding reproducibility criteria before publishing positive results.
• Select articles for publication in highly visible venues based on the quality of study methods, their rigorous implementation, and astute interpretation, irrespective of results.
• Adopt formal post-publication downward adjustment of claims of papers published in prestigious journals.
• Modify current practice to elevate and incorporate more expansive data to accompany print articles or to be accessible in attractive formats associated with high-quality journals: combine the “magazine” and “archive” roles of journals.
• Promote critical reviews, digests, and summaries of the large amounts of biomedical data now generated.
• Offer disincentives to herding and incentives for truly independent, novel, or heuristic scientific work.
• Recognise explicitly and respond to the branding role of journal publication in career development and funding decisions.
• Modulate publication practices based on empirical research, which might address correlates of long-term successful outcomes (such as reproducibility, applicability, opening new avenues) of published papers.

[http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0050201#journal-pmed-0050201-box002]

>>>Extended Version<<<

The Market for Exchange of Scientific Information: The Winner’s Curse, Artificial Scarcity, and Uncertainty in Biomedical Publication

[http://medicine.plosjournals.org/archive/1549-1676/5/10/supinfo/10.1371_journal.pmed.0050201.sd001.pdf]

Guest Blog

More Evidence on Why We Need Radical Reform of Science Publishing / Richard Smith

PLoS Medicine invited Richard Smith, former editor of the BMJ and current board member of PLoS, to discuss an essay published this week by Neal Young, John Ioannidis and Omar Al-Ubaydli that argues that the current system of publication in biomedical research provides a distorted view of the reality of scientific data.

"For me this paper simply adds to the growing evidence and argument that we need radical reform of how we publish science. I foresee rapid publication of studies that include full datasets and the software used to manipulate them without prepublication peer review onto a large open access database that can be searched and mined. Instead of a few studies receiving disproportionate attention we will depend more on the systematic reviews that will be updated rapidly (and perhaps automatically) as new results appear."

[http://www.plos.org/cms/node/409]

News Coverage

The Economist [10-09-08] : Scientific Journals: Publish and Be Wrong

"Dr Ioannidis made a splash three years ago by arguing, quite convincingly, that most published scientific research is wrong. Now, along with Neal Young of the National Institutes of Health in Maryland and Omar Al-Ubaydli, an economist at George Mason University in Fairfax, Virginia, he suggests why."

>>>With Comments<<<

[http://www.economist.com/science/displaystory.cfm?story_id=12376658]

Newsweek [10-06-08] : Don't Believe What You Read, Redux / Sharon Begley

"Bottom line: when it comes to 'the latest studies,' take what you read with a grain of salt.

[http://www.blog.newsweek.com/blogs/labnotes/archive/2008/10/06/don-t-believe-what-you-read-redux.aspx]

Related

Why Most Published Research Findings Are False / John P. A. Ioannidis / PLoS Med. 2005 August; 2(8): e124 / August 30 2005

[doi:10.1371/journal.pmed.0020124]

Summary

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

[http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0020124]

The Wiki: An Environment For Scholarly Conversation and Publishing

The Wiki: An Environment For Scholarly Conversation and Publishing

Gerry McKiernan, Associate Professor and Science and Technology Librarian, Iowa State University Library

Ames Iowa USA

Science in the 21st Century:

Science, Society, and Information Technology

The Perimeter Institute for Theoretical Physics

Waterloo, Canada

September 09 2008 / 18:00 – 18:20

"The Medium Is The Message ... The Audience Is The Content"

Marshall McLuhan. Understanding Media: The Extensions of Man. New York: McGraw-Hill, 1964.

A "wiki is a ... collaborative space ... because of its total freedom, ease of access, and use, [and] simple and uniform navigational conventions ... ." "[It] ... is also a way to organize and cross-link knowledge ..." Ward Cunningham, Father of The Wiki (Leuf and Cunningham, 2001, 16). Most wikis provide the user with a set of navigation or utility tools such as the ability to create and edit a page, view recently changed pages, and rollback to previous page versions. In addition, many wikis include a discussion forum for proposed page changes.

Among its many perceived benefits are its potential for facilitating a more creative environment and expanding knowledgebase, and a significant ability to harness the power of diverse point-of-views in creating collaborative works.

In this presentation, we will speculate on the Wiki as a digital environment that not only supports current scholarly practices, but more importantly, offers a framework for their enhancement and transformation.

[http://www.public.iastate.edu/~gerrymck/Science21StCentury2008.ppt]

A/V For Presentation Available
Windows Presentation, Windows Video File, Flash Presentation, MP3, PDF

==>I Recommend The Flash Presentation<==

>>[http://pirsa.org/08090056/]<<

*****

BTW: May I Recommend A Most Excellent Bed & Breakfast

If/When You Visit Waterloo

[It's The Home of Research in Motion / BlackBerry]

Sugar Bush Guest House B&B
[http://www.sugarbushguesthouse.com/]

*****

BTW-2: The Wiki: An Environment for Scholarly Conversation and Publishing is based upon

“Wikis: Disruptive Technologies for Dynamic Possibilities” PowerPoint presentation delivered at Digital Libraries à la Carte: Choices for the Future, Tilburg University, The Netherlands, August 23, 2005

[http://www.public.iastate.edu/~gerrymck/TICER2005.ppt]

(accessed 21 September 2008)