Formative peer review

Back in 2013 I wrote a long blog post on peer review, summarizing what I had heard at a few conferences on the subject of scholarly evaluation.

Since then, of course, the discussion has evolved and a few philosophers have weighted in. The topic of revisions and alternatives to peer review has also surfaced as part of the Swiss National Foundations Open Access Strategy.

Last year a new philosophy journal, the Public Philosophy Journal (PPJ) has started with a new peer review process, they call it “formative peer review“. This takes an interesting direction. (I have links to the journal and all other references below.)

In my post from 2013 I spent some time laying out the different kinds of data collected in evaluation metrics (citation data, behavioral data and attention data -the latter two often referred to as “altmetrics”) and the new technical possibilities of new peer review systems with the move to digital publications (open peer review, community participation at some level of the review process, or post-publication reviewing altogether).

I did refer to some critical opinions on the blind peer review system as well as on evaluation metrics. The starting point of the Public Philosophy Journal’s editors is clearly a critical attitude towards both metrics and blind peer review – and even academic culture as a whole.

Let’s start with metrics. As mentioned before, metrics whether based on citation data, behavioral data (downloads, time readers spend engaging with your text etc.) and attention data (e.g. twitter mentions) is a way to evaluate a work without engaging with it, without reading it. It is therefore used by funding agencies and non-peers. With academia.edu, of course, there comes in another use which one might critically qualify as narcissistic, or on a lighter tone one could note how it speaks to our “gaming instincts” (and also triggers gaming behavior). Currently academia.edu tries to sell you the data for a premium price you pay in order to satisfy your narcissism or play the game. But it is probably only a question of time until academia.edu metrics are used by funding bodies as well. Metrics is just the technical cornerstone of the academic mirror of current neo-liberal attention-economy. Still, metrics might indicate scientific value. But only if popularity of a text or a researcher indicates scientific value more than marketing success. ( (The publication in a scientific journal with the highest score in altmetrics in 2016 was by then president of the U.S. Barack Obama on healthcare reform. So it measured the influence of the author probably more than the text’s scientific relevance – of course, this is an extreme example (taken from Martina Franzen, reference below)). To sum up the dangers with Christopher Long, “Scholarly metrics […] incentivize clickbait scholarship”.

But let’s now move to blind peer review. There have long been advocates of moving to open peer review, breaking the “veil of anonymity” (Claire Skea). And there have been implementations, especially in newer Open Access publications which did not already have a reputation to risk by moving to a new system of review.

There are some problems with anonymity, but also some advantages. Sometimes anonymity invites poor quality, hasty reviews with exclusive focus on criticism and evaluation. This is unsatisfying foremost for authors, but arguably even for reviewers who would intrinsically perhaps prefer to go into details and deliver constructive criticism. But under the veil of anonymity there is not much incentive to invest much time into the reviewing. On the positive side, anonymity might protect reviewers from retaliation and thus cancel the effects of hierarchy and power in academia for the sake of scientific quality. Imagine a graduate student reviewing a paper by one of the main scholars on her PhD-subject. Negative criticism might impede on her career-chances.

Instead of simply weighing pros and cons of blind vs. open peer review, the editors of the Public Philosophy Journal take a somewhat more holistic approach to the question. They don’t just want to find the most effective method. They want to champion a different academic culture, less based on competition and evaluation and more on collaborative relationships.

How does it work? In formative peer review an author of a draft selects a peer reviewer to publicly engage with the draft. Then starts a complicated process where another reviewer is selected and where they write two reviews, one private and one to be published with the paper. In the process both the paper and the published reviews are modified in response to mutual criticisms and replies. The direction this is meant to go is that reviewing and submitting become more interesting and more responsible intellectual activities. If I interpret the editor’s phrase correctly that “reviewers are asked to bring their best selves to the process” one of the ideas behind it is that anonymity brings out their worst selves. I am not sure about anonymity in itself, maybe hierarchy brings out our worst selves more than anonymity. And in the traditional blind peer review, reviewers and editors are given anonymous hierarchical powers over authors. So to me non-anonymous collaborative or “formative” review sounds like a very good idea.

Scientific evaluation is a complicated subject, not least because scientific quality is already a complicated subject. It is not clear whether competition and adversality is actually furthering scientific quality as much as the current institutions (of evaluation and quantification) imply. Therefore it is very good to see some criticism of academic culture being put into practice with alternative evaluation- (or collaboration) processes.

Sources:
Franzen, M. ; Joy, E.; Long, C. (2018) Humane Metrics/Metrics Noir. http://dx.doi.org/10.17613/M6WP9T61M

Long, C., (2017). ‘Practising public scholarship’, Public Philosophy Journal, Vol.1, No.1, pp.1-6. http://dx.doi.org/10.25335/M5/PPJ.1.1-1

Skea, C. (2018). ‘The veil of anonymity: the perils of peer review’, Philosophical Musings (blog). https://claireskea.wordpress.com/2018/03/05/the-veil-of-anonymity-perils-of-peer-review/

Posted in scholarly communication | Leave a comment

Meiner e-library

Seit Kurzem findet man auf explore fribourg einige Bände der Philosophischen Bibliothek von Meiner. Es handelt sich um Klassiker, die alle auch in Papierversion in den Unibibliotheken zur Verfügung stehen. Man hat aus dem Uninetzwerk oder mit VPN Zugriff auf die Volltexte. Um pdfs runterzuladen (max.50 Seiten pro Buch) muss man ein Benutzerkonto in der e-library von Meiner erstellen.

Posted in Volltextsammlung - Collection de textes | Leave a comment

zotero: search from zotero in library

If you use zotero as your bibliography-tool, you might have wanted to quickly look up whether the document is available in your library, on amazon etc.

Unfortunately the procedure to add these “look-up engines” is a bit complicated. I have therefore created a file that you place in your zotero installation which gives you a few options: Rero, Crossref, Nebis, Amazon, google-scholar.

engines

Place this file at the following location of your zotero-installation (replace “yourusername” with your username on the operating system):

C:\Users\yourusername\Zotero\locate

or for MacOS

/Users/yourusername/Zotero/locate

Then rename the file to engines.json (delete beforehand the preinstalled file with this name)
Then re-start zotero. In the “locate” menu (an arrow-sign) to the right upper corner of zotero you now have a scroll down menu of the different options: CrossRef, Rero Lookup, Google-Scholar, Amazon and Nebis Lookup.

The NEBIS-search allows also for a quick search in Swissbib.

Posted in Recherche Tools - Outils de recherche | Leave a comment

Guide to writing a bibliography

Following the link below you will find a rather extensive guide to writing a bibliography. Especially for “non-standard” documents like discussion-forums and course material, I find it useful.
For most students and teachers the important thing is to have a consistent format and it is less important to follow one particular standard. Still, it is useful to consult standards in order to be consistent.
https://www.hesge.ch/heg/sites/default/files/infotheque/guide_ref_eng.pdf

Posted in Recherche Tools - Outils de recherche | Tagged | Leave a comment

Historisches Wörterbuch Wörterbuch der Philosophie – Online

Eine neue elektronische Ressource ist ab sofort verfügbar: Das von Schwabe herausgegebene Historische Wörterbuch der Philosophie – Online.
Hier ist der Direktlink:
https://www.schwabeonline.ch/schwabe-xaveropp/elibrary/start.xav#__elibrary__%2F%2F*%5B%40attr_id%3D%27hwph_productpage%27%5D__1510129429975

Es ist ausserdem unter den Datenbanken verfügbar:

http://bcu-guides.unifr.ch/fr/subjects/db

Und im Guide zum Fachbereich Philosophie:

https://bcu-guides.unifr.ch/fr/subjects/philo-de

Posted in Nachschlagen - Référence, Ouvrages particuliers - Einzelwerke | Tagged | Leave a comment

Coverage Oxford Scholarship Online – Philosophy

Our coverage of e-books on Oxford Scholarship Online-Philosophy now extends to 2015 including publications from January 2016.
Access it here (VPN or On-Campus)

Posted in Volltextsammlung - Collection de textes | Leave a comment

Unpaywall – a tool to find open access versions of articles

From April 4th 2017 there is a free tool available as a firefox or google chrome browser add-on which automatically searches for legal open access versions of research articles behind a paywall. Here is a link (you can install the pre-release versions right now):
http://unpaywall.org/welcome

This is fully legal and differs in that respect from Sci-Hub.

UPDATE 4.1.2019

Firstly I need to qualify my remark that Sci-Hub is illegal. In Switzerland it is not illegal to download papers from Sci-Hub.

Also, there is a little user guide to Unpaywall and another tool Open Access Button. It was sent to me from another unifr-library and I have adapted it with examples from philosophy:

ADAPT_extensions_open access button_unpaywall

Posted in Recherche Tools - Outils de recherche | Tagged | Leave a comment

Explore, E-books

Quelques compléments au cours d’intro à la recherche documentaire concernant Explore Fribourg:

Dans le cours je recommande Rero Explore Fribourg pour la recherche d’articles de revues électroniques et pour les e-books. Je précise toutefois qu’un résultat négatif sur Explore ne signifie pas automatiquement que le titre n’est pas disponible à Fribourg. En effet, pour les e-books, les résultats qu’on obtient avec Explore sont très inégaux et difficiles à généraliser. Pour cette raison je donne ici quelques compléments, en partie simplement pour illustrer à quel point il faut essayer différentes pistes. Ceci est dû au fait que, contrairement à un catalogue classique, Explore travaille avec les données des éditeurs qui ne sont pas systématisées entre les différents éditeurs.

  • 1. Il est facile de trouver une version électronique d’un Cambridge Companion par son titre. Il faut faire attention de ne pas chercher dans Rero Explore (tout rero), mais bien dans Rero Explore Fribourg.
  • 2. Il est plus difficile de trouver une entrée dans un Cambridge Companion. La recherche par l’auteur et le titre de l’entrée n’aboutit pas (en novembre 2016) directement à l’entrée, mais normalement au Companion lui-même.
  • 3. Pour les Blackwell Companions, on peut arriver à l’entrée en tapant le nom de l’entrée plus le mot “Companion”; auteur + titre de l’entrée n’aboutit pas, ni à l’entrée, ni au Companion.
  • 4. Pour les Oxford Handbooks on a du succès avec titre de l’entrée et auteur.
  • 5. Pour l’International Encyclopedia of Ethics (c’est plutôt une ressource électronique qu’un e-book) on ne peut pas trouver une entrée. Mais on trouve l’encyclopédie.
  • 6. Pour les Intelex Past Masters (différents oeuvres d’auteurs, e.g. Wittgenstein, Anselme, Austin, etc.) on peut taper le nom de l’auteur plus “Intelex”.
  • 7. Les e-books catalogués par rero se trouvent facilement. Le lien pointe vers “E-book Central” pour consulter le texte pendant quelques minutes librement, ensuite il est nécassaire de faire un prêt et de télécharger le pdf.
  • 8. Les environ 1100 e-books d’Oxford University Press se trouvent aussi sans problèmes dans Explore Fribourg.
  • Posted in Volltextsammlung - Collection de textes | Tagged , | Leave a comment

    academia.edu, business and open access

    Many researchers are publishing pre-prints at academia.edu. This is the networking platform for researchers certainly most used by members of the philosophy-department at Fribourg.
    But what is academia.edu exactly? What is it’s business model? And how does it compare to self-archiving repositories (such as philpapers or reroDoc)?

    Here is an interesting critical blog-post on the subject by a researcher from LSE (but his personal opinion, not official LSE):

    What does Academia_edu’s success mean for Open Access? The data-driven world of search engines and social networking

    In short, the author explains that the business model is centered around exploiting the data collected from the usage of academia.edu. The data are sold to business R&D departments who would want to know, for example, which topic is trending among scientists in this domain or that. He urges that one should be attentive to a new way (a part from the now fairly known way practiced by the content “providers” Elsevier, Wiley-Blackwell, Springer etc.) scientist’s mostly publicly funded work can be exploited by enterprises at not much additional cost for themselves. Whereas the editor’s business model is centered around providing paid access to content, academia.edu’s (and Elsevier owned Mendely’s) business provides paid access to usage-data.

    Posted in scholarly communication | Leave a comment

    OSO now 1196 e-books accessible in Fribourg

    Our coverage of Oxford Scholarship Online: Philosophy has been extended and we now have every online-publication in Philosophy from Oxford University Press from February 2003 to January 2014. The total of available e-books amounts to 1196.

    Go here -if you are on the network of the University.

    Posted in Volltextsammlung - Collection de textes | 1 Comment