Peer-Review Under Review
in [Relativity]
|
From: paparios on 6 Apr 2010 18:35 On 6 abr, 16:20, "Juan R." González-Álvarez <nowh...(a)canonicalscience.com> wrote: > papar...(a)gmail.com wrote on Mon, 05 Apr 2010 10:16:43 -0700: > > > Well, the fact is that in most cases journal indexing, such as ISI, has > > a very relevant part on the academic career of researchers. The impact > > factor of a journal, while created to guide libraries on their journal > > purchases, is now taken as a very good measure of the quality of the > > researcher and, so, it strongly determines that researcher's future. So > > changing any part of the actual peer review process, would severely > > impact all those academic processes. My opinion is that it will never > > happen. > > Maybe, but I was really asking for your opinion about the suggestions done > for improving it. > > Another day, we could talk about ways to implement them > into certain academic rigid structures. > What is the point? 99.9% of all papers in indexed journals are written by researchers working for universities, public and private funded research centers and research laboratories from big companies. The survival of those journal depends on getting enough funds to support their production, which in turn depends on things like impact index (a low impact index journal will not attract many libraries for paid subscriptions). Since, as I pointed out, researchers are measured according as how well their research is considered by their peers, meaning the requirement of publishing in high impact journals, it is quite obvious where these paper will be submitted. You mentioned the case of Schon. It appears that case did not negatively affect the reputation of either Science, Nature or Physical Review journals. Their impact indexes have remain the same. Miguel Rios
From: eric gisse on 7 Apr 2010 00:20 Ken S. Tucker wrote: [snip noise] Nothing left.
From: "Juan R." González-Álvarez on 7 Apr 2010 06:48 paparios(a)gmail.com wrote on Tue, 06 Apr 2010 15:35:56 -0700: > On 6 abr, 16:20, "Juan R." González-Álvarez > <nowh...(a)canonicalscience.com> wrote: >> papar...(a)gmail.com wrote on Mon, 05 Apr 2010 10:16:43 -0700: > > >> > Well, the fact is that in most cases journal indexing, such as ISI, >> > has a very relevant part on the academic career of researchers. The >> > impact factor of a journal, while created to guide libraries on their >> > journal purchases, is now taken as a very good measure of the quality >> > of the researcher and, so, it strongly determines that researcher's >> > future. So changing any part of the actual peer review process, would >> > severely impact all those academic processes. My opinion is that it >> > will never happen. >> >> Maybe, but I was really asking for your opinion about the suggestions >> done for improving it. >> >> Another day, we could talk about ways to implement them into certain >> academic rigid structures. >> >> > What is the point? Given in the works that some people insists on deleting. Richard Smith Editor BMJ [1]: "Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless in detecting fraud." Fytton Rowland [2]: "45% expected to see some changes in the peer-review system within the next five years" Wendy Warr talk at ACS and Mitch [3]: "It can delay publications for months. An editor can make or break a paper by sending it to the author's friends or competitors. Historically biased against women, single authors, etc... It costs reviewers' time (she gave a statistic that 41% of reviewers would like to be paid). Reviewers tend to favor conservative science and not far-out new ideas. Difficult finding qualified reviewers for multidisciplinary work. Basing the quality of a paper on 2 reviewers, basically just 2-data points, is statistically insignificant. As more papers are being submitted the burden for reviewers is increasing." PHILIP WARREN ANDERSON [3]: "In the early part of the postwar period career was science-driven, motivated mostly by absorption with the great enterprise of discovery, and by genuine curiosity as to how nature operates. By the last decade of the century far too many, especially of the young people, were seeing science as a competitive interpersonal game, in which the winner was not the one who was objectively right as the nature of scientific reality, but the one who was successful at getting grants, publishing in Physical Review Letters, and being noticed in the news pages of Nature, Science, or Physics Today." JULIAN SEYMOUR SCHWINGER [3]: "The pressure for conformity is enormous. I have experienced it in editors' rejection of submitted papers, based on venomous criticism of anonymous referees. The replacement of impartial reviewing by censorship will be the death of science." Finally I want to remark that several journal are not avoiding the problems with peer-review but already using improved Peer-review systems to increase the quality of the journals: BMJ, eMJA, ETAI, Psycoloquy... High-quality journal as Nature are already improving their systems and even maintain a blog about the topic [4] I am glad that they report in their recent 2009 post [5]: "The Royal Society of Chemistry's Dalton and Faraday discussion meetings provide a unique mix of traditional peer review coupled with both comment (by peers) and responses from the authors, but require members of a particular research community to assemble at a conference. It is in some ways similar to the grant proposal review process at, for example, the US National Institutes of Health. However, such a process is clearly not a viable option for every one of the vast number of papers submitted for publication. The journal Atmospheric Chemistry and Physics uses a system in which, after initial assessment by an associate editor, manuscripts are posted online for comment. After referee reports are received, these are also posted online with the manuscript along with author rebuttals. If eventually accepted, a paper is formally published in the journal, whereas those that are not remain available (and citable) as online 'discussions'. This differs from the preprint servers Nature Precedings and arXiv because there is an initial assessment of the suitability of the work (based on more than just scope). Perhaps a hybrid system could be the solution. Traditional peer review, and a decision to publish, could be followed by a fixed period in which any interested party could post questions or comments and the authors are given the opportunity to respond — all moderated by an editor — before a final version of the article (including comments and responses) is preserved for the record. This would again require a large change in the habits of the community — authors, reviewers and publishers — and previous experiments with commenting on published papers have been far from conclusive." Because it shares several interesting points done in [6] and ignores systems as ArXiV :-D REF [1] Alternative Peer Review: Quality Management for 21st Century Scholarship. www.public.iastate.edu/~gerrymck/APR-1.ppt [2] The Peer-Review Process, Learned Publishing 15 no. 4 (October 2002): 247-258. [3] http://www.canonicalscience.org/publications/canonicalsciencetoday/20100401.html [4] http://blogs.nature.com/peer-to-peer/ [5] http://blogs.nature.com/peer-to-peer/2009/10/nature_chemistry_on_improving.html [6] http://canonicalscience.org/publications/canonicalsciencereports/20082.html -- http://www.canonicalscience.org/ BLOG: http://www.canonicalscience.org/publications/canonicalsciencetoday/canonicalsciencetoday.html
From: paparios on 7 Apr 2010 08:39 On 7 abr, 06:48, "Juan R." González-Álvarez <nowh...(a)canonicalscience.com> wrote: > papar...(a)gmail.com wrote on Tue, 06 Apr 2010 15:35:56 -0700: > > > What is the point? > > Given in the works that some people insists on deleting. > ... > I am glad that they report in their recent 2009 post [5]: > > "The Royal Society of Chemistry's Dalton and Faraday discussion meetings > provide a unique mix of traditional peer review coupled with both comment > (by peers) and responses from the authors, but require members of a > particular research community to assemble at a conference. It is in some > ways similar to the grant proposal review process at, for example, the US > National Institutes of Health. However, such a process is clearly not a > viable option for every one of the vast number of papers submitted for > publication. The journal Atmospheric Chemistry and Physics uses a system > in which, after initial assessment by an associate editor, manuscripts are > posted online for comment. After referee reports are received, these are > also posted online with the manuscript along with author rebuttals. If > eventually accepted, a paper is formally published in the journal, whereas > those that are not remain available (and citable) as online 'discussions'. > This differs from the preprint servers Nature Precedings and arXiv because > there is an initial assessment of the suitability of the work (based on > more than just scope). > > Perhaps a hybrid system could be the solution. Traditional peer review, > and a decision to publish, could be followed by a fixed period in which > any interested party could post questions or comments and the authors are > given the opportunity to respond all moderated by an editor before a > final version of the article (including comments and responses) is > preserved for the record. This would again require a large change in the > habits of the community authors, reviewers and publishers and previous > experiments with commenting on published papers have been far from > conclusive." > My point of view goes more with the following first part of that very same cite of yours, that you forgot to post: "Perceived lapses in the peer-review process often receive a lot of attention, but the majority of researchers declare themselves satisfied with the system even though they would like to improve it. If it is imperfect or broken, how do we fix it?" First thing is to show it is imperfect or broken. Surveys such as presented in documents like "Peer review: benefits, perceptions and alternatives" (http://www.publishingresearch.net/documents/ PRCsummary4Warefinal.pdf) agree with the opinion that the great majority of researchers do like this peer-review system as it is. Just a few results of that survey: "Peer review is widely supported by academics, who overwhelmingly (93%) disagreed in our survey that peer review is unnecessary. The large majority (85%) agreed that peer review greatly helps scientific communication and believed (83%) that without peer review there would be no control." Miguel Rios
From: "Juan R." González-Álvarez on 7 Apr 2010 16:45
paparios(a)gmail.com wrote on Wed, 07 Apr 2010 05:39:19 -0700: > On 7 abr, 06:48, "Juan R." González-Álvarez > <nowh...(a)canonicalscience.com> wrote: >> papar...(a)gmail.com wrote on Tue, 06 Apr 2010 15:35:56 -0700: > > >> > What is the point? >> >> Given in the works that some people insists on deleting. ... >> I am glad that they report in their recent 2009 post [5]: >> >> "The Royal Society of Chemistry's Dalton and Faraday discussion >> meetings >> provide a unique mix of traditional peer review coupled with both >> comment (by peers) and responses from the authors, but require >> members of a particular research community to assemble at a >> conference. It is in some ways similar to the grant proposal review >> process at, for example, the US National Institutes of Health. >> However, such a process is clearly not a viable option for every one >> of the vast number of papers submitted for publication. The journal >> Atmospheric Chemistry and Physics uses a system in which, after >> initial assessment by an associate editor, manuscripts are posted >> online for comment. After referee reports are received, these are >> also posted online with the manuscript along with author rebuttals. >> If eventually accepted, a paper is formally published in the >> journal, whereas those that are not remain available (and citable) >> as online 'discussions'. This differs from the preprint servers >> Nature Precedings and arXiv because there is an initial assessment >> of the suitability of the work (based on more than just scope). >> >> Perhaps a hybrid system could be the solution. Traditional peer >> review, and a decision to publish, could be followed by a fixed >> period in which any interested party could post questions or >> comments and the authors are given the opportunity to respond — all >> moderated by an editor — before a final version of the article >> (including comments and responses) is preserved for the record. This >> would again require a large change in the habits of the community — >> authors, reviewers and publishers — and previous experiments with >> commenting on published papers have been far from conclusive." >> >> > My point of view goes more with the following first part of that very > same cite of yours, that you forgot to post: I do not forgot it, I cited the relevant part related to the proposals done in another work [6]. You have now snipped the six references and most of the content of my message. > "Perceived lapses in the peer-review process often receive a lot of > attention, but the majority of researchers declare themselves satisfied > with the system even though they would like to improve it. If it is > imperfect or broken, how do we fix it?" > > First thing is to show it is imperfect or broken. This was already done. The same Nature blog (you sniped the link) contains the data. More data is given in the other six references, which you also sniped. > Surveys such as > presented in documents like "Peer review: benefits, perceptions and > alternatives" (http://www.publishingresearch.net/documents/ > PRCsummary4Warefinal.pdf) agree with the opinion that the great majority > of researchers do like this peer-review system as it is. The same text you cite above says: "the majority of researchers declare [...] they would like to improve it." In another of the references that you sniped the [2]. It was found that: "45% expected to see some changes in the peer-review system within the next five years". > Just a few results of that survey: "Peer review is widely supported by > academics, who overwhelmingly (93%) disagreed in our survey that peer > review is unnecessary. The large majority (85%) agreed that peer review > greatly helps scientific communication and believed (83%) that without > peer review there would be no control." Which agrees with my point. This thread is about IMPROVING peer-review not about eliminating it. This has been emphasized many times here... -- http://www.canonicalscience.org/ BLOG: http://www.canonicalscience.org/publications/canonicalsciencetoday/canonicalsciencetoday.html |