-- See? Nobody cares -- few pages currently link to this one.[email me at d.w.corne@macs.hw.ac.uk if you wish to comment and/or discuss including your own additions at the end].
If I am missing something, please let me know and I will add your viewpoint in a suitable place. Most readers will, on seeing examples below, shout "well of course that's outrageous; that goes without saying, surely!?" But, sadly, there are those who excel at such practices, and teach others to do the same. One of my greatest concerns is that I would guess they justify this by suggesting that everyone does it; thereby, slandering you as well as me. One of my other concerns is that some of you may not even believe that these things regularly happen -- e.g. that if an academic has published on or received funds for X, they must have had original ideas, be expert at X, or made a real impact at X. Sadly it happens, to the detriment of others (careers, families) and to the detriment of progress. To avert too much disbelief I have avoided describing some of the most outrageous things that I know occur.
Most of us don't seek or need advice about how to do a good review, and just naturally try to do a helpful, constructive review, using the yardstick "would I find this helpful if it was a review of my own paper?". Generally, be fair, constructive, take the time to understand it, and if you don't, suggest that it could be more understandable and in that case give some benefit of the doubt. Justify your negative points and your positive points, since this helps the decision-makers, as well as gives extra information to the authors. If you haven't much time but want to return the review anyway, then why not say so in the review itself; this might help the authors if they want to question the final decision.
But:
Don't do any of the following. In most cases, these are violations of the trust placed in the peer review system. Reviews are a service to the community, not an opportunity for reviewers to advance their own careers, other than by having their CV reflect that you have been granted this trust.:
Good paper writing has all of these qualities: clarity; logical development and delivery; thoughtful and sensible review of related work; proper evaluation involving (usually) statistical tests; well-balanced conclusions and well-balanced assessment of limitations.
Bad paper writing includes the following features:I have found that younger researchers are in a particularly vulnerable mindset when it comes to their initial proposal reviewing tasks. In my own experience, with no exceptions, their first feeling is to do precisely what is asked of them, and as well as they possibly can. That is, to provide a thoroughly considered and fair assessment of the proposal in front of them. But (I am talking here about a handful of many that I have spent time with, which is much more than just those at my own institution(s)) it often happens that certain unusual aspects of this situation then hit them hard. Viz, "... this is a proposal from a competitor, competing with my group over a small pot of funds. I am in an immensely powerful position here. Can I trust others to do the right thing in this situation? And, come to think of it, there are some issues with their ideas that may be more major than I first thought ...". This is the type of case where the integrity of the research climate immediately around them will have a major effect on what that researcher wlll do (and then continue to do, and eventually train others to do).
The correct practice here is of course abundantly clear: go with your first thought. But if you feel you cannot do an objective job in any particular case, inform the funder that you cannot do the review.
Everything in section 3 also applies here. I have seen examples of all of these (but I hasten to remind any outside peer-controlled science reading this: the norm is one of proper behaviour). But the stakes are higher and improper behaviour in this respect is even worse. I really feel people who engage in the first two of section 3, in the context of proposal reviewing, should lose their funding, jobs, or both.An additional aspect of poor proposal reviewing that seems more prevalent for proposals than papers is the practice of nitpicking. A proposal for funding (if the funders' guidelines differ then of course ignore this) is a statement of new ideas to address important problems, with an approach that seems like it may work well. But if it is not in perfect English, or if you think that 3% of the research ideas are flawed, or if you think there is a better way to do one or two things in the plan, then this is irrelevant to whether or not it should be funded. Sadly, our current system makes it difficult for strong proposals to get funded if even one review nitpicks about such things, especially if this gets (outrageously) translated into a lower overall assessment by the offending reviewer. We are getting better at this, but nitpicking still remains an effective strategy for the few bad apples out there.
There are several guides available on how to write proposals well. Clarity is one of the key points, of course. Meanwhile, practitioners of bad practice in this area generally use methods akin to those in section 4. I have little more to say on this point, other than what I see as a poor practice in which a small number of proposers use funding rather than achievement in their track records. This one is perhaps much more subjective than most, but I find it an abuse of the system. There are two pages available in most proposal scenarios, where those pages are to be used to provide track record information for the reviewer. If I review a proposal to solve problem Y, and the track record of the proposer shows that they have in the past solved related problem X, then that tells me something useful and positive. But if I review a proposal to solve problem Y, and the track record says that the proposer has received £2M for addressing problem X, that tells me less. Presumably if you had solved it, or made any useful increments in science with that funding, you would have summarised and proudly elaborated these achievements. So, I find it much less convincing. The trouble here, in the context of bad practice, is that (again, this one is much more personal opinion -- I don't expect all to agree) `Funds but no achievement' track records generate a vicious circle whereby the funds get bigger and achievements little or no greater, since many reviewers or panels (very wrongly, in my view) assume that funds implies achievement.
Pressure for impact, esteem indicators, and funding can sometimes tempt us to forget ourselves and use opportunities for networking (or maybe just email) as opportunities for unrelenting self-promotion. Even worse, some may be led to believe that this is what conferences and workshops are for. Conferences and workshops are set up for the presentation and/or discussion of research ideas and directions, hopefully with plenty of social time set aside for general gossip and relaxation as well as more opportunity to bounce science around. Naturally, as part of your networking activities, you may be keen to make the acquaintance of X and lead X to discover that you are an excellent authority on something or other, in the hope of some invited esteem indicators eventually emerging from this. You may also be keen to hang around with industry person Y since there could be a chance of obtaining funding from Y's company. That's fine, at least in moderation (otherwise it just becomes really annoying, and probably has the opposite effect). What is not fine is unfairly and/or wrongly dismissing others' work or reputation as a device in order to aid your pursuits. It is common sense that this is wrong; it gives industry a bad impression of us too. Fortunately I have very really seen or heard of this in action, but I know that it is common practice in at least one group in the Northern Hemisphere. As a result, some industry funds get poorly spent, the company involved is less likely to fund academics in future, and so on and so on ... Meanwhile, some journals get questionable editors, who then participate in other questionable practices, which harm science in myriad ways.
A related questionable practice is self-invitation. The general pattern is as follows: one sees news of a conference or journal, in which one has not been invited to be a PC member, session chair, tutorial presenter, editorial board member, and so on. One then contacts the organisers or editor-in-chief and offers oneself for one or more of these roles, perhaps citing much relevant qualification and experience. There is a broad spectrum of acceptability in this sort of practice, ranging from reasonable and forgivable (relatively rare) to outrageous. The problems here are: (1) It seems like an attempt to boost one's CV at the expense of the general credibility of others' CVs. People generally accumulate such invitations (non-self ones) on the basis of others' respect for their published work and/or their known qualities in such roles; hence they receive and accept such invitations, and the volume of such in their CV, weighted by the relative prestige of each item, speaks to the overall esteem with which they are held. But, the more such CV material is gained by self-invitation, the less true this is in general, and strong-looking CVs of non-self-inviters are automatically brought into question. (2) It seems like a salesperson-like manipulation of human nature -- self invitations often have a chance of success simply because the organisers/editors in receipt of them like to be polite, give benefit of doubt, and so forth. It puts such people in a difficult position. Finally, I had suggested, above, that there was a reasonable/forgivable extreme on the self-invitation spectrum. Some of these are cases where the inviter is young, new to the game, and keen to do professional scientific activities, and seems to have gathered or supposed that this is the appropriate way to start to build a CV; that's forgivable, but not right. A potentially reasonable case, however, is when the inviter clearly thinks that they have been left out of a programme committee (say) accidentally, and the motivations for the invite are the right ones (helping the process, and staying in touch with the latest work). I have recently done this myself (so, now wonder I think it's OK)! And, I do not include that particular conference PC membership in my CV/www page. A similar and arguably appropriate case is where the self-inviter is appropriately qualified and experienced, but feels they have not been on the radar for such invitations in the past, but perhaps now have a new job or new freedoms at work, and they make this known to various in the community. Well, it's a very grey area, but the main point is that it is wrong to seek to boost your CV in a way that discredits the true quality of others' CVs. The acid test with regard to self-invitation is, I suppose: would the self-inviter have clearly been invited anyway had the organisers/editors known of their credentials and availability?
Proper research conduct simply means honesty and integrity in all your research-associated activities, including giving due credit to what has been done before, treating the peer review processes with respect by providing substantial submissions and helpful and thoughtful reviews, and accomplishing your peer proposal review activities with zero self interest.
I have observed a small number of bad apples spread some poison, and I have been appalled at the overall effect. Maybe I'm naive, maybe I'm over-reacting, maybe a certain amount of improper conduct is felt acceptable?? What do you think? It is certainly unacceptable on the EPSRC college or on other research council peer groupings, concerning which explicit statements of proper conduct exist. However, that doesn't stop a small number violating them in pursuit of self-interest at the expense of the rest of us.
For any who may be in doubt, I can't pretend that the goodies always win and the baddies will always fall.. Conduct of the improper kind is tempting to some, and can be very successful in terms of personal career, so long as our peer review systems and related structures are imperfect; and, sadly, these systems will always be imperfect. For very similar reasons, organised crime has many attractions and benefits for those involved. It is wrong, don't do it; be a scientist.
-- See? Nobody cares -- few pages currently link to this one.