Guidelines for reviewing an open textbook

I’m not sure that there is much of an immediate ‘market’ for these posts on issues around developing an open textbook (or maybe everyone is sensibly enjoying the summer), but for those who may in the future be contemplating self-publishing a textbook, I though it may be helpful to draw on my experience in authoring both commercial and open textbooks to lay out some guidelines for reviewers of open textbooks.

The issue

I discussed in my previous post the need for independent reviews of a self-published open or academic textbook, and the criteria I used in selecting reviewers.

Commercial publishers, when commissioning reviewers, usually send a letter or a standard document that sets out guidelines for reviewing a book in its first, full draft before printing and distribution, to ensure both consistency between reviewers, and to identify to reviewers what the publisher is looking for. Although sometimes the publishing editor will require responses to elements that are specific to a particular book, there are also a number of guidelines that are pretty generic.

The situation is somewhat different for a self-published textbook, where it is the responsibility of the author to decide whether to get independent reviews and if so, to provide appropriate guidelines to the reviewers. At the same time, many guidelines will be similar for both types of book, but there are also some aspects of open publishing that require specific guidelines. I have outlined in blue below those that are specific to open textbooks or to my book in particular.

Of course, many reviewers will have their own criteria in assessing a textbook, and they are to be encouraged to use such criteria and make them explicit in the review. At the same time, it is my experience that most reviewers welcome guidelines as to what to comment on, and this is particularly true for an open textbook.

I contacted BCcampus, which obtains independent reviews of all its open textbooks before making them available, and they provided me with a set of questions for reviewers, and I have added some of my own.

Target audience

It is important first of all for the author to be clear as to the primary audience that is being targeted by the book. In my case, it was faculty and instructors in post-secondary education wishing to ensure that their teaching is relevant to the needs of contemporary learners and students. In another case, it may be first year undergraduate students. So one general question for reviewers is:

To what extent is the book successful in meeting the needs of its primary market?

Other questions for reviewers

1. Does the book meet the requirements of a scholarly work? Is it research and evidence-based, and does it provide a critical analysis of the key issues in the field?
2. Does it provide evidence-based, practical guidelines for faculty and instructors that will help them improve their teaching?
3. Does it cover adequately the main contemporary issues in teaching in a digital age?
4. Is the book well written? Does it read well? Is it well organized and structured? Are there errors of grammar or serious typographical errors? Are the graphics and cases appropriately chosen?
5. What major changes, if any, are needed before you can recommend this book? What minor changes would you like to see?
6. If this book was to be offered to a commercial publisher, would you recommend it for publication?

Question 6 may seem a little odd, but my aim here is to ensure that the book meets the same standards as commercial publishing, where there is the added risk of financial loss for a commercial publisher if there is no market for the book, or if the book is not good enough to attract new readers over a period of time. While these risks do not apply to free, open textbooks, the fact that it is judged suitable for commercial publication will carry weight with those looking to ensure that the book meets quality standards.

There may be other questions or guidelines that will be specific to your book that you may want feedback on.

Practical considerations

I specified a length for the reviews of between 800-1,500 words, and that the review would be covered under a Creative Commons CC-ND license. This means the reviews cannot be changed without permission of the writer of the review. However, reviewers would be free to publish the same review in an academic journal, if they wished, and the review could be re-used by, for instance, the author for marketing purposes (but unedited).

I sent out invitations to reviewers within two months of the full publication of the book. Ideally, on hindsight, the invitation should go out almost immediately after full publication, but not before, as it is important for reviewers to see the whole book in context. I gave a suggested deadline of two months to do the review.

I did not offer a fee for the review, but a small fee may be appreciated, as it is a substantial piece of work if the review is done properly.

I am waiting until all three reviews are submitted before posting them, so the reviewers will act independently and not be influenced by someone else’s review.

Over to you

Do you feel that this process (including selection of reviewers, as covered in the previous post) ensures the same degree of independence and quality of peer assessment as you would find for a commercially published book? If not, what suggestions do you have to improve the process?

Even if this process was followed, do you think that there will still be concerns about adopting an open textbook or referencing it in student work?