Wednesday, August 23, 2017

A Publication Milestone … and a Puzzle to Mind

By Kai Chan

A couple weeks ago, my 100th paper or chapter was published in the peer-reviewed literature. Why do I feel so contorted?
Kai, contorted, right from the beginning
of my time at UBC (and not always
happy about it!).


The short answer is that this milestone provoked a realization that I’m getting sucked in to a pursuit about which I am deeply ambivalent.
On the one hand, I believe strongly in the value of peer-reviewed publications as a means of fostering crucial learning towards a deeper and broader understanding of life on Earth and how we can sustain it along with human prosperity. When I’m interviewing prospective students to ensure a good fit between us, we talk about the purpose of academic publications. It’s certainly not a perfect system, but I know of no better way to contribute rigorously and reliably to the body of knowledge upon which human society fundamentally depends. If we’re doing research and scholarship that addresses important problems, we should do it with reference to what others have found—acknowledging explicitly how our research builds upon many important contributions from others. It seems fitting and important, then, to also contribute our learning back to that body of literature.
In those same conversations with prospective students, we also discuss the pitfalls of publication-motivated research. It’s a classic case of Goodhart’s Law, where the metrics of academic publishing (the h-index, i-index, impact factors, etc.) have become the targets of an academic career, thus somewhat perverting their utility. These metrics certainly capture some elements of excellence and of scholarship’s contribution to society’s needs. For instance, I’m proud of the role some of my best-cited papers with Terre Satterfield and others (e.g., this one) have played in helping enrich the dialog about culture and values regarding ecosystem services and the environment. But other papers of mine seem to get cited well despite much smaller roles in effecting change.
So success by metrics is not the success I seek. There are plenty of ways to pervert these proxies of academic contribution, for example by realizing success through the h-index, etc., but not achieving true success in advancing and disseminating needed knowledge. There are also endeavours that contribute crucially to society’s knowledge and use of this knowledge, but that yield little progress by these metrics. Much science engagement (public and policy outreach) goes unrecognized that way—more on that to come in future posts.
Those conversations with prospective students usually conclude with an asserted interest in publishing but also in guarding against Goodhart’s Law. My students and I are all committed to a reflective pursuit of academic success that also includes those activities that are important but not necessarily rewarded academically (e.g., engaging with policy makers, writing policy briefs and op-eds, joining environmental and social justice advocacy groups).
After more than a dozen papers published already this year, it seems pretty clear that I’m spending a lot of my time publishing and not nearly enough on my other sustainability-science passions, including CoSphere (a Community of Small-Planet Heroes …, to make it easy to have net-positive impacts on nature).
In my defence, this distribution of time is not a result of my making decisions in a vacuum to write papers and more papers. Every paper and chapter this year except one was led by others, generally my students and postdocs, who need these papers as markers of their excellence. Even the paper and chapter that I did lead were in close partnership with my students and postdocs, and I hope they will serve them well (both are also intended to advance CoSphere). But regardless of how I got to this point, it remains the case that I am spending so much time on the papers themselves that I have little time for CoSphere, or those other engagement activities.
I suspect I’m not the only one feeling this way. From our recent Global Young Academy survey (just submitted) and various conversations, I know that many of us are strongly motivated to ‘better the world’ through our science and engagement. But it seems that despite that motivation, a litany of invisible or barely visible norms and pressures are thwarting these good intentions—at least somewhat—and leading me and my colleagues to spend more time than we might easily justify on the pursuit of metrics of personal acclaim. (It’s clearly different for those seeking to get academic jobs or tenure, who have to play by the rules of the game—but as a full professor, that justification doesn’t apply to me.)
I don’t have any magic solutions, but for myself, I’m going to seek to right my course somewhat by diving into a highly practical applied sabbatical in 2018-9, perhaps in the seat of Canada’s national government.
How about you? Do you feel any unease about your relationship with publishing? Or not? Have you managed to align your passions with your actions? If so, please share your insights—for our sakes!

8 comments:

  1. Thank you Kai. Well stated. Your ambivalence mirrors a struggle of my own. I want the measure of my work to be captured in changes in how we think about and treat the natural world, not citation counts and h-indices. I tell my students to stay focused on impact and that will give them satisfaction at the end of their careers. And yet, I know that promotion and tenure is built on the maximizing of these metrics. And as I grow older and more experienced, I see the pursuit of publication-motivated research as a force that sucks us all in, becoming the end unto itself. It reminds me of the scene in Dead Poets Society where Robin Williams tells his students to rip the introduction of a poetry book out because it advocates an mathematical equation (the Pritchard scale by Dr. J. Evans Pritchard, Ph.D.) for measuring the quality of a poem. In the same way, we are increasingly striving for numerical measures for the quality of knowledge, and ultimately the quality of our life's work; my h-index is bigger than yours. And I watch as awards, accolades, raises and prestige are given out for success in attaining these metrics. Worse, I find that many of my peers and colleagues have developed sharp elbows in their quest for those accolades, often attained by knocking others down. I have no answer other than to strive to live the life you seek and remain true and authentic to yourself and your life's purpose.

    ReplyDelete
    Replies
    1. Thank you so much for this thoughtful comment, Andy! I'd forgotten that great anecdote from DPS. Great to be reminded of it.

      Delete
  2. Hi Kai,
    From this side, I am not that interested in publishing scientific stuff per se, but to communicate it, in an understandable way. Science is important (from my perspective) if it can be communicated...as knowledge, in general. In my case, my personal commitment is to communicate what I am doing or what others are doing (in the fields that I am knowledgeable). I am the editor of a journal on native forests, that tells in simple words why those are important, what research is going on and what are the implications in the policy realm. As in life...everything in balance!

    ReplyDelete
    Replies
    1. Super to hear about the journal, Jennifer--what's the name? An excellent endeavour.

      Delete
  3. Hi Kai, I feel for you. I crossed the same juncture with my 100th paper a few years ago, which invoked the same self-reflection (maybe this is a consequence of our Leopold training?!). It was this reflection that motivated me to pivot from running a lab and writing papers to something new and unknown - launching a University Center that seeks to make our knowledge useful and relevant in conservation practice. My publication rate has definitely taken a hit since launching the Center, but I am ok with that. Once over the 100 paper mark, I feel liberated about this metric, and motivated only to publish my very best work. I admit that there is a small part of me that yearns to remain at the "cutting edge" of my field, but for the most part, my efforts and energies have shifted to the scholarship of figuring out what works in conservation. It sounds like you are on to something similarly exciting with CoSphere. Wishing you well in this and other new endeavors! Thanks for the thoughtful and provocative post.

    ReplyDelete
    Replies
    1. Thank you so much for this, Leah. I think you're right about the Leopold training, and it's great to know that several of us are struggling with the same issues. Your center at ASU is truly inspiring!

      Delete
  4. Great post, Kai! I definitely struggle with this every day, particularly being at the point in my career (without a permanent position) where choosing to strive for the academic route involves an almost single-minded pursuit of publications, and yet with enough experience to know that there is very little clear value for real-world improvement from each paper that goes out. This probably already exists, but it would be interesting to map out the theoretical ideal flow of problem-science-solution-publication-implementation, and then see if there is any evidence that it truly is working this way. In any case, it's always reassuring to know that you're not alone in feeling unsure about this stuff!

    -Andrés Cisneros-Montemayor

    ReplyDelete
  5. Great to think about this important topic Kai! Another strategy is to find ways to make our papers useful to our partners. Academia does carry a lot of weight and NGOs and government agencies or other partners often don't have the resources to conduct research and certainly not the connections to publish it. But a published paper that highlights their efforts and struggles can help them to win grants or partnerships or to attract needed attention. I also feel that the process of doing the research it self can be useful, if we really work with on the ground partners. What we (the as yet established generation) could really use from those already established is help shifting the metrics by which we will be judged such that the work we put into making our research relevant will be valued by those that pay for that research and that might hire us. I feel that so many of my fellow students are doing amazing and useful work that is highly collaborative with on the ground partners.

    ReplyDelete