Attribution – Is It Really a Big Deal? / L’Attribution – faut-il en faire un cas?

David Phipps, RIR-York

Many people worry about attribution.  How much influence did we have on an outcome? Many people except York’s Knowledge Mobilization Unit, that is. This post starts some thinking that arose with the help of the Knowledge Brokers Forum… and some time by a pool.

Nombreux sont ceux qui s’inquiètent à propos de l’attribution. Quel degré d’influence avons-nous eu sur un résultat? De nombreuses personnes, mais pas au sein de l’Unité de mobilisation des connaissances de York, cependant. Ce billet présente une réflexion qui a surgi grâce au Forum des courtiers de connaissances… et au temps passé sur le bord de la piscine.

Attribution [John Mayne, CDN J. Prog. Eval. (2001) 16(1): 1-24] is the degree to which research as well as other inputs informs a decision. If an impact happens a long time after the research is completed then it is harder to attribute impact to the research study. In a networked and complex environment there are multiple inputs into any decision confounding the ability to attribute impact to a particular study. This is generally agreed to be true. Or is it?

On November 18, 2012, I began a discussion on the Knowledge Brokers Forum listserv:

*******************************************************

“A colleague and I were discussing the issue of attribution in knowledge exchange and I thought I would read up a little about it. So I turn to my usual sources when I want to read someone else’s work instead of do it myself (!):

  • Nutley et al: Using Evidence
  • Bennet & Bennet: Knowledge Mobilization in the Social Sciences
  • Strauss et al: Knowledge Translation in Health Care

“Attribution” is not to be found in any index in these books.

A search in google or google scholar isn’t much help because the word “attribution” usually comes up as part of a Creative Commons license.

So I am wondering… is attribution really a big deal or do we talk about it without much of an evidence base (for more on this see my earlier blog Knowledge Hypocrites)?”

*******************************************************

I received 30 comments from around the world – for detail on comments see the compilation of comments- Attribution: KBF Responses

In York University’s Knowledge Mobilization Unit we rarely see attribution as an issue. As described in a recent book chapter our work has helped inform the cooling policies for the City of Toronto, a new funding program for United Way of York Region, a sustainability program for rural businesses and a new way of delivering immigrant settlement services in York Region as examples. When we are able to demonstrate non-academic impact of our work it is often directly attributed to the collaboration we supported. When speaking of the Green Economy Centre, Valerie Ryan, CEO of Nottawasaga Futures, said, “We could not have done this without the Knowledge Mobilization Unit”.

So what’s the big deal about attribution?

I think the role of a particular actor in knowledge mobilization/research utilization is important – see table below:

Actor Does it matter Why or Why Not?
Academic research institution No In a REF world institutions will create good news stories about any impact, no matter how small, no matter how little research contributed to the impact.
Researchers Maybe Depends on how the researcher is being measured which might depend on if they are employed by a university or a think tank or a community based researcher.
Think Tank/NGO Yes Think Tanks and NGOs are often funded based on ability to demonstrate impact and the attribution of their efforts.
Funders Maybe Private donors want to understand what impact their funding made. Academic research funding councils are beginning to care about impact but they are concerned with narratives more than attribution.
Knowledge brokers No If the goal is to broker successful collaborations or access to knowledge then the impact arising from those collaborations is nice but anecdotal.
End users No They got an impact. Attributing that to research or any other input is less relevant than the impact itself.
Evaluators Yes This is what they do. They are employed by those who care about impact and attribution.

Academic institutions routinely deal with attribution in technology transfer and commercialization. When an academic institution grants a license to a patent the attribution issue is dealt with up front by agreeing on a royalty rate. The royalty rate acknowledges there will be other inputs en route to market; however, the academic institution will create a REF case study regardless of a small or large royalty rate.

But that’s not the whole story.  A KB Forum response from Larysa Lysenko connected me to an article from Annette Boaz and colleagues. Only a portion of the article discusses attribution and in that discussion were two little sentences:

“The process by which the research is done might also have an impact on policy. For example, research in collaborative projects may have an impact prior to the production of research outputs.”

This was my “Aha Moment”. As described in a recent paper, York works almost exclusively in a co-production paradigm where the decision maker partner is an active participant in the research endeavour. The decision making process for our research partners occurs continuously during and after the collaboration. Decisions may be informed during the research process, not solely after the research is concluded.

The Knowledge Brokers (No – above) in York’s institutional (No – above) Knowledge Mobilization Unit promote a co-production method in which impacts may precede outputs. Attribution is therefore not a big deal for us. This certainly doesn’t deny the importance of Attribution in other contexts and the role of methods such as contribution analysis and productive interactions to address the issue. It’s just not a big deal for us.

And I was sitting by a pool in Ft. Lauderdale reading Annette’s paper when I had my “Aha Moment” underscoring the need to get away and just think.

David showing off his ResearchImpact T Shirt in Ft. Lauderdale

David showing off his ResearchImpact T Shirt in Ft. Lauderdale

Advertisements

How Are You Using Evidence?

By David Phipps, RIR York

My cat GoGo likes to interrupt my work. Whether she is sitting on me or sitting on my papers or competing with my laptop for my lap she is always interrupting my work.

But in this picture (below), GoGo also reminds me how valuable Using Evidence continues to be.

Sandra Nutley and her colleagues Isabel Walter and Huw T.O. Davies published their book, Using Evidence: How research can inform public services, in 2007. There has been TONS written on the broader subjects of knowledge mobilization, research utilization, implementation science etc since then but this book is truly a foundation for our work, especially for those of us working in social services and social policy. The book covers early concepts and hypotheses to contemporary approaches to enhancing the impact of research on policy and on services.  I used Sandra’s book in a recent Open Access book chapter as part of the literature review (why do it myself when she has already done it for me). In that chapter, my co-authors Krista Jensen (@atomickitty) and Gary Myers (@KMbeing) and I articulated three take home lessons from the literature (there are more, but these were the three important to us at the time):

  1. KMb is a social process
  2. Efforts to enhance KMb need to be interactive and focus on the relationships between researchers and decision makers
  3. KMb happens at the level of the individual and is only beginning to emerge at the organization and the system/sectoral level

Even now, six years later KMb is still only emerging at institutional and systems levels. This underscores the currency of Using Evidence.

All that to say, if you haven’t read the book do so. It’s 33 pages of references means you don’t have to go anywhere else to find relevant literature up to 2007. It is foundational for our work. You can even get the book on Amazon.

I shared this picture with Sandra with whom I collaborated on a paper earlier this year. Thanks to our co-author Sarah Morton (@CRFRtweets) for writing about our article.

So let us know… how are you Using Evidence?

Using Research to Influence Family Services and Policies

The following was first published on Centre for Research on Families and Relationships’ (CRFR) blog and is reposted here with permission.
CRFR co-director Sarah Morton and colleagues Sandra Nutley (Director of the Research Unit for Research Utilisation, www.ruru.ac.uk) and David Phipps (Director of Research Servics & Knowledge Exchange at York University, Canada) offer seven lessons, and associated challenges, to improve how research is used in policy and practice in a recent article in the new journal Families Societies and Relationships.
  1. Set realistic ambitions and expectations: Research is one form of knowledge that policy makers and practitioners will be using, but it is rare for research to have the definitive word.
  2. Improve research strategies to ensure they address relevant issues and expand our knowledge base rather than unwittingly replicate existing studies. Reviewing research and evaluation processes helps to ensure that research responds to relevant issues and address the main knowledge gaps.
  3. Shape – as well as respond to – policy and practice debates: Take up opportunities to influence policy and practice debates when they appear, – rather than waiting for opportunities to open up, work with advocacy organisations to raise issues of concern and get debates going.
  4. Create dialogue around research by pulling together different perspectives: Research on its own does not create change, but it can influence it. Encourage dialogues between people that recognises research needs to interact with practice experience and tacit knowledge.
  5. Recognise the role of dedicated knowledge broker organisations and networks: There are increasing numbers of knowledge broker organisations and networks who can help to facilitate the creation, sharing and application of research-based knowledge.
  6. Target multiple audiences to increase the reach and impact of your message: Disseminate research findings into wider political and public debate, alongside more targeted approaches. This might be targeting influential people, participating in media debates, speaking at policy and practice conferences and seminars or responding to consultation processes.
  7. Evaluate, learn, improve: Knowledge exchange is still an immature discipline; only through improved evaluation and learning will our understanding of effective strategies develop over time.
Don’t forget that challenges remain:

Social Media for Knowledge Mobilization / Les médias sociaux au service de la mobilisation des connaissances

David Phipps, RIR – York

Blogs, presentations on Slide Share, videos on you tube, twitter, Delicious bookmarks, Linked In but not facebook. These are our social media tools and now we have published a book chapter reflecting on these tools and their application to knowledge mobilization.

Blogues, présentations sur SlideShare, vidéos sur YouTube, signets Delicious, LinkedIn (mais pas Facebook), voici les outils que nous utilisons sur les médias sociaux. Nous avons publié un chapitre de livre qui présente ces outils ainsi que leurs applications pour la mobilisation des connaissances.

A few weeks ago Krista Jensen, David Phipps (both from RIR-York) and Gary Myers (www.kmbeing.com) published a book chapter titled “Applying Social Sciences Research for Public Benefit Using Knowledge Mobilization and Social Media”. This was published by the open access publisher, In Tech, in a book titled “Theoretical and Methodological Approaches to Social Sciences and Knowledge Management” that was edited by Asunción López-Varela.

The chapter didn’t attempt to review the literature since this is a large body of work and has been done elsewhere. The chapter started out with these elsewheres by presenting three perspectives on knowledge mobilization: 1) Knowledge To Action cycle (Ian Graham and colleagues in Ottawa); 2) Collaborative Entanglement (Bennet & Bennet) and 3) Research Use by Sandra Nutley and colleague from the Research Unit for Research Utilization, University of Edinburgh. We challenged each of these three perspectives but chose them because they each built on the other conceptually drawing the reader into deeper and more contextualized understandings of the subject but concluded that there were three take away messages from these literature reviews:

  • KMb is a social process
  • Efforts to enhance KMb need to be interactive and focus on the relationships between researchers and decision makers
  • KMb happens at the level of the individual and is only beginning to emerge at the organization and the system/sectoral level

Word cloud of key words in profiles: Followers of RIR

These three messages were then illustrated by citing the practices of Institute for Work and Health and PREVNet, both of whom promote the use and uptake of research into policy and practice. In addition to these two Canadian networks these principles were illustrated by three international examples of university based knowledge mobilization activities: Community University Partnership Program of the University of Brighton, Research in Action Project of the Institute for Health Policy at the University of Texas and the Centre for Families and Relationships at the University of Brighton.

The chapter then presents in detail the knowledge mobilization services of York’s Knowledge Mobilization Unit. Drawing on our “recipe book” published in Scholarly & Research Communications in December 2011 we go further to present some success stories arising from our work: United Way York Region Strength Investments, Parkdale Activity & Recreation Centre Heat Registry, Green Economy Centre of South Simcoe and Evaluation of the Inclusivity Action Plan of the Regional Municipality of York.

Continue reading

Sandra Nutley – In Her Own Words / Sandra Nutley – dans ses propres mots

David Phipps, RIR-York 

Sandra Nutley has written and will continue to write seminal works on research use. Read her articles and her books but also attend a presentation she makes because it is there that she distills her many words into just a few.  And she makes them all count.

Sandra Nutley a écrit et continue d’écrire des travaux de référence sur l’utilisation de la recherche. Ne manquez pas de lire ses articles et ses livres, mais surtout, ne perdez pas l’occasion d’assister à ses présentations. C’est à ces occasions qu’elle synthétise sa pensée complexe en quelques mots, chacun d’un vaut son pesant d’or.

I have had the pleasure of hearing Sandra Nutley speak on numerous occasions in the UK and in Canada. I have read nine of her papers, working papers and even a forthcoming book chapter.  Of course, I have read her book, Using Evidence that she wrote in 2007 with Huw Davies and Isobel Davies. In fact I have purchased over 30 copies of her book since it has been published using them in workshops and meetings.

Her writing is on the must read list for all knowledge brokers. Even though her writing is fairly accessible, Sandra writes for academics not for practitioners because Sandra is an academic. That’s where going to hear her speak is a wonderful complement to reading her scholarship. Most recently I shared the stage with Sandra at the 10th Anniversary conference of the Centre for Research on Families and Relationships at the University of Edinburgh. Sandra presented straight forward advice that was accessible to scholars, practitioners and partners from the policy and community sectors. She presented seven lessons for those seeking to enhance the use of academic research by decision makers. Thank you to Sandra for permission to reprint these which we reproduce with reflections on our knowledge mobilization practice at RIR-York.

Nutley’s Lesson RIR-York Reflections
Set realistic ambitions and expectations about research use York’s KMb Unit promises to use best efforts to connect researchers to decision maker partners. We do not promise to deliver the best evidence to inform decisions.  That is up to the collaboration.
Improve supply of relevant, accessible and credible evidence, but don’t stop there York translates academic articles into ResearchSnapshot clear language research summaries and publishes these in an on line searchable database. These serve as a starting point for knowledge brokering, they are not an end in themselves.
Shape as well as respond to the demand for evidence in policy and practice settings (consider working with advocacy organizations) We work closely with the United Way of York Region to build community capacity for engaging in research and create a culture of collaboration between the university and community partners.
Develop multifaceted knowledge exchange strategies, more than just packaging and knowledge translation (players and processes are more important than the products) As above, ResearchSnapshots are only the starting point for knowledge brokering. York’s KMb Unit then uses user pull (research translation help desk, Lunch & Learn), knowledge exchange (KM in AM, Research Forums) and co-production (interns, social media) methods to support the co-production of knowledge by researchers and their decision maker partners.
Recognize the role of dedicated knowledge broker organizations and networks Together with York, our RIR partners (Memorial University of Newfoundland and Labrador, Université du Québec à Montréal, U. Guelph, U. Saskatchewan, U. Victoria) have invested in an institutional capacity for KMb services.
Target multiple voices to increase opportunities for evidence to be part of the policy discourse York’s KMb practice routinely engages the community sector, Regional and municipal agencies, provincial ministries and the hospital sector in conversations with university researchers and students. We are considering how best to include colleges and the private sector in the conversations.
Evaluate knowledge exchange strategies to improve research use and learn from this We did a formal evaluation of the KMb Unit in 2009/2010. We posted the formal evaluation and our response on Mobilize This! and we published a peer reviewed journal article. Our KMb practice is participatory and self reflective, embedding learning as we go.

Sandra summarized her research knowledge in straight forward practice relevant advice presented in clear, concise language (well, “policy discourse” might not be clear language). Thank you Sandra for practicing what you preach and for helping us reflect on our own knowledge brokering practice.

Getting Back to Basics / Retour aux fondements

By David Phipps (RIR-York)

Inspired by a campfire and a few beers, David Phipps (RIR-York) reflects on Using Evidence and looks back to the future of knowledge mobilization.

Inspiré par un feu de camp et par quelques bières, David Phipps (RIR-York) réfléchit à l’utilisation des données probantes et jette un regard sur l’avenir de la mobilisation des connaissances.

I was camping this weekend. I have been camping with this same group of friends on this same weekend for years so rain or shine we go and have a great time.  We have a great time doing almost nothing for three days.  We sit around a fire.  We play games.  We get caught up with friends we don’t see too often. We read. And we have a couple of beers (okay, more than a couple).  This year it rained. Our tents were damp but nothing could dampen our spirits (thank you Bud Light Lime). Because it was raining, I took some time out to work on a book chapter we have been invited to write for an open access book titled Social Sciences and Humanities – Applications and Theories. Part of the chapter is, of course, a lit review so I have been revisiting some foundational KMb literature.  

After a lovely time with Sarah Morton and Sandra Nutley this summer (thank you Gary Myers, @kmbeing for your blog), I took some time this weekend and returned to Using Evidence that Sandra Nutley published in 2007 with her colleagues Isabel Walter and Huw Davies. I got back to KMb basics while getting back to the basics of living in a tent and cooking over a camp fire.

When I first read Using Evidence in 2007 York’s KMb Unit had been operating for one year.  We had just hired our second full time staff person and we had more KMb enthusiasm than KMb talent. In 2011, York is leading ResearchImpact-RéseauImpactRecherche, we have published on our work, spoken internationally and I have met Sandra twice and look forward to seeing her again as we both speak at the CRFR 10th Anniversary National Conference in Edinburgh in November. It is interesting to revisit foundational literature like Using Evidence and see the literature through a lens of experience. I can now synthesize key messages and see the KMb forest where before I saw only a collection of KMb trees.

There are lots of key messages arising from Sandra’s book.  The three I take home after reflecting on our five years of KMb service are:

  • KMb is a social process
  • Efforts to enhance KMb need to be interactive and focus on the relationships between researchers and decision makers
  • KMb happens at the level of the individual but future efforts will explore KMb at the level of the organization/system

There are two implications of these take home messages for York and RIR:

  1. If KMb is a social process then social media tools should be able to contribute to the process of KMb
  2. Systems level KMb need to use interactive methods to support individual KMb relationships

York and all the RIR universities are building KMb services at the institutional level to serve a system of researchers and their (primarily) local research receptor organizations. We have also frequently blogged about the role of social media in KMb (search the “social media” tag on Mobilize This!) and most recently on August 25. In our book chapter we will present some evidence we have collected about how we are using twitter to support a KMb community of practice for KMb stakeholders.

In 2007 we were learning the basics.  Four years later we return to the basics so we can look to the future.  Interactive relationships between researchers and decision makers are the foundation of KMb. In the future we will develop system level KMb supported by social media so that we can continue to build on the basics and better foster those interactive relationships.