Maximizing the Benefits of Research / Maximiser les bénéfices de la recherche

David Phipps, RIR-York

While busy brokering and building capacity for knowledge mobilization York’s Knowledge Mobilization Unit has also been publishing peer reviewed articles on their practice. These 10 publications (and more are on their way) are posted in the Knowledge Mobilization community of York’s institutional repository.  This new monthly post will feature a different article and the accompanying ResearchSnapshot clear language research summary.

Tout en travaillant au courtage des connaissances et au renforcement des capacités de mobilisation, le service de MdC de York a publié, dans des revues avec comité de lecture, des articles portant sur ses pratiques. Ces dix publications (d’autres suivront bientôt) sont accessibles à partir de la page thématique de la MdC, dans les archives informatiques de York. Cette nouvelle parution mensuelle sera composée d’un article accompagné de sa capsule “FlashRecherche” – un résumé vulgarisé des travaux présentés.

ResearchSnapshot logo

 

Maximizing the benefits of research 

What you need to know. When research is easier to access, it supports closer collaboration between the different groups that are affected by it. Universities, communities, government agencies and businesses can improve their collaboration with other sectors to apply research findings to real world problems and maximize the impacts of research.

What is the research about?  Knowledge mobilization (KMb) and social innovation gets university research into the hands of policy makers, businesses, and community groups. These stakeholders increase the social, economic and environmental impacts of research by using it to improve the wellbeing of people and our planet. Thus, research must speak to different industries and communities to see its effect on the social economy. A stronger social economy can emerge if we work together, finish projects, join knowledge, and set goals. This study explains the relationship between people doing research, people who need that research, and its relevance to society. KMb and social innovation finds ways to collaborate and communicate it to make the world a better place.

What did the researchers do? The authors studied literature and practices in universities, community groups, and the government. They wanted to see how effectively research was being used after it was completed. They reviewed social innovation trends and suggested ways to make research easier to access and understand for these stakeholders.

What did the researchers find? A brief description of research findings allows interested stakeholders to recognize and access the full report quickly. Social Innovation can thrive when we share our research findings and open up communication between different sectors. Knowledge brokers play an important role in KMb. They help stakeholders in different sectors connect with research to improve its impact. The authors also suggested ways to improve communication and collaboration among government agencies, universities, and community groups. These included:

  • Improve KMb strategies to strengthen the impact of research and social innovation;
  • Develop sustained funding programs to help researchers and their community partners collaborate more effectively;
  • Open and increase communication among government, community groups, businesses and researchers;
  • Train and create a community of KMb and social innovation leaders and practitioners and stay connected.

How can you use this research? Businesses may use this research to improve innovation and social enterprise through access to research. Policy makers may consider developing a strategy to improve relations with universities through KMb. Academic researchers may also use this work to leverage investment in their research and maximize social innovation through their findings. Community groups can access research easily and use it to improve current and future programs and services. Community-based research also becomes more accessible to different universities when partnered with universities through KMb.

About the Researchers:  David Phipps is Executive Director, Research & Innovation Services at York University. Naomi Nichols is a Research Associate for York University and the Canadian Homelessness Research Network. Johanne Provençal is the Acting Director, Strategic Initiatives, Office of the Vice President Research Innovation at the University of Toronto. Allyson Hewitt is the Director of Social Entrepreneurship and Advisor of Social Innovation for SIG@MaRS Discovery District located in Toronto, Ontario.

Reference: Nichols, N., Phipps, D., Provencal, J., Hewitt, A. (2013) Knowledge mobilization, collaboration, and social innovation: Leveraging investments in higher education. Canadian Journal of Nonprofit and Social Economy Research, 4(1): 25-42.

The full article is available online here.

Advertisements

Quantitative Evaluation (and a little shameless self-promotion) / Évaluation quantitative (et un peu d’autopromotion éhontée)

By David Phipps, RIR-York

Amanda Cooper (@ACooperKMb) recently released her evaluation of 44 Canadian Research Brokering Organizations. She presents a quantitative method for evaluating the effort of a system of knowledge mobilization.

Amanda Cooper (@ACooperKMb) a récemment dévoilé son évaluation de 44 organisations canadiennes de courtage de recherche. Elle présente une méthode d’évaluation quantitative visant à mesurer les efforts d’un système de mobilisation des connaissances.

Knowledge mobilization struggles with evaluation.  Evaluating an individual instance of knowledge mobilization is feasible with the right base line and pre/post intervention metrics. But rolling that up and evaluating a system of knowledge mobilization (like any one of the knowledge mobilization units in the ResearchImpact-RéseauImpactRecherche network) has so far proven challenging.

So thank you, Amanda Cooper (Assistant Professor in the Faculty of Education at Queen’s University, Kingston, Ontario). Amanda recently posted a report titled “Knowledge mobilization in education: A cross-case analysis of 44 research brokering organizations across Canada”. Amanda developed a quantitative methodology to evaluate the efforts of Canadian research brokering organizations (RBOs). The methodology is based on the evidence about research utilization. We know that people centred methods encourage greater research use than do those based solely on making package knowledge accessible to decision makers. In the words of Sandra Nutley and her colleagues in Using Evidence, “[p]ersonal contact is crucial … studies suggest that it is face-to-face interactions that are most likely to encourage policy and practice uses of research” (page 74).  In Amanda’s methodology points are assigned depending on how the RBO employs products (12 points), events (20 points) and networks (20 points) as well as overall features (20 points). You can see that more points are assigned to people centred methods (events and networks) than are assigned to purely product based methods. How points are assigned is detailed in Appendix B of her report.

Amanda used RBO’s web sites as the data source and scored each of the 44 RBOs on a scale out of 100. Amanda cites ResearchImpact as one of the RBOs but the data she used pulled from York’s Knowledge Mobilization Unit. The Harris Centre, another RIR member, is also included separately as one of the 44 RBOs.

Key Point #1: This is a quantitative methodology that is reliable and reproducible citing satisfaction with the inter-rater reliability testing of the tool and the average intra-class correlation coefficient.

Key Point #2: This method evaluates a system of knowledge mobilization not the efficacy of an individual knowledge mobilization intervention.

Key Point #3: This method measures the efforts of Canadian RBOs. It does not measure impact of the RBOs efforts. That more effective RBO efforts will result in greater impact of those efforts is a testable hypothesis, but it makes sense that this would be the case.

Key Point #4 (shameless self-promotion alert): RIR-York achieved the highest score in this study.

Each with a score of 81%, RIR-York tied with the Fraser Institute and Canadian Education Association as the top performing RBOs. Fraser Institute achieved this score with a budget of $12.8M. CEA achieved this score with a budget of $2M. York’s budget for knowledge mobilization is approximately $250,000. RIR-York accomplished the same effort on a fraction of the budget. The data from the top nine ranked RBOs is presented below.

Rank

 

Organization

 

 

 

Type*

 

 

 

Size (FTE)

Operating

Expenditures

Score on KMb Matrix (%)

1

1.2.1 RI

NfP, university research centre

 

 

Small (3)

$250 000

81

1.2.4 Fraser

NfP, think tank

 

Large (60)

$12,808,690

81

1.4.2 CEA

Memb, network

 

Small (9)

$2,044,892

81

2

1.2.4 AIMS

NfP, think tank

 

Small (5)

$872 234

78

3

1.2.0 CCL

NfP, general

 

Large (77)

$20,583,490

76

1.2.3 The Centre

NfP, issue-based

 

Large (25)

$5,685,000

76

4

1.2.0 TLP

NfP, general

 

Large (74)

$5,293,039

75

1.2.1 HC

NfP university research centre

 

 

Med (11)

75

5

1.2.0 CCBR

NfP, general

 

Med (12)

74

6

1.1.2 E-BEST

Gov, district level

 

Small (6.5)

72

7

1.2.1 CEECD

NfP, university research centre

 

 

Small (9)

69

1.2.2 P4E

NfP, advocacy

 

Small (9)

$537,806

69

1.2.3 LEARN

NfP, issue-based

 

Large (33)

$3,000,000

69

8

1.2.1 HELP

NfP, university research centre

 

 

Large (50)

$7,200,200

67

9

1.1.3 CSC

Gov, standards

 

Large (20)

$3,849,254

65

We need more research like this into the processes of knowledge mobilization, engaged scholarship and community based research. Much of what we know comes from individual studies of individual instances of knowledge mobilization. As these activities become more embedded in institutions and systems we will increasingly need research on these systems and how they create infrastructure to support the individual instances. You can read more on other methods for evaluating the impact of research like Payback and Productive Interactions in a 2011 Special Edition (Volume 20, Number 3) of the journal, Research Evaluation.

Thank you to Amanda for your important contributions to this emerging field.

Knowledge Hypocrites / Les hypocrites de la connaissance

By David Phipps (RIR-York; @researchimpact)

We are all knowledge hypocrites.  Neither researchers nor knowledge brokers practice what we preach. David Phipps (RIR-York) reflects on why this is and cites examples from RIR’s work to illustrate some early attempts to effect change.

Nous sommes tous des hypocrites de la connaissance. Ni les chercheurs, ni les courtiers de connaissances ne pratiquent ce qu’ils prêchent. David Phipps (RIR-York) réfléchit sur les causes et donne des exemples issus du travail du RIR pour illustrer les tentatives entreprises pour changer cet état de fait.

Many of us working in knowledge mobilization are hypocrites. I am a knowledge hypocrite. You are likely one as well.

I have previously blogged about the need for knowledge brokers to base their practice on evidence from research. I also charged KMb researchers to connect their research to KMb practitioners. I was recently speaking to a couple of researchers from KT Canada and raised this with them.  I told them that most of the KT Canada research is not helpful to KMb practice. Many KT Canada researchers produce single studies with little systematic reviews to provide actionable messages to knowledge brokers. Although all KMb/KT researchers advocate that research should be made accessible to practitioners, KT Canada researchers rarely do (an exception being John Lavis’ group at McMaster who create user friendly summaries of their systematic reviews). I compared much KT scholarship to grains of sand. Important in their own right but as a knowledge broker I need to know what the beach looks like not understand every grain of sand.

KT researchers do not practice what they preach.

But just so we share this responsibility equally, most knowledge brokers are similarly hypocritical. We advocate for the use of research to inform practice but we don’t pursue KMb/KT research to inform our own practice. We advocate that using research needs to be part of an organization’s culture. Employers need to create incentives and time to engage in research.  At RIR-York we always try to set aside one day per month to engage with the literature.  With our busy schedules it is always the first day to be co-opted for operational purposes.

Most knowledge brokers do not practice what we preach either. There are a few of problems that underpin this hypocrisy.

1) KT researchers in health consider health service providers and policy makers as their audience.  KMb researchers in education consider teachers, school boards and policy makers as their audience. But knowledge brokers are also audiences for all KT/KMb research. At RIR-York we try to make some KMb/KT research accessible to brokers through the on line KMb journal club. Posting one broker interpretation of scholarly researcher each month, the five journal club posts have received over 2250 views. This illustrates that there is an appetite for broker accessible versions of scholarly research.

2) There are few venues for knowledge brokers and KMb/KT researchers to interact; therefore, we rarely do. One exception is the upcoming Canadian Knowledge Mobilization Forum in Ottawa June 19-20. The conference theme is “bringing the art and science of KMb together”. In my keynote address I shall be challenging the audience to bring KMb science and KMb practice together in an evidence informed way (evidence informed practice and practice informed evidence).

3) Knowledge brokers do not see themselves as researchers, even in a participatory sense. Knowledge brokers have a lot of practice based knowledge to share. We make videos and blogs and tweet and that’s all critical but it doesn’t capture the attention of university based researchers who privilege peer review as their only currency. Taking lessons learned from our community based researcher colleagues we need to think of practice based research in a participatory fashion and collaborate with KMb/KT scholars to develop research evidence that is useful to our practice. Check out RIR-York’s peer reviewed publications posted in York’s institutional repository, and yes, we are developing ResearchSnapshot clear language research summaries of these papers such as the one produced on our 2009 Evidence & Policy article.

We remain knowledge hypocrites. Until researchers receive time and incentives for making their research broadly accessible and knowledge brokers receive time and incentives for accessing that research we shall remain hypocritical.  Well-meaning indeed, but hypocritical. The system won’t change overnight but it won’t change at all if we don’t start to seek out KMb/KT researcher/practitioner collaborations.

Walking amongst Canada’s knowledge mobilizers: lessons and comparisons from a visit to York University Toronto

The following blog story was first published on the Centre for Research on Families and Relationships (CRFR) on January 12, 2012. It is reposted here with permission.

Sarah Morton Co-Director (Knowledge Exchange) reflects on a recent visit to Canada

I was fortunate enough for the ESRC to fund me for a visit to York University, Toronto Canada as part of my PhD studentship looking at how we assess the impact of research. The visit also chimed with my role as Co-Director (Knowledge Exchange) at the Centre for Research on Families and Relationships. This blog reflects on some of by observations about the ways we do KE or KMb on opposite sides of the pond.

David Phipps, Director, Research Services & Knowledge Exchange at York was a wonderful host, who set up many meetings and exchanges with like-minded individuals and made me feel very welcome.

So the language is immediately different. No-one in my networks in the UK uses the term Knowledge Mobilization – and the ‘z’ makes it appear very North American to us. I also enjoyed the term ‘transition into lunch’ used at a conference! But overall there was more to unite than divide the respective communities in Scotland (UK) and Toronto. CRFR’s model of doing KE was still an exemplar, and there was lots of interest in my work on increasing and assessing research impact.

Day one I gave a presentation at the Ontario KMb community of practice – a wonderful way to make connections and set up further meetings. What interested me about the CoP was that it was a real mix of university-based and public/community based KEpractitioners (in health, housing, schools), who all talked the same language around knowledge use. The CoP is large – over 100 members, so 40 can turn up to a meeting at any time, and a fairly high level, knowledgeable discussion of KE issues is possible in that forum. I struggle to think of an equivalent here, although we hope to set up a new KE network from CRFR in the Spring.

David Phipps’ Knowledge Mobilization Unit at York is part of theKE and Commercialisation department, but with a specific decision to make the KMb in social sciences distinctive, especially in terms of engagement with the local community. There are two main members of staff in the unit: Michael Johnny, and Krista Jensen, assisted by project staff, and graduate students at different times.What I found distinctive about their approach compared to my experience in the UK was:

  • A university-wide approach to Knowledge Mobilization
  • Having a help desk at events where community members can ask for assistance from the university – I can’t imagine some universities here offering such a service!
  • Running plain language training and then producing briefings of peer-reviewed research across the university
  • KM in the AM – breakfast community meetings
  • Annual KMb Expo – where community groups engage with the unit
  • Extensive use of social media: blog and twitter @researchimpact (although some groups in the UK are catching up with this and it has inspired me to do more @crfrtweets)

David and I met with some civil servants in the Ontario Provincial Government from across departments to have a fairly informal chat about KMb and government. They have an emerging KMbnetwork within government with some dedicated resources, not unlike the KE unit within the Scottish Government. Many of the issues of trying to work across departments, timing and accessibility of research, and how to show the importance of research in creating impact resonated with concerns in Scotland and the UK.

Other meetings with students in Ben Levin’s department in OISE, with the Children’s Welfare Organisations and with other KEprofessionals in local authority, water board (!) and women’s health added to a very rich and rewarding visit. I even had dinner with my colleague from Edinburgh Sandra Nutley – a rare opportunity to catch up outwith work!

David and I have continued our collaboration, with David giving a keynote presentation at our 10th anniversary National Conference ‘Influencing Society: the impact of social research’; us presenting some joint work on the role of knowledge brokers at the London Conference ‘Bridging the Gap between research, policy and practice’ in December 2011, and working on a joint paper on the same topic. I hope to return to Toronto in 2012, to present my work on assessing research impact and continue this fruitful collaboration.

Sarah Morton