The ripple effect: Do recommendations made by think tanks make an impact in policymaking?

Introduction

Impact encompasses influencing public policy, including its effect on the public policy-making process, the community, and governance.  So, how does a Think Tank know that the research it conducts is impactful, useful, and, yes, helps contribute to the policy-making process?

To achieve this, think tanks need to understand the components that imply the fact that their research has made an impact, and these include:

1. Have their recommendations made any impact or influenced the policy-making process?

For example, White Papers, analyses conducted by them, and drafting actionable recommendations

2. Have the training programmes conducted by them for Government Officers been practical and useful? 

For example, developing and delivering governance tools, Social Accountability Tools

3. Has their engagement with media been helpful and reached a wider audience?

For example, writing on topical subjects, participating in relevant debates and discussions and publishing their outputs in the print media

4. Have they leveraged their connection with grassroots-level organisations and communities to partner with the Government for implementing projects and programmes?

For example, act as intermediaries between communities and government to translate data from the field to viable policies?

What are the dimensions of Impact?

Generally, most think tanks have a well-defined strategic approach to understanding the types of impact they require to achieve. Broadly, they include:

• Setting agenda: where they create awareness on topical issues to help feed into policies
• Policy impact: Develop evidence-based reports using data to inform and influence laws. Programmes and policies
• Build capacity: conducting customised training programmes for relevant stakeholders
• Academic partnership: sharing experiences and ideas on academic platforms 
• Organising networking and collaborations: moderating joint stakeholder consultations to help in joint actions 

Types of impact

Every think tank should understand the types of impact which are required to be used appropriately, and they include:

• Indirectly making decisions based on what others think or act
• Delayed establishment of facts after several years of advocating
• Invisible-ideal for internal reforms. 

Think tanks should ensure that, in order to measure impact, they need to:

• Have the right metrics and customise them based on the requirement
• Use data effectively by understanding how they relate to your impact metrics
• Disseminate your results with diverse stakeholders in a manner they understand-simple, valuable and relevant. 

In the case of new and young think tanks and organisations, there may be some challenges, which include:

• Being visible since most often policy makers turn towards bigger think tanks and organisations
• Inefficient networking abilitie,s which hamper access to policymakers
• Due to localisation, a limited approach may not help in scalability
• Constraints on resources impeded the use of specialised experts.

While there have been some tested ways to measure impact, think tanks and organisations can also look at new ways to assess impact, and they could include:

• Conduct a stakeholder mapping where the number of meetings and consultations could be recorded
• Introduce a testimonial bank to capture feedback/appreciations from policy makers and stakeholders
• Use digital footprints to track citations, shares and social media analytics
• Identify and track how you have influenced essential aspects in policy making, including debates on budgets, the introduction of new programmes/polices and addressing emergencies.

Opinions from some articles on impact

It is interesting to note that some think tanks have attempted to measure impact based on specific indicators.

Organising Events

In an article titled How on earth do you measure the impact of your events? Caroline Cassidy focuses on how they came up with facts that are important while planning an event, and what are the parameters to measure the impact? They developed an assessment tool to build learning into an event they were planning. They identified four categories to ensure that it aligned with their engagement strategy, which will help to:

• Stimulate new areas to discuss
• Engage with a broader network of stakeholders for diverse discussions
• Encourage networking
• Create awareness

Based on this, they developed scores and, depending on the event, focused on the most critical category to be measured. After organising several events, they noticed some patterns emerging, which helped them identify the gaps they needed to fill when organising future events.

On Think tanks | How on earth do you measure the impact of your events?

In another article, Louise Ball poses four questions related to communication to measure impact, titled “Four Questions to Assess Your Research Communications Impact.” The four questions, which were well designed, include:

1. Did you have a good plan to start with?
   1. Right audience at the right time
   2. Customised content to ensure that the said audience relates to the same
   3. High-quality outputs to hold the interest of the audience
2. Did you reach the right people?
   1. How many people read your outputs, using analytic tools
   2. How many people downloaded your outputs
   3. Region, no. of times
3. Was your work high-quality and useful?
   1. Followed the set Standards and Guidelines while producing an output
   2. Has no readability issues
   3. Structured logically for easy understanding
   4. Grammatically correct
4. How was it used?
   1. Use a feedback method to assess the usefulness of comments
   2. Identify the number of times it was shared or cited
   3. Engage with the audience/readers when required

On Think tanks | Four questions to assess your research communications impact

Engaging with policymakers is another approach to measuring impact. In an article I wrote with Deeksha Rao titled Policy Impact: A think tank’s perspective. We focus on the extent of policy benefits and the cost of implementation, and state that the impact can be classified as either academic or non-academic. Since it is not an easy task to measure impact on policymakers, we developed some questions and answers, and they include:

• Have the recommendations suggested been implemented/considered by the Government?
• Have policymakers acknowledged you as a Knowledge Partner and invited you to discussions/debates on topics related to policymaking?
• Does the media contact you to take your opinion on topical and relevant matters in the areas of policy-making?
• Have citations for your outputs increased in government documents and scholarly articles?
• Has the number of donors increased?

One can also include the number of times you have engaged with Government departments, as well as the number of times donors have accepted your proposals and supported your organisation’s or projects.

On Think tanks | Policy impact: a think tank’s perspective

Conclusion

It is commonly understood that, on the one hand, measuring policy impact can be complex and challenging, and on the other hand, it is important for think tanks to be effective in shaping public policy. Measuring impact also helps think tanks to:

• Establish credibility with stakeholders
• Showcase their research to stakeholders
• Identify areas where their research has made an impact
• Continue engaging with funding agencies and policymakers.

To conclude, impact matters since it shifts policy from being just a statement of intent to being a tool for real change in governance, development and citizen well-being. It also helps make the government accountable and ensures that policies are designed to address problems. Impact also helps identify what works and what does not, enabling the government to be more adaptable.

Finally, measuring impact means measuring trust, effectiveness and accountability in governance, and one needs to understand that a goal starts a journey and impact defines a destination.