Metrics
SIAA Initiative

Social Impact Analysts Association .... SIAA Working Group Principles

Burgess COMMENTARY

Peter Burgess

Open file on SIAA Working Group on Principles of Social Impact Analysis ... Mapping Exercise (PDF)

The text of the PDF follows:

• Accountability The acknowledgment of responsibility for actions, including the obligation to be answerable for resulting consequences to those affected
• Approach A particular set of principles or guidelines around how to carry out a social impact analysis (e.g. GRI)
• Attribution The extent to which impacts are caused by an activity
• Impact A change resulting from an activity
• Materiality The process of deciding which impacts should be measured
• Principles The building blocks within approaches to measuring impact
• Scope Covers the purpose of the analysis, level of accountability and activity under analysis
• Stakeholders Groups of people (sometimes as organisations) that are affected by or effect an activity

•  Principles sets that focused on an organisation’s objectives and analysing impact of intended outcomes. For example Charting Impact, ISO, Charity Commission; and
•  Principles sets that focus on analysing the wider impact of the organisation, both positive and negative on the issues it aims to affect. For example GRI, Accounting for sustainability and SROI.

1. 1. Scope

   Principles in this category focus on the clarity of purpose, and understanding the activity being considered through the impact analysis. For example some approaches recommend first considering the key reasons for carrying out the impact analysis and therefore how the results will inform and improve practice. It also encompassed the level of accountability

2. organisations have for the activity.
3. 2. Measurement

   Two different types of language emerged around how to go about data collection and measurement. One focused on identifying and measuring against ‘issues’, while the other 4. focused on measuring and assessing ‘outcomes’. Some of the principles guide how measurement occurs (methods for measurement), whilst others focus on deciding what should be measured (identifying which outcomes occur). Mention of stakeholder engagement in planning evaluation and agreeing methods is prevalent.

4. 3. Interpretation

   This strand focused on how the information available through the measurement was interpreted. It focuses on mitigating risk and ensuring unbiased and rigorous analysis orreview of the results. The principles here concentrate on use of evidence, rigour, transparency and clarity during interpretation.

5. 4. Reporting

   Many of the principles focused more on reporting than measurement or interpretation. Reporting was defined as a summary of the information collected and the resulting insights. We found that this often related to a particular audience, for example policymakers or donors.

   Within reporting there is again a strong emphasis on rigour, use of evidence, balance, completeness and transparency. Only a minority of sets refer to benchmarking and comparability.

   Only SROI and SCBA include a specific principle about assurance and quality checking of assessment. Also, SROI and CBA are the only sets which refer to the financial value of impact.

6. 5. Learning from and use of impact results

   The final strand outlines how lessons learnt from the impact measurement, review and report feed in to an organisations decision making and strategy. This is clearly a priority but guidance was inconsistent across the different principle sets. The review identified a clear overlap between use and learning principles and those which guide the scope of the social impact analysis.

The review found three common approaches as follows:

•  Using results to improve on performance;
•  Using results to make better strategy and policy decisions; and
•  Using results to improve understanding of the outcome itself.

Many more themes were evident in the sets of principles reviewed.

Sustainability, while not identified as a specific strand, appears in many of the principles. Unsurprisingly there are examples in the principles for Accounting for Sustainability, AccountAbility and in the OECD principles under stakeholder inclusiveness. In AccountAbility’s case this refers to the need to involve your stakeholders to ensure your operation remains sustainable and responds to their concerns. OECD refers to the need to 5 include donors and partners to ensure a project’s sustainability if key stakeholders leave. GRI also talks about the ‘sustainability’ context – by which it means not organisational sustainability but ecological sustainability.

A thematic cluster of leadership, governance and responsibility was clear, as well as importance of having a policy on monitoring and evaluation. Also most of the principle sets imply that organisations are accountable for their performance and impact, but only five of the principle sets directly mention accountability.

Organisational review and assessment of the organisation’s capacity to deliver on aims featured in Charting Impact and IIRC principles, but was more widely implicit in the idea of context and completeness. Surprisingly there was little discussion of proportionality.

Same terms, different meanings

Several terms appear repeatedly across different principle sets but underlying meanings varied.

•  There was variety in the requirements for stakeholder inclusiveness
•  ‘Balance’ in GRI is about being objective about organisational performance, compared with ‘balance’ in ICGPSIA which refers to considering negative impact and equity resulting from the intervention.
•  Completeness appears in different principle sets to refer both to what is included within the boundaries of the report and in its treatment of the issues which may affect the assessment overall. It overlaps with context in that regard.
•  Sustainability repeats as a theme but can refer to ecological terms or the organisation’s ability to sustain itself financially over time.

Conclusions

Main areas of agreement

The principle sets reviewed can be grouped according to what they say organisations should be accountable for: their objectives, or their wider impact. The principle sets which address the wider impact involve stakeholders differently (more closely) and must evaluate what is relevant to their impact. The principle sets also diverge through their use of language, broadly grouping into those which focus on ‘issues’ and those which focus on ‘outcomes’. Valuation features more prominently in those principles designed to inform choices. There is clearly a set of overarching themes which run through the whole impact cycle.

•  A focus on outcomes, what has changed and key issues.
•  Importance of clarity of intent and purpose of the social impact analysis.
•  The need for suitable stakeholder inclusiveness.
•  Rigour and use of evidence, though not much emphasis on external assurance.
•  Transparency, clarity and completeness of analysis and reporting of impact results.
•  Cyclical nature of impact analysis and using results to improve performance, to make better strategy and policy decisions, or understanding of the issue overall.

•  Quality and assurance of data.
•  Benchmarking and comparability appeared only in a minority of cases.6
•  Proportionality, meaning the approach should be appropriate to the scale and ambitions of the intervention and the organisation running it.

Following consultation at the Berlin Conference in November 2012 a SIAA thought paper, ‘How can organisations approach the practice of social impact analysis?’ has been produced by SIAA Chairman Jeremy Nicholls.

In addition following the most recent working group meeting in February 2013 there was a consensus that SIAA should use this mapping exercise and the follow up thought paper to develop a code of ethics or similar guidelines, around values, behaviour and practice social impact analysts should aspire towards.

To help develop this code the SIAA staff team plans to do qualitative interviews, and where possible focus groups, with SIAA members and friends over the period of June-September 2013 to look in more detail at practitioner level experience. We intend for this work to be complementary to other principles and good impact codes, and support SIAA’s work with practicing and aspiring social impact analysts.

Questions, comments and suggestions about all areas addressed are welcome via hello@siaassociation.org or on our website www.siaassociation.org.

Appendix A – Principles Working Group participants

• Kim Bond – Solace Womens Aid & University of East London, UK
• Philippa Lynch – Local Government Association, UK
• Jeremy Nicholls – SROI Network & SIAA
• Jenni Inglis – Vie (for Life) and SROI Network
• Richard Kennedy– CAN & SROI Network
• Guy Beigel – Midot, Israel
• Rosemary Maguire - nef consulting
• Bokani Tshidzu – Vertigo Ventures, UK
• Tris Lumley –NPC, UK & SIAA
• Federica Luciolli – Red Ochre, UK
• Ruth Whateley – SIAA
• Sarah Bailey - SIAA
• Isabel Newman – CAF Venturesome
• James O’Sullivan –SIAA
• Claire Coulier – SIAA
• Helen Heap – Tomorrow’s People7