We all seek validation
Most of us are very familiar with the kind of validation offered when others like or share our social media posts. This emotional validation involves processes of uncovering, accepting, approving, and hopefully understanding, the emotional experience of another person.
Organisations also seek a similar form of validation when they monitor engagement with their social media channels or seek a net promoter score from their members or clients. The validation offered by social media is a shallow version of that offered by close personal relationships of course. It is also only one of the many ways validity concepts are able to inform our life and work.
I have previously noted that validity, value, and valence concepts are used in the governance and management of non-profits, and that each refers in some way to notions of ‘strength’ and ‘worth’.
Having dealt with value and valence perspectives in several recent posts (see links below), this post reflects on some uses of validity in non-profit governance and management. While ‘reliability’ is a companion concept to validity, I will defer comment on the relationship between these ‘partners’ until a later time.
Validity in board deliberations
The validity of the measures your board and management team use to monitor and assess your non-profit organisation’s performance and conformance is a key determinant of your governance effectiveness, and so it warrants consideration.
Along with accuracy, precision, and reliability, validity is one of the core concepts in the evaluation of logical arguments and scientific research.
When deliberating on decisions, both deductive and inductive reasoning are used. Inductive reasoning aims at developing a theory, while deductive reasoning aims at testing an existing theory. Inductive reasoning moves from specific observations to broad generalisations, and deductive reasoning is the other way around.
In deductive reasoning “an argument is valid if and only if it is necessary that if all of the premises are true, then the conclusion is true; if all the premises are true, then the conclusion must be true; it is impossible that all the premises are true and the conclusion is false.”
https://web.stanford.edu/~bobonich/terms.concepts/valid.sound.html
Conversely, therefore, an invalid argument is a deductive argument that fails in providing conclusive support.
When judging the validity of arguments used by those debating a proposal or resolution before the board, the soundness or cogency of those arguments needs to be assessed. The tests applied in these circumstances are summarised in the header chart above.
Validity of performance measurements
The MELD model of reflective governance promotes performance measurement and evaluation within programs or organisations to systematically and regularly track indicators of outputs, outcomes, and impacts. This information is intended to enable stakeholders to take timely corrective action and increase the likelihood of program success. As observed by Nkwake, whether it does or not depends a good deal on the appropriateness of the indicators used.
“Effective performance measurement largely depends on the use of appropriate indicators. Indicators are operationalization of program results (outcomes and impacts). Inaccuracy in the use of appropriate indicators affects construct validity. Preconditions for construct validity of indicators include specificity, clarity, directness, achievability, relevance, and adequacy. Assumptions arise whenever these preconditions are unexamined or taken for granted.”
Nkwake A. (2015) Validity in Performance Measurement. In: Credibility, Validity, and Assumptions in Program Evaluation Methodology. Springer, Cham. https://doi.org/10.1007/978-3-319-19021-1_8
The importance of using valid indicators may seem obvious. However, in my work with non-profit directors and managers over the years, I have often found they (or their boards) have been using vague, unclear, or unachievable indicators in strategic plans and/or associated performance management measures.
Context matters
The more common uses of the term ‘validity’ in board deliberations relate to the strength of arguments, positions, analyses, and conclusions. However, just as valence was recognised for its multi-faceted meanings, validity is another term that is applied in many diverse ways – as listed in the chart below.
The effect of context on the meaning we assign to validity was partially recognised by Hammersley and Atkinson in their 1983 observation:
“Validity is relative to purposes and circumstances … Data in themselves cannot be valid or invalid; what is at issue are the inferences drawn from them.”
Hammersley, M., & Atkinson, P. (1983). Ethnography: Principles in practice. London: Tavistock.
Depending on the matter under consideration, quite different meanings could attach to the concept of validity. It could be considered something of a ‘slippery’ concept. Consequently, it requires some care in its application. Given the variety of definitions and processes attached to the term, selection of the most appropriate validity measures is important if you are to avoid misunderstandings or misdirected energy.
In my post on the multi-focal views one could take about organisational culture, I listed seven lenses (disciplines) that may be used. These were drawn from the fields of Psychology, Anthropology, Sociology, Social Psychology, Political Science, Economics, and Governance. A number of these fields use validity concepts differently from each other, while those relying primarily on statistical validity use a more standard definition and methodology. The following selection of quotes from authors in various fields illustrates the effect of context on the meaning attached to the term.
Those fields using statistical validity are likely to use some or all of the validity measures described in the following chart. This highlights the three main groups of validity types used in statistics: translational validity; construct validity; and criterion validity.
The context in which non-profits seek to apply validity concepts could include (amongst others):
- strategic reviews, in which a result is valid if it achieves the intended outcome
- confirming that the organisation has addressed conformance requirements and so remains a valid registered entity with regulators
- evaluating the data, analyses, and conclusions offered regarding key metrics and performance (e.g. KPI) reports
- assessing arguments advanced for or against proposed measures
- reviewing the efficiency of business processes and workflows
- conducting incident analyses or adverse event ‘post-mortems’, including health and safety events
- conducting assessment or testing program reviews
Ensuring that your director development programs include education about the appropriate uses and forms of validity is another way you can improve the effectiveness of your organisational governance.
See also:
https://www.frontiersin.org/articles/10.3389/fpsyg.2018.02425/full
2 thoughts on “Validity and Validation in your non-profit”