Evaluation – Part 1
We use the term ‘on balance’ as a shorthand way of saying that we have come to a decision or choice after considering the power, influence, or ‘weight’ of both sides of a question or issue. This invokes metaphoric reference to a set of balance scales – as in the ‘scales of justice’ (see header image).
Evaluation skills, sometimes described as good judgment, are fundamental to good governance. The EDM (Evaluate, Direct, Monitor) Governance Model acknowledges this. This article is the first in a short series looking at some aspects of evaluation in the work of non-profit directors and managers.
Weighing options
Regrettably, most non-profit governance and management decisions involve more than two options or alternatives. Simple choices between good and bad options are rare. If the issues were that simple, then they could probably be resolved by reference to a checklist or filter system, without having to include them on a board agenda. Often enough in governance deliberations, we can also be faced with a choice between ‘least worst’ options rather than ‘best case’ scenarios.
Debates over complex public policy issues inevitably involve more than two perspectives, unless they are seen through the lens of a polarising media story or a ‘school debating club’ approach. These perspectives contrive to restrict debate to black and white positions, with ‘government’ and ‘opposition’ sides taking a stance for or against a given claim or contention.
Even the choice between action and inaction usually involves additional sub-options, such as whether to act one way or another. For inaction, you could choose to leave the matter off the agenda, or include it, but recommend that the situation simply be monitored for significant developments.
We usually employ arguments for and against each of the options to develop a collective view on which of the options is the most robust, and therefore likely to offer the most satisfactory response to the situation. The criteria we employ to make judgments regarding our option preferences, are discussed further below under ‘weighing evidence’.
Weighing arguments
Deliberative processes in non-profit and for-purpose settings are much less about winning an argument than they are about negotiating best possible outcomes for key stakeholders.
Simply counting the number of arguments for or against the proposal would not pay sufficient regard to the relative value of some criteria compared with others. That’s where recognition of the evidence called upon to support each argument comes into play.
The importance of employing evidence-based decision-making has been formally recognised by the International Standards Organisation with the adoption of ISO9000.
Weighing evidence
Identification of arguments for or against a particular proposal or position, and mapping these so they can be fully examined is a good start in weighing the arguments, but when we acknowledge that not all arguments are supported by the same standard of evidence, we recognise that we need to attach some form of importance ranking or weight to the criteria we apply to our decision making. This would tilt the scales in recognition of our values, strategic priorities, and commitment to evidence-informed decision-making.
When we establish a tender process or initiate CEO recruitment and selection, we are happy to identify criteria for use by a tender committee or selection panel. This helps to ensure that an objective decision can be reached on the preferred candidate. Skill in crafting such criteria exists in most boards and senior management teams, however, the establishment of evaluation criteria for other kinds of decisions is not always addressed. Taking the time to agree on the evaluation criteria, and their relative importance, will be rewarded when the time comes to make a decision.
For complex and high-value decisions, I have found argument maps to be a helpful aid to the deliberative process. There are numerous desktop and online mapping systems available, but I have preferred Rationale* and Bcisive* for many years. The capacity to unpack the debate, capture the supportive and opposing arguments, identify the evidence underpinning those arguments, and the sources of that evidence, is particularly helpful to a board seeking to weight or rank the arguments according to the standard of evidence they rest upon.
*No referral fees or commission arrangements apply.
When assessing whether a board sought access to relevant data and analyses to support their decisions, courts will seek to confirm that directors informed themselves to a level expected by a reasonable person before making their decision. Mere access to the relevant data would not be sufficient of course. The extent to which probing questions were asked and answered also enters into consideration.
Weighing consequences
Certainly, when we assess the likelihood and severity of adverse consequences from action (or inaction) on a given issue, we are weighing the consequences. This only considers the question of what could go wrong of course, and a balanced approach to deliberation would also look at the value proposition, and ‘benefit dividend’ for our client, member, or community. The sweet spot which (at minimum) balances benefits, costs, and risks, should be identified in board decision making, with a preference for proposals in which benefits outweigh costs and risks.
Higher-order governance
The evaluative skills involved in weighing options, arguments, evidence, and consequences are examples of higher-order critical thinking skills. This aspect of evaluation will be explored further in Part 2 of this series.
See also:
https://cebma.org/resources-and-tools/
https://www.cebma.org/wp-content/uploads/Kovner-2014.pdf
https://advocacyguide.icpolicyadvocacy.org/231-what-is-evidence-based-decisionmaking
https://www.iso9001help.co.uk/ISO-9000-evidence%E2%80%90based-decision-making.html
2 thoughts on “The Scales of Governance: Weighing options, arguments, evidence & consequences”