Fact-checking in Knowledge Management

Fact-checking is an unfortunate part of political life, but what is its role in Knowledge Management?

Image from wikimedia commons and IFLA

Fact checking – the act of checking factual assertions in non-fictional text in order to determine the veracity and correctness of the factual statements in the text – is here to stay in the world of news media. Every political statement, especially those from politicians known to “bend the truth”, is subjected to independent checks, sometimes by the newspapers and broadcasters themselves, and sometimes by independent bodies.

Sometimes the disputes over the veracity of statements turns into a dispute between fact-checkers, as in this story from Forbes magazine;

“I was sitting in the Hong Kong airport and watching Chris Matthews interview Republican presidential candidate and U.S. Rep. Michelle Bachmann, of Minnesota, about a number of topics. Bachmann had been in the news that week because she had claimed that, “now we have the federal government … taking over ownership or control of 51 percent of the American economy.” Matthews couldn’t resist jumping on the statement, noting that MSNBC’s fact checkers couldn’t come up with anything close to that number. Bachmann flippantly replied something like, and I’m paraphrasing, “Well, Chris, you have your fact checkers and I have mine. I think I’ll go with mine.”

Facts are important if people are to make correct decisions, whether this is in politics or in organisations, and it is organisations where Knowledge Management comes in. If organisational knowledge is to be used to support effective decisions making, then that knowledge needs to be based on factual evidence.

Facts and KM

Although we might cynically anticipate the need to check facts in politics, where politicians have long bent facts to their own ends, things are different in organisations. The issue we need to deal with is not political bias, but cognitive bias. All of us are human, and all of us are prone to effects such as confirmation bias (which leads us to select observations that reinforce what we already think) and attention bias (where we often fail to notice things we are not looking for).

These biases mean that what we record in our knowledge bases and knowledge assets can be as much opinion as fact, or even prejudice and misconceptions.

So how do we deal in facts, when applying Knowledge Management within an organisation?

  • Firstly when creating knowledge assets, we address the issue of validation. Generally the community validates, either through discussion and dialogue, or through co-creation of material and co-editing of wikis or collaborative documents. As Peter Kemper said in my Lessons Learned Handbook, “A Wiki should be self correcting. The moment I write something wrong, people all over the company will notice and become alarmed”.
  • Secondly in our customer-facing knowledge bases, the contact agents will constantly testing every article to see if it answers the customers problem, and will flag, review or correct content which they find to be wrong. The watchword is “Re-use is Review” – knowledge is tested in re-use and corrected where wrong. 
  • In community of practice discussions, we again allow the community to self-validate. If someone posts an incorrect answer into a discussion forum, members of a mature and trusting community will quickly offer corrections. If there is diagreement, the community can talk this through. 
  • Then in After Action reviews and Lessons Learned meetings, we discuss what actually happened, and aim firstly for ground truth, and secondly for root cause analysis. Where people express opinion, we ask for the stories on which the opinions are based, and we look for the factual core of the story, before we look for interpretation.
  • The lesson learning system is also where the knowledge assets are tested and reviewed in use. Many times, the action associated with a lesson will be to correct, edit or update the relevant knowledge asset. 

So we rely on analysing evidence at the source of knowledge creation, and on validation and self-correcting mechanisms within the wikis and knowledge bases.

Validation and fact-checking is important in Knowledge Management, because knowledge management which is not founded on fact, but on opinion, is not knowledge management at all.

View Original Source (nickmilton.com) Here.

Knowledge, Justified True Belief, and David Brent’s dance

If Knowledge is Justified True Belief, then what does “Justification” entail?  A recent New Scientist article, and a BBC charity video, give us some pointers.

BBC Comic Relief charity video including Ricky Gervais’ character “David Brent”

The April 1, 2017 edition of New Scientist magazine has the theme of Knowledge, and contains a set of Knowledge-related articles, the first of which explores the definition of Knowledge as “Justified True Belief” (aka the JTB definition).

I am not an epistemologist, but from what I read there are many type of knowledge, and this definition only applies to one type.  There is Propositional knowledge (“I know that Paris is the capital of France”), there is Acquaintance knowledge (“I know (am acquainted with) Paris”), and there is Ability knowledge or know-how (“I know how to drive in Paris”).   In other languages there may be different words used for these different types of Knowing, which is why Knowledge is often a term lost in translation. The Justified True Belief definition applies only to Propositional Knowledge.

For propositional knowledge, the three parts of Truth, Belief and Justification are important. To know something you must first believe it, and for Knowledge to be factual it must by definition be true, or else it is falsehood (although the definition of True is not easy. Given the half-life of facts, “True” often means “True for now, as far as we know”). However Belief is not enough, and that is what brings us to David Brent, and the question on whether he really knows how to dance.

Illusions of belief and David Brent’s dance

Most of us believe things that are not true, and in particular overestimate our own Ability Knowledge. For example,

  • In a survey of faculty at the University of Nebraska, 68% believed they were in the top 25% for teaching ability, and more than 90% believed they were above average. 
  • 87% of MBA students at Stanford University believed their academic performance was above the median. 
  • In ratings of leadership, 70% of US students put themselves above the median. 
  • In ability to get on well with others, 85% of students put themselves above the median and 25% rated themselves in the top 1%.
  • 93% of the U.S. students and 69% of Swedish students in a survey put themselves in the top 50% for Driving Abiility for safety,
  • in another survey almost 80% of participants evaluated themselves as being an above-average driver.

This is the “Superiority illusion” described by Wikipedia. We consistently overestimate our Ability Knowledge, and the estimates above are of course nonsense. 80% of people cannot be above average, and 25% cannot be in the top 1%.  To make things worse, it is often the people who know the least, who overestimate their knowledge the most (the Illusion of Confidence).

Ricky Gervais’ comic creation David Brent is funny largely because he consistently overestimates his own ability, and this becomes very obvious to the viewer in the embarrassing situations that follow.  In the video above,  he honestly believes he not only knows how to dance, but knows how to dance better than the professional.

It is his firm belief, but is it Knowledge? Is it justified? We can see, through our own reaction and that of his Office colleagues from 3 minutes into the video, that his self-knowledge is in fact totally unjustified, and that it is false opinion, and not knowledge at all.

So why is this important to Knowledge Management?

Knowledge management seeks to improve the Ability knowledge of knowledge workers in the organisation, and thus help them perform better. It requires some way of sharing, co-creating and transferring knowledge – making it more widely available than just leaving it in the heads of the experts. But how much of the expert knowledge is justified? And how much is “David Brent” knowledge – asserted with confidence, but actually false?  Just because an expert confidently believes something, does not mean it is true. 
When seeking to transfer knowledge, I think that we often (or maybe always) need to transform Ability knowledge and Acquaintance knowledge into Propositional knowledge. An expert can be able in a topic and acquainted with a topic, but to teach or coach a non-expert, they usually need to transfer a proposition like “I know this is a good, or useful, or recommended way to do something”, or “I know that this is a bad, or dangerous, way to do something, which you should therefore avoid”. (I have to say that I could well be wrong here, not being an epistemologist).
Once you turn your knowledge into a proposition, it needs to be justified.
  • Teaching material should be justified as “good things to learn from”
  • Tips and Hints should be justified as “good things to consider”
  • Recommended practices should be justified as “approved practices to follow”
  • Guidelines and standard procedures should  be justified as “as far as we know at the moment, the most reliable way” to do something.

Justification comes through two mechanisms:

  • Justification through experience. Know-how knowledge is justified if it reliably results in above-average performance. The justification requires not just a correlation between the knowledge and the performance, but a justifiable causal link. Saying “I did X and my project succeeded, therefore doing X will help project success” is no more justified than saying “I wore red socks to the football and my team won, so wearing red socks will help team success”. 
  • Justification by the community of peers and experts. Once the community agrees that “doing X is recommended, as a way of enabling project success” you have community justification. 
These two mechanisms are also seen in the scientific method, where scientific results should be repeatable, and externally reviewed. David Brent’s dancing would not be justified, as it would consistently by rejected by any dancing community as being repeatedly awful. 
We need deal with the issue of justification in our own Knowledge Management programs, lest we end up with unreliable knowledge, the equivalent of David Brent trying to teach others how to dance. Because if it’s not justified, it’s not knowledge.

However we also need to recognise that justifying knowledge is not an easy thing to do, and that any justification is probably only provisional.  As the New Scientist article concludes:

Various attempts have been made to tighten up the standards of justification, and provide a definition of knowledge everyone can agree on. (However) all these epistemological investigations point us to one fact that we are wont to forget: that knowing something is a far richer, more complex state than merely believing it. The ability to distinguish between fact and opinion, and to constantly question what we call knowledge, it vital to human progress and something we cannot afford to let slip”.

View Original Source Here.

Skip to toolbar