Posts

Improvement or optimisation?

Does investing in training deliver a return on investment to your organisation?

Training return on investment or not?

As with most questions like this, there are many schools of thought. The first difference is between those who promote measuring training return on investment and others who argue that trying to measure it is pointless.

The mainstream view holds that measurement can help evaluate some kind of return on investment (RoI), although factions argue about the best way to achieve this. Another perspective accepts that, while evaluating RoI may be training’s holy grail, trying to measure it may be pointless. According to Personnel Today, they argue that trying to measure RoI on learning is not cost-efficient, the formulae aren’t robust and most organisations are not even interested in the topic.
Possibly the best-known mainstream approach to training evaluation is Kirkpatrick’s four levels. Essentially the levels are a sequence of levels of evaluation for training:

1. Reaction – participants views often recorded on simple multi-choice questionnaires (often referred to as “smile sheets” or “happy sheets”) – usually at the end of a course or session.
2. Learning – the increase in knowledge, skills, or competence. This is usually assessed during the course, either in practical work or tests.
3. Behaviour – how well delegates transfer learning to their work. This stage seeks to assess changes in behaviour due to training. It may form part of a wider culture change initiative. This usually happens a few months after the training, possibly by interview but more usually some type of observation.
4. Results – the final outcomes from participation in a training program (usually financial, time or performance-based. Particular industries might focus on specifics such as safety for example)
Some practitioners find it difficult to move beyond Levels 1 and 2 when the most valuable information might exist at the later levels. Kirkpatrick-certified consultants “start with the end in mind,” moving back from Level 4 to start with the desired results before designing and developing the programme.

JJ Phillips proposed the addition of a fifth level, to measure RoI which is achieved by putting a value on the achievements that are found at the fourth level and comparing them to the total cost incurred in providing the training.

A provider’s view

From our point of view, as training providers, those in the second school of thought who think measuring RoI is pointless, might only be interested in how well a training course is received by the trainees, so evaluation will typically be based on questionnaires that ascertain the delegates’ perceptions. The mainstream group, however, may ask a training provider to demonstrate a return by using metrics and formulae that they approve of, and these might be different for each organisation, which is a bigger challenge.

Which school of thought is right?

The answer is probably that both viewpoints are valid to some extent and a more specific answer will depend on a combination of factors that are unique to your organisation; however, let’s consider some of the evidence that is available.
An important issue must be the quality of any training. With mandatory and vocational courses, the content and syllabus will be well established, so any questions of quality will tend to be focused on the method of delivery. Management training is different, however. The delivery is still important, but several authors have argued that the content of management training is influenced by fads, fashions and theories peddled by management gurus that aren’t supported by any evidence that they work. Even top business schools such as Stanford and Harvard have been criticised for teaching pseudo-scientific management models.

Investing in existing talent is a strategic issue for organisations, according to research by the University of Portsmouth on behalf of the Chartered Institute of Personnel and Development. Their report explores both qualitative and quantitative metrics and introduces the idea of moving from measures of return on investment to return on expectation, which could be difficult to define.

The Oxford Handbook of Evidence-Based Management contains a simpler example of how the return on investment of a bank’s leadership development programme was measured. Half of the sample of managers were randomly assigned to receive the leadership development and the remainder was left as a control group with no training. The people who worked for the managers were surveyed and the results showed increases in the charisma, intellectual stimulation and consideration of the managers who were trained compared to those who weren’t.

As suggested at the beginning the answer to whether investing in existing talent improves skills and loyalty is not straightforward, but it becomes clearer by using an evidence-based approach.


This is an update of an article that appeared in the New Statesman magazine in 2014.


in-house professional training development

What does good professional training and development look like?

Professional Training and Development in Context

A simple way of thinking about professional training and development in the workplace is in support of formal learning, while some ‘learning to learn’ skills may also help with informal learning which happens in normal daily life. Developing knowledge in an organisation requires formal and informal learning approaches to be integrated because neither will enable knowledge to be acquired if it is the only approach taken.  Therefore, the first conclusion is that formal corporate training and development should consider informal learning as both a predecessor to and successor of any formal intervention.

Formal learning

Formal learning activities have the goal and process of learning defined by the organisation. Also, it occurs in the work context to develop peoples’ skills and knowledge through a structured programme of lectures, discussions, simulations, role plays and other instructional activities.  The training is planned and directed by a professional trainer, which might seem a good thing, but a key criticism of this formality is that it occurs outside the context of daily practice.

Informal learning

Informal learning is the most prevalent form of workplace learning, integrated with daily work routines, triggered by an internal or external stimulus, maybe unconsciously, and can be both haphazard and influenced by chance.  More strictly, it is an inductive process of reflection and action linked to learning with others.

in-house business process mapping training workshop

CBMSc in-house process mapping

Formal professional training and development

Another important factor in professional training is that it applies to an adult audience. However, the literature is not complete or consistent in defining good practice.

A partial list includes:

  • Sensory stimulation theory
  • Reinforcement theory
  • Cognitive-Gestalt approaches
  • Holistic learning theory
  • Facilitation theory
  • Experiential learning
  • Action learning
  • Adult learning (Andragogy)

Current practice in much professional training and development draws on many of these areas.  For example, facilitation theory underpins much of the corporate training in developed economies by proposing that learning occurs through the trainer acting as a facilitator, establishing an atmosphere in which learners feel comfortable considering new ideas, recognising there is often resistance to changing currently held ideas, assumptions and preferences.  Reinforcement theory supports the common practice of awarding certificates of completion.  The evidence base for learning styles is not solid, but most professional training involves a mixture of activities that address preferences for learning based on Kolb’s research findings that adults learn in four ways through:

  • experience
  • observation and reflection
  • abstract conceptualisation
  • active experimentation

This post doesn’t have the space to discuss the ideas around learners emotional responses, but the influence of external factors on learners emotional states is important and their experiences will shape their openness or hostility to different learning activities.  For example, while some people enjoy role plays, others find them incredibly stressful and disturbing.

Research base

Some of the research in the field is low quality, so there is more work to do.  However, reputable sources do exist and they offer some guidelines that can be applied.  Consequently, we use these in designing and developing CBMSc training services.

For example, there are five key principles for adult training that say it should be:

  • immediately useful
  • relevant
  • welcoming
  • engaging
  • respectful

Knowles has done most in recent years to highlight the importance of understanding how adult learning differs from approaches for children.  His ideas support the five principles above by suggesting that adults:

  • Bring a lot of experience that trainers can use as a resource.
  • Expect to influence what they learn, how they are educated, and how they will be evaluated.
  • Respond to active participation, which should be included in the design and delivery of education.
  • Need to be able to see applications for new learning.
  • Need their responses to be acted upon when asked for feedback.

Conclusion

There is more to learn about the most effective methods for professional training and development.  However, for now, some key points to consider are:

  • Formal training and development is only ever a brief interruption to a constant process of informal learning and try to integrate with this.
  • There should be a clear goal and process
  • Trainers should act as facilitators more than teachers
  • Development should contain a mix of approaches leading to relevant, practical and actionable results
  • Treat everyone with respect

And finally, the main omission from the literature, from our perspective, is the importance of evidence-based content.  There have been some promising initiatives in the airline industry which seems to be taking a lead in upgrading their training to be more evidence-based, and the CIPD is also showing leadership in this area.  We need more safety-critical industries to follow the air transport industry lead, as well as other professional institutions to emulate CIPD.

Read more

Improvement or optimisation?

LinkedIn and Bad Science

Bad Science

I’m more aware of bad science than ever since reading Ben Goldacre’s book of the same name a few years ago. Familiarity bias and proximity bias no doubt!.  So, another day, and another LinkedIn post from a well-meaning consultant using “science” to support their argument. This post links to a YouTube video about business leadership.  It makes a perfectly reasonable point about how much interconnectedness there is in the world. The author could support their argument using ideas from Systems Thinking, for example.  However, they invoke, and then mangle, the idea of entanglement from quantum mechanics; a favourite subject for purveyors of woo.

Quantum confusion

Entanglement states that the way sub-atomic particles interact means their quantum state cannot be described independently regardless of physical location. In other words, a change in the spin of one entangled particle will be replicated in the other no matter how many light years separate them.    Now, this might be OK if the entanglement of quantum theory was limited to an analogy, but we are urged to believe that this long-distance connection between sub-atomic particles explains some mystical connection between all humans.  Why? Well, according to the author it is because, of course, we are all made of atoms!

Logically, this is the equivalent of stating that humans must be born from clouds because we mostly consist of water and, you guessed it, water comes from clouds too!

Scientific method

The principles of the scientific method are to test hypotheses to find out if they stand up.  The assumption is that most hypotheses will be replaced by better ones as we learn more. Sometimes they withstand testing and experiment sufficiently well to become a theory that provides a robust explanation of something significant.  A good example is the theory of evolution.  Non-Scientists often misunderstand the meaning of the word theory in a scientific context.  The misunderstanding is given away by statements such as: “Evolution? It’s only a theory”.

Free speech is important so, by all means, everyone should be free to post any idea that crosses their mind.   However, if you don’t understand the science you want to cite, please remember the quote from Chris Morris’s excellent “Brass Eye” TV series: “…there’s no evidence for it, but it’s a scientific fact”.  At least then we’d know you were only joking even if you are unaware yourself.

 

Lazy Management development

Lazy Management development

Body language is often included as part of management development programmes.  It’s easy to see why.  Telling people that crossed arms indicate resistance to what you are saying seem reasonable based on ‘common sense’.  However, as with many favourite topics on training courses, the evidence to support a lot of this stuff is either flimsy or non-existent.

The problem for management development

An excellent April 2013 New Scientist article “Lost in translation: Body language myths and reality“, by Caroline Williams, highlights a number of myths about “reading” body language.  Sadly, many of these myths are presented as ‘facts’ in management training and development programmes.  The giveaway is that they are presented with no evidence to support them. This lazy repetition of assertions that make the trainer sound insightful or that are simply attractive sound bites is bad practice and could cause damage by perpetuating myths that are either untrue or,at best, half-truths.

An example

For example, how many trainers involved in management development will admit to repeating one of the canards cited in the article rather than checking the evidence first.  Maybe you have heard it or used it yourself.  The assertion is that 93 per cent of our communication is non-verbal, and only 7 per cent is based on what we actually say. This figure came from research conducted over 40 years ago by Albert Mehrabian, a social psychologist at the University of California, Los Angeles. He found that if the unspoken message conveyed by tone of voice and facial expression differed from the word being used (for example, saying the word “brute” in a positive tone and with a smile), people tended to believe the non-verbal cues over the word itself. From these experiments Mehrabian calculated that perhaps only 7 per cent of the emotional message comes from the words we use, with 38 per cent coming from tone of voice, and 55 per cent from other non-verbal cues.

The source of the myth

The article explains that Mehrabian has spent much of his time in the past forty years explaining that he never meant this formula to be generalised, and that it only applies to very specific circumstances — when someone is talking about their likes and dislikes. Mehrabian claims that “unless a communicator is talking about their feelings or attitudes, these equations are not applicable” and so the oldest stat in the body language book isn’t quite what it seems. As Williams points out, if we really can understand 93 per cent of what people mean without using words, we don’t need to learn foreign languages and we would never get away with telling a lie.

Needless to say, we believe this is another reason to support the value of evidence-based training as part of management development.