The Effectiveness of Training: Seeking Measurable Results

Training plays a vital role in building and maintaining an effective, high performing workforce.  NZ organisations invest thousands of dollars each year in ensuring their employees are continually learning and developing to improve their performance and reach their goals. Organisations also face the challenge of retaining talent and there is pressure to promote learning and provide opportunities for employees to grow and develop, and progress their careers.

So to what extent do organisations actually benefit from the many training courses they sign their employees up to each year? The real impact of training extends beyond the classroom or training environment but how much of this learning is applied back on the job? The generalisation of learning, the application of skills and behaviours learned in a training environment to work, and the maintenance of these skills and behaviours over time can be a major challenge for organisations. This is where training evaluation becomes such an essential activity.

Evaluating training can inform on ways to improve the transfer of skills and behaviour learnt through training. Kirkpatrick’s (1994) training evaluation model is the most common and widely recognised model of training evaluation and evaluated training across four levels:

1. participants’ affective reactions
2. knowledge acquisition or learning
3. behaviour change or application of training on the job
4. and organisational results

All four of Kirkpatrick’s levels can inform decisions about how to redesign and improve formal training programmes. How are participants reacting to the training material itself? How much are they actually learning? Is behaviour changing as a result of this? How is training impacting on the organisation’s bottom line?

With budget cuts and change rife in NZ’s public sector organisations, seeing measurable results and a return on investment from employee training becomes even more important.  So, why is it consistently reported that organisations evaluate affective reactions and learning, with very little focus on behaviour and results criteria? The world around us continues to evolve and become more scientific, yet how we measure training still remains largely anecdotal.  There are important influences on transfer of learning back to the work environment, such as trainee characteristics (self-efficacy, motivation), supervisory support and post-training environment (e.g., feedback and reinforcement). However, measuring behaviour change back on the job creates a culture of learning and a sense of accountability. If trainees are expecting some kind of follow up activity or assessment they are held accountable for their own learning and application (Saks & Burke, 2012).

There is no doubt that new scientific and technological advances in HR and I/O Psychology will play an increasingly important role in how we learn. The question is how do we “up the ante” on how we evaluate training effectiveness to keep up?

Saks, A.M., & Burke, L.A. (2012).  An investigation into the relationship between training evaluation and transfer of training. International Journal of Training and Development, 16 (2), 118-127

The Science of Team Development – What Works?

Through a quick Google search, you can see a raft of team building options, focused primarily on team “bonding” and fun activities to boost morale – rock climbing, building bikes and competing in scavenger hunts! While these activities create a sense of fun and boost morale which is important to the success of a team, an article I read recently spoke to the science behind team development. The extent to which team development initiatives actually enhance team performance is a question I’ve often asked myself in the past.  This article has helped me to understand why this question isn’t that straightforward – not all interventions are created equal! So what works?

Two overarching categories of team development – team training and team building.  Understanding the science behind these two quite different interventions can help organisations to better utilise them. On one hand, team training focuses more on knowledge acquisition and practising the skills and attitudes required for effective performance.  Team training is most effective in preparing new teams by ensuring that the group has a shared understanding regarding their purposes, goals, and the behaviours necessary to work together effectively.

Team building on the other hand, has evolved over time from focusing just on interpersonal and social interactions to looking at how these can help the team achieve results.  Klein and colleagues (2009) looked at the impact of team building on team outcomes for 20 studies. All four approaches to team building they investigated were found to have a moderate effect on outcomes, with goal setting and role clarification being the strongest. In addition to this, team building had the strongest impact on affective and process outcomes. This article presents empirical evidence supporting team building and suggests that team building is most effective for teams that need clarification and better understanding of their roles in the team and with teams that already have experience working together. Further, having clear, challenging goals can improve team motivation and help work as a whole. There also needs to be a willingness to speak up in order to diagnose needs and outside facilitation is necessary (Shuffler et al., 2011). So it’s not just for dysfunctional teams, but teams that could do with clarity and perhaps a bit of re-energising!

The effectiveness of team development interventions depends on how well they meet the distinct needs of the team at the time. Team building is most effective for resolving team breakdowns and reducing conflict, and also for improving communication and trust within existing teams. On the other hand, team training is better for providing the knowledge and skills necessary for team work. To me, this means that those designing team development initiatives need to understand individual and team dynamics. As professionals in the area of HR and I/O Psychology, we need to better understand why and how our clients and organisations will benefit from team development. So how can we get to the heart of team issues and better diagnose their needs?

Shuffler, M.l., Diaz Granados, D., & Salas, E. (2011).  There’s a science for that: Team development interventions in organisation.  Current Directions in Psychological Science, 20(6), 365-372.

Don’t Judge a Book by it’s Cover

Think back to the last time you met someone new…

What kind of assumptions did you make about that person based on, for example, their handshake, their posture and what they were wearing? It’s no secret that we make a number of snap judgments within seconds of meeting someone for the first time. With the popularity of social networking sites, sometimes this happens before we have even met them! For example, Sparko and Zebrowitz (2011) argue that we over-generalise our perceptions of babies (being vulnerable and needing our protection) to the extent that adults with “baby faces” (i.e., facial qualities of large/wide eyes and rounder faces) are often perceived to be warmer and less assertive than their “mature faced” colleagues. How much can we rely on these inferences?

Despite all the words of caution about not judging a book by its cover, it seems that we just can’t help ourselves! In the absence of other information, these split second inferences happen quickly and often unconsciously. Research suggests that from just milliseconds of exposure to facial structure alone, people form impressions about traits such as competence, intelligence, aggressiveness and trustworthiness (Willis & Toderov, 2006). Unfamiliar faces (with neutral expressions) were presented to research participants for between 100 and 1000 milliseconds, after which they were asked to make a series of trait judgments and indicate how confident they were in those judgments. Judgments that were made after only being exposed to faces for 100ms were highly correlated with judgments made in the absence of such time constraints. Interestingly, with longer exposure, people’s opinions didn’t necessarily change; they just became more confident in them!

First impressions do count. They can have significant social consequences as they often influence our expectations of, and behaviour toward others. There can be huge risks for employers who rely on these too heavily. Consider selection interviews and the hiring of new employees for example… these implicit assumptions can cause real harm! Given how quickly these inferences can happen, and how far they can stretch, first impressions can cloud our ability to make objective, rational decisions. How closely do you attend to yours?

Sparko, A.L., & Zebrowitz, L.A. (2011).  Moderating effects of facial expression and movement on the babyface stereotype. Journal of Nonverbal Behaviour, 35, 243–257.

Willis, J., & Toderov, A. (2006).  First impressions: Making up your mind after 100ms exposure to a face.  Psychological Science, 17(7), 592-598

Avoiding Identity Crises in Times of Change

My colleague Heather recently blogged about the frequency of change in organisations today and how important it is for change to be carefully planned. Obviously the planning and communication around change are paramount; otherwise employees can end up relying on informal communication (and gossip!) which has the potential to increase stress and uncertainty! This got me thinking about other important considerations around coordinating change, especially group dynamics.

While change sometimes (but not always) leads to some roles being disestablished, employees remaining in the organisation are often re-grouped and re-organised.  For those of you who are unfamiliar with Social Identity Theory, social identification is the tendency for us to identify ourselves according to the social groups to which we belong. This includes the organisation and the business groups we work with, and provides us with a theoretical framework for understanding the group dynamics associated with change.  Restructuring, mergers, and other change initiatives often force employees to give up their old group identity and re-categorise themselves as members of a new organisation and or group.  This impacts on how people define themselves within the organisation.  People react differently to change. Some may accept the evolution of the organisation quite freely, whereas others may reject the new and cling to the old, despite the new structure. From a psychological perspective, new groups may fail to become a group at all, and continue to favour their prior group membership and social identity. This can result in the emergence of a “Them” versus “Us” mentality and lead employees to treat one another unfairly. In reality, the stronger these biases become, the more difficult adjustment becomes. 

How can organisations going through change overcome problematic employee attitudes as a function of changes to their identity? How can they facilitate employee adjustment to maximise the chances of successful change and transition? More effort may need to be directed at creating a new sense of group identity and de-emphasising previous group membership. Perhaps team recognition could refocus attention to a common goal? In addition to this, leadership and formal communication should provide clarity for employees and groups should be provided with opportunities to interact both formally, and informally. Even things as simple as seating arrangements (i.e., having groups sit in an interspersed manner) could make a world of difference. In the planning of change, how can we better predict what issues could arise so we can manage them more proactively?

Confronting Emotions at Work

Emotions and feelings are often dismissed as being irrelevant to work or just simply ignored, thus supporting the avoidance-based coping strategies that research tells us are ineffective. People can’t necessarily just “build a bridge and get over it”.  This type of approach is not only problematic at an individual level; what kind of culture does this foster in an organisation? My guess is probably not one that supports open communication, team work, and support.

Our emotions are not necessarily a result of what happens to us, but how we appraise or perceive those events. That is, how we think about them. For example, it’s not the event of being late for a meeting that makes us feel bad, but what we think about being late for work that leads to this experience.  There are ways for emotional situations to be managed in the workplace, in a way that builds constructive relationships rather than dismissing feelings as being irrelevant to work and shutting down the lines of communication.  Distraction and disputation techniques are two commonly used examples of these. 

Helping people to reframe confronting emotions is known as disputation. This can help people to see more clearly and open their mind up to a different perspective in order to deal with the situation. Yet, immediate emotional responses can cloud this rational judgment and we need to assess the situation and the emotional reaction first.  For those of us who want a practical solution and to fix things straight away, it’s not always the right time. The key differentiator depends on the intensity of emotion at the time.

Distraction techniques are useful when emotions are heightened. Things such as going for a walk, listening to music, making a cup of tea, taking three deep breaths, or calling a friend can help to reduce the initial intensity of the emotional reaction. When this subsides, then disputation techniques can be applied. This is the best time to start supporting people to identify potentially irrational thoughts, review how useful these are, and to consider alternative ways of thinking and feeling. This often involves questioning what the implications are of continuing current patterns of thinking and how more constructive thought processes could be beneficial.  What are some of the strategies for managing your emotions at work that have worked for you in the past? What are some of the triggers that cause you to reach the threshold to move from disputation to distraction?

Bad Jobs versus Unemployment

The fact that having a job provides us with an income, social contact, security, and a sense of purpose is undeniable. Furthermore, having a job has also been shown to be related to increased health and wellbeing. However, a paper published by the National Centre for Health at the Australian National University suggests that some jobs are better for us than others, with the “quality” of a job being pivotal to determining whether it is harmful or beneficial to our health. Jobs that are poorly organised, where there is excessive demand, an employee has little control over what one’s work involves, and where employees are concerned about the security of future employment, can be as harmful to one’s health as having no job at all.

What’s interesting is that the effects of these work stressors are not confined to any specific level of job, and in fact they operate independently of job status and income. Work stressors can have a harmful impact at all levels, including managerial and professional roles. This doesn’t mean that anyone who feels a little under pressure at times should throw in the towel and quit their job, but this research got me thinking about working conditions. Why do people stay in poor quality jobs? For some, it may be a matter of deciding which is the lesser evil, a poor quality job or unemployment?

In the last quarter of 2011, the unemployment rate in New Zealand was at 6.6%.  But this research suggests that people in poorly organised roles, jobs lacking control, and in which they feel insecure, are likely to be just as badly off (in terms of their health) as that 6.6% of unemployed individuals.  With the current state of the economy and the rising cost of living expenses, I’m guessing that a lot of people may be willing to put up with significantly more negative factors in their current roles, than they might during better economic times. Some people may also be putting up with poor working conditions in their job as it is a stepping stone to another, more desirable role. Others may not be aware of the impact that their job is having on their health and wellbeing, or may not realise another role may be more fulfilling.

While unemployment is an important statistic, do we also need to consider the quality of work that is being offered? What proportion of Kiwis are working in poor quality jobs and what impact does this have on their lives? What responsibilities rest on the shoulders of employers in terms of improving the quality of work? 

Health and Wellbeing at Work: More Than Just “An Apple a Day”

A substantial proportion of our waking life is spent working, so it’s not entirely surprising that wellbeing at work has become an increasing area of interest for many individuals. However, organisations may be unwilling to invest in interventions or programmes  that promote such wellbeing, unless it is going to impact on their bottom line.

A number of studies provide evidence to support the idea that participation in organisational wellness programs can result in reduced stress levels, lower absenteeism, higher job satisfaction and increased productivity (Parks & Steelman, 2008).  However, most of the programmes reported are focused on physical health (e.g. gym memberships, health and fitness groups) and education (e.g. nutritional information).

My thoughts are that while physical health is important to overall wellbeing, this is only one piece of the puzzle.

The World Health Organisation’s definition of health, and New Zealand’s own Health and Safety legislation, extends beyond physical health to also include mental and social health. However, there appears to be little research linking specific organisational practices and interventions to mental and social wellbeing.  Nevertheless, many consulting companies and organisations offer a range of wellness programmes addressing the more psychological and emotional aspects of wellbeing. These offerings are designed to aid employees’ ability to cope with stress and bounce back from adversity, for example resilience training and stress management programmes. These types of interventions have the potential to not only assist employees in coping with pressure at work, but also teach people transferable skills that can help them cope with pressure in other aspects of their lives as well.

As I/O and HR researchers and practitioners we need to provide evidence to support these interventions and link them to individual and organisational outcomes; otherwise many organisations may not consider the value the programmes can contribute to individual mental health and organisational outcomes.  

So, how do we measure the effectiveness of these interventions and where do we start?

Good Faith vs. Privacy: Food For Thought

For those readers with the inclination to follow the recent Massey University employment court case ruling, I’d imagine you have had some interesting conversations concerning the implications of this ruling for I/O psychology practitioners.  That was certainly my experience, specifically around privacy issues!

In a nutshell, Massey’s 2009 restructure meant that in some areas, existing positions had been disestablished and current employees had to enter into a selection process to compete for a smaller number of new positions that had been created.  Internal applicants were provided with information such as a job description, interview topics, and selection criteria.  Summaries of results were subsequently given to unsuccessful applicants. Requests from internal applicants for additional information relevant to themselves and other applicants were declined by Massey on the grounds that this was considered to be “evaluative material provided in confidence”.  This decision was then challenged in court.

In a surprise move the Judge ruled that the internal applicants were entitled to all information and materials that influenced the selection decision. As this case concerned a selection process during an organisational restructure, it was decided that the Employment Relations Act (2000) effectively “trumps” the Privacy Act (1993). Under the duty of “good faith”, this means that internal applicants were entitled to all information relevant to the selection decision. This not only included all evaluative material supplied by the interview panel about the individuals themselves, but also information relating to other candidates!  This ruling raises some really thought-provoking considerations.  For example:

• How might candidates perceive a selection process in which their notes and evaluation material are accessible to their competition? How would this impact on the quality/quantity of an applicant pool, or perceptions of justice within the organisation?
• The rights and obligations that arise out of existing employment relationships differ to those where an external candidate is participating in a selection process. Would this mean that there would be differential treatment of internal over external candidates?
• No psychometrics were used in this case, but if they had been, what would be the potential consequences of test results no longer being confidential? How would HR and I/O practitioners balance their ethical and legal responsibilities?

What are your thoughts on these and other considerations/implications?  Really looking forward to hearing what you think!

Resilience: Recognising Our Strengths

I recently attended an interesting presentation on coaching and developing resilience. The key focus of this presentation was self-reflection and recognising our strengths.

Sound straightforward? That’s what I thought…

I quickly discovered that it’s much easier said than done.  Reflecting on our own strengths and achievements can be difficult, particularly if we keep shifting the goalposts. Continuously looking at the gap between our current and ideal selves can leave us feeling dissatisfied. By recognising our strengths more often, it’s easier to remind ourselves of these when things don’t go according to plan.

One way for organisations to create and sustain resilience in their employees is to encourage this strengths-based self-reflection by sharing and celebrating successes.  These achievements don’t need to be big, bold success stories. The simple act of acknowledging our progress can help us to recognise our strengths, appreciate and support one another, as well as ease the process of developing and achieving our goals for the future. How do you celebrate success in the workplace?

Have you ever sat down with your colleagues to reflect on what you have achieved at the end of the week? Or perhaps considered which of your personal strengths helped you to get there?

How do you think you would benefit from taking the time to reflect in this way?

Focusing on our strengths certainly doesn’t imply that we should walk around in a state of blind optimism, but there is a lot to be gained from learning to balance our focus between what we want to achieve in the future, and what we have achieved so far.

I would encourage you all to try it. Trust me… it’s not as easy as it sounds, but the benefits are worth it.