Data collection

Key stakeholder consultation

• Interviews were conducted with stakeholders (N=9) from within and outside of General Practice and / or medicine. The final sample included representatives from the public and private sector with expertise in Selection and Assessment and Equality and Diversity, as well as academics with expertise in the area of Equality and Diversity.
• Stakeholder interviews were approximately 30 minutes long; three were conducted face-to-face and six via telephone. All the information gathered as part of the stakeholder consultation has been made anonymous and stakeholders’ contributions will be kept confidential.
• Stakeholders were asked to comment on three key areas:
  1. Knowledge or experience of group differences in selection performance.
  2. Perceptions of the nature and causes of such group differences and knowledge, experience or perceptions of the necessary guidance for selectors / candidates or effective intervention strategies aimed at reducing group differences.
• Interview responses were transcribed and descriptors extracted via a qualitative coding process to identify key themes. Individual themes were further grouped in to six broad themes.

Overview of key themes from stakeholder consultation

1. Best practice in selection and assessment
2. Quality and diversity in organisations
3. Trends in group differences
4. Cultural differences
5. Individual differences
6. Psychological factors

The majority of organisations represented by the stakeholders consulted did not necessarily focus on ‘place of qualification’ as an area of interest for equal opportunity and diversity monitoring. The outcomes of the stakeholder consultation are therefore likely to inform the broader issues of ethnicity and consideration of equality & diversity issues in relation to selection.

Key theme 1: best practice in selection and assessment

The most significant theme occurring during stakeholder consultation was the importance of rigorous and standardised assessment processes, designed and evaluated in line with best practice guidelines (e.g. British Psychological Society).

• Assessment and selection processes that are robustly designed and supported by thorough selector training will reduce the adverse impact against minority groups; any group differences are unlikely to be the result of bias in the process but other external influences.
• The GP selection process has been rigorously implemented with exercises designed to reflect job relevant characteristics and informed by a comprehensive job analysis process. The standardised GP selection process already follows best practice guidelines and its validity and reliability demonstrated. The process therefore delivers to similar standards of assessment practice as other stakeholder organisations in the public and private sector.

Some of the characteristics of this theme are as follows:

• Merit-based assessment processes which are supported by a rigorous job analysis, designed in a standardised and objective manner and evaluated at all levels to identify any possible areas of bias.
• Equal representation of ethnic groups amongst selectors and during any pilot phases.
• Formal selector training in assessment principles and awareness of cultural differences.
• Familiarisation: ensure candidates understand the format and objective of exercises, how they will be measured and provide opportunities to practice where appropriate.
• External validation or review of processes (e.g. specialist diversity advisors) to identify possible cultural bias in language or exercise used.
• Objective assessment of training standards and equivalence of qualifications for international applicants / those who achieve primary qualifications outside of the UK.

Key theme 2: Equality and Diversity in organisations

Organisational practice in relation to equality and diversity, candidates’ familiarisation with organisations and positive action or targeted diversity initiatives are the core elements of this theme.

The consultation process highlighted that candidates’ familiarisation with an organisation and the associated culture can impact on their success both in selection and training. In addition, the extent to which organisations proactively engage in equality and diversity training or awareness activities and positive action initiatives (e.g. targeted recruitment) can reduce possibilities for inadvertent discrimination. Some of the characteristics of this theme are as follows:

• Equality and diversity training / cultural awareness training: need greater evidence base of the effectiveness of such interventions but they could support organisations in being more aware of diversity practice; E&D is not given enough focus in training.
• Cultural programmes may help people understand difference in contexts; mentoring / advice and information about how to use support networks not just access them may be useful.
• Integration and cultural familiarity can be issues for individuals moving to the UK from overseas; if these individuals are not supported they may seek out support networks within their own cultures, in turn reducing possibility of integration with UK culture.
• Familiarisation with an organisation’s structure/processes as well as the organisational culture can give candidates more confidence entering selection processes.
• Targeted attraction events / diversity initiatives: use as opportunities to further support or coach minority groups in selection processes and familiarise them with organisations.

Key theme 3: trends in group differences

Stakeholders identified a number of similar patterns to those identified in GP, in terms of group differences in performance during selection. In addition a number of stakeholders referred to their investment in continued equal opportunities monitoring and selection data analysis in order to identify patterns in performance; this is similar to the activities conducted in relation to GP selection (i.e. annual equalities impact reports).

• Whilst similar patterns to GP selection emerged in relation to age and gender, these were not necessarily as significant or consistent as the pattern of ethnic group differences (i.e. White candidates outperforming those from a number of other ethnic groups). The consultation process did suggest however that there was still no conclusive evidence to explain why such performance differences emerge among ethnic groups.
• A number of stakeholders referred to differences in the applicant pools for their selection processes compared to that of the GP selection process. For example within GP selection there may be a narrower applicant pool (i.e. candidates have already undergone a certain level of specific medical training). For some of the stakeholders we consulted they were managing a much wider range of applicants whereby an undergraduate degree may be the minimum level of attainment; there may therefore be motivational differences within the applicant populations.
• Stakeholders identified similar patterns to GP selection in terms of a higher proportion of younger applicants successful at selection compared with older candidates. Results are likely to be caused by a number of factors but could be an artefact of age where the applicant pool includes a small minority of applicants who reapply, without necessarily developing competence, and therefore are older with each year of re-application.

In summary some of the characteristics of this theme are as follows:

• Still a lack of conclusive evidence to explain why ethnic minority groups perform worse in selection and generally in terms of academic performance.
• Proactive monitoring and evaluation of equal opportunities and selection data is conducted by other public and private sector organisations in order to analyse trends and seek to understand the nature of group differences.
• Most stakeholders noted the following group differences in selection or performance (generally but not universally): White candidates outperforming all other ethnic groups, females outperforming males and older candidates performing less effectively than younger candidates..
• Greater proportions of ethnic minority groups, compared to White candidates, with lower levels of academic attainment at selection and other points of measurement once selected (e.g. performance review).

Key theme 4: cultural differences

This theme refers to differences in candidates’ cultural backgrounds (as defined by their nationality, ethnicity or place of origin). A difference in candidates’ cultural frames of reference was repeatedly offered as a significant influential factor or possible
explanation for differential performance during selection (as well as in training and organisational performance) amongst ethnic groups. Some of the characteristics of this theme are as follows:

• Differences in the training and practicing environments of non-UK doctors compared with the UK culture could create difficulty. Organisational cultures of UK versus non-UK healthcare systems could contribute to group differences when assessed in UK environment.
  • Roles and responsibilities perceived differently between cultures e.g. nurses form part of the team in UK, in other cultures this role may be seen to ‘serve’ doctors.
• Potential negative attitudes towards overseas doctors (negativity or scepticism from the public and media highlighting incidence of overseas doctors referred to GMC for fitness to practice). This stereotyping or attitudinal shift could impact on opportunities and experiences they have practicing or training in a UK context.
• Values developed as a result of parental influence / cultural background e.g. some cultures place higher value on education (perceptions in some ethnic groups that education is not ‘cool’).
• Differences in cultural values and beliefs e.g. within Asian culture there is an emphasis on family networks and supporting large numbers of dependents and there is stigma associated with ‘failure’; such factors could cause additional pressures which affects performance.

Key theme 5: individual differences

This theme refers to differences inherent in the individual candidates or their personal experiences. Candidates’ socioeconomic backgrounds, educational experiences and educational level of attainment as well as their English language proficiency were considered to be some of the individual differences referred to as potentially offering explanations for ethnic group differences in performance. Some of the characteristics of this theme are as follows:

• Similar issues exist within other public sector and private sector organisations in relation to English language proficiency; candidates whose first language is not English are less successful. If using English as a second language this can result in slower reading and processing of information; has a significant implication for timed exercises/tests.
• Mixed perceptions regarding impact of socioeconomic status/background although overall the majority of stakeholders view this as having a significant impact on the educational and training opportunities candidates are exposed to as well as the network of contacts they can draw upon for support and guidance. Newly arrived overseas doctors may lack similar networks.
• Differential opportunities to develop relevant competencies or skills; e.g. differences in training experiences, interactions with peers and educators, methods of teaching and structure of learning could contribute to differences in training experience and therefore level of knowledge and competence developed.
• Performance in selection likely to be explained by variation in experiences prior to this point i.e. training, work experience etc – need to understand nature of this variation.

Key theme 6: psychological factors

During discussions, a number of psychological factors were considered as potentially contributing to explanations of performance differences. Levels of motivation, achievement commitment, personal values, and the impact of self-esteem, behaving consistent with stereotypes and testing anxiety were all identified as key factors to consider within the scope of this matter. Some of the characteristics of this theme are as follows:

• Unconscious bias is increasingly growing in popularity as a theoretical explanation of adverse impact and differential performance between ethnic groups during assessment and selection processes.
  • Raising awareness of this contributes to reducing the impact it has. Tests to assess unconscious bias are also available to assist individuals in understanding their own personal unconscious biases.
• Test anxiety is still a common problem and can be reinforced when individuals are aware of diversity statistics which indicate poor performance in their demographic group.
• Ethnic group differences particularly exist when highly cognitively loaded tests are used under strict time pressured conditions (even in tests designed rigorously and checked for cultural bias).
• Understanding of issues faced by diverse groups and how individuals cope or what additional support they need might be useful.
• Impact of social network on performance was also considered. This is linked to cultural integration and the psychological impact of isolation or a sense of belonging can have on individuals’ performance and interaction with others.