Tender for the development of the next phase of the In Training Assessment Profile

Tender for the development of the next phase of the ‘In Training Assessment Profile’ iTAP. This is the development of an online self – assessment questionnaire for trainees who might need additional support early in their postgraduate medical training programme.

1. Background

Health Education England (East Midlands area) has worked collaboratively with Work Psychology Group (WPG) on an in depth quantitative analysis of data from the e-portfolios of GP trainees who began training in 2010, 2011 and 2012. This analysis found 6 pieces of evidence/scores in the e-portfolio and recruitment assessment scores that were good predictors of trainees either failing a mandatory professional exam and/or requiring additional training time. A statistical matrix was then developed to use these 6 predictors to provide an overall assessment on a trainee’s likelihood of failing an exam or requiring additional training time. Although not perfect, the matrix was able to predict from data of the first 6 months of training which trainees might require additional support so as to prevent them from needing additional training time or failing exams.

The next stage of the project is to develop an online self-assessment questionnaire so as to define why this group of trainees are at risk of failing exams and/or requiring additional training time and what additional support might be appropriate for them. Although the questionnaire will help to indicate what issues the trainees might have and what support would be appropriate it would also be the basis of a structured interview with an educator.

The iTAP matrix and the self-assessment questionnaire should be regarded as closely integrated and part of the same process.

2. Scope of the contractual agreement.

It would be expected that this work would be in 4 phases.

Phase 1 – Literature review with aim to provide draft iTAP questionnaire.

This should include

• Review of academic literature relating to trainee progression in Postgraduate medicine with particular reference to why medical trainees might not progress satisfactorily.
• Review of interviews and questionnaires relating to postgraduate medical trainees in difficulties that are in common and current use in postgraduate medicine.
• Review and understanding previous support programmes carried out in the East Midlands that might be relevant to the development of the iTAP questionnaire.

Phase 2 – Consultation with stakeholders to finalise questionnaire

Although this work has been developed within the context of GP training in the East Midlands area of Health Education England it is important that it is now developed so it can be applicable to all postgraduate medical trainees across the UK. There should be

• Face to face consultations with relevant stakeholders i.e. different training programmes and different areas.
• Incorporation of trainee feedback and educators such as ARCP panel members.
• Questionnaire should be usable online. There should be liaison to facilitate this process and should undertake user testing and quality assurance of the questionnaire prior to trainee completion.

Phase 3 - Piloting and Evaluation

There should be an analysis and evaluation which should include

• An analysis of group differences in the iTAP questionnaire as a normative group will need to be involved as well as the ‘at risk’ target group.
• A psychometric evaluation of the questionnaire properties and sample. This should include evaluation content and construct validity using factor analytic techniques and classification of norm groups.
• Develop provision of individual trainee feedback and production of guidance document to enable interpretation and feedback.
• A full understanding of the iTAP matrix so that any modifications to the iTAP matrix can be made as a result of this evaluation.

Phase 4 – Reporting of the Questionnaire results

This should include

• Descriptive analysis and summary of iTAP questionnaire responses
• Creation of PowerPoint of summary of results.
• A face-to-face presentation of the results with further discussion as to how to develop the interview phase of iTAP.

Areas of responsibility which will be retained by Health Education England.

The recruitment and identification of trainees to this process will be carried out by Health Education England.

Timetable for work

The time taken will run consecutively.

Additional information

Tenders are required to provide full contact details as well as a statement setting out in detail how you will deliver the services set out in this specification.

The overall evaluation will be based on the following factors relating to your submission.

• Experience – This should include experience assessments and evaluations in postgraduate medical education in the UK. In particular experience in translating research into sustainable and effective solutions.
•  Administration – Experience with educational healthcare providers.
• Evaluation – Previous experience in evaluating assessments in postgraduate medical education with experience in publishing in peer reviewed academic literature.

The closing date is Saturday 30 April. Please send your proposals to Vicky Gleeson, Head of Integrated Services, at victoria.gleeson@nhs.net.