Bulletins
The Exam: Don’t forget the laboratory!
Charles Singer
Chairman of the Panel of College Examiners in Haematology
Spring 2007 marked the introduction of the new format examination approved by College Council in 2005. The old format Part 2 Oral Examination continues to be run for trainees who have passed the old format Part 1 Essay and Practical Examination which has been discontinued.
Part 1 Essay Paper
This paper differs from previous Part 2 essay papers. There are four compulsory questions each covering a major topic of the four major parts of the specialty (Coagulation, General & Laboratory Haematology, Haematological Oncology & Transfusion Medicine) rather than a choice of four essays from five options. This format is based on educationalist principles that this makes the exam more reliable as all candidates sit exactly the same questions and that providing a choice which in theory might favour weaker candidates, does not in fact do so as they consistently make the wrong choice! The questions are structured in a way that encourages a candidate to combine book-learning with clinical and laboratory experience and to demonstrate an ability to evaluate options in the context of laboratory investigation or treatment of a haematological disorder rather than to simply reproduce a list of facts learnt from a text-book.
Part 1 MCQ Paper
This paper consists of 125 MCQs. Fifty take a ‘best from five’ format and fifteen take an ‘extended matching’ format each containing five questions in a topic area. The questions were developed at Examiner Workshops begun in 2005. These have used question formats recommended by National Board of Medical Examiners in the USA to devise tests of a candidate’s ability to apply knowledge or judgement rather than tests of a candidate’s recall of facts. The paper is structured to provide 25% of the questions from each of the four major parts of the specialty and is blueprinted across a broad range of the curriculum. An Angoff exercise is undertaken on the final paper to determine the pass mark by a group of 20 examiners representing the range of Consultant Haematologists.
Candidates must pass both these papers to pass the Part 1 Examination. There is no compensation between these papers.
New Format Part 2 Practical & Oral Examination
This examination is based on the structure of the old format Part 1 Practical & Oral Examination with a pass standard equivalent to the responses expected of a non-specialist consultant colleague. The papers have been developed by groups of specialist examiners. The Oral Examination is conducted at each centre, uses a defined question structure, examines all four major parts of the specialty and candidates are scored against validated marking descriptors.
The final papers are submitted to an Angoff exercise by a panel of 20 examiners to determine the pass marks. Candidates’ scripts from all Centres are marked together at the Examination Board by teams of specialist examiners comprising a mixture of examiners involved in the development of the papers and examiners involved in each centre. The scripts are submitted to a Limen exercise, the result of which are compared to the Angoff score and the final pass mark determined. The final outcome for borderline candidates is determined after a review of all their papers and their performance in the separate components of the Oral Examination.
Spring Part 1 Examination Results
64 candidates attempted the Part 1 written examination. 39 candidates passed overall (61%). 41 candidates passed the Essay Paper (64%). 51 candidates passed the MCQ Paper (80%).
Spring New Format Part 2 Examination Results
The examination was held at 4 centres: London (St Thomas’s), Edinburgh, Cambridge and Bristol. 23 candidates attempted the examination. 9 candidates passed overall (39%). The pass rates in each component were: Morphology 70% (n= 16), Coagulation 74% (17), Transfusion 70% (16), Oral 87% (20).
7 candidates (30%) failed a single component (5 Coagulation, 1 Morphology, 1 Transfusion). 5 candidates (22%) failed 2 components (Morphology was a component in 4) and a single candidate failed 3 components (Morphology, Transfusion & Oral). No candidates failed the Oral Examination alone.
Autumn Part 1 Examination Results
This examination was attempted by 64 candidates, of whom 41 (64%) passed the Essay Paper and 48 (75%) passed the MCQ Paper yielding an overall pass rate of 61%. A very small number of candidates passed the Essay Paper but failed the MCQ Paper.
Autumn New Format Part 2 Examination Results
This examination was held at 4 centres: London (Royal Free), Glasgow, Newcastle and Leicester. 38 candidates attempted the examination. 17 candidates passed (45%).
The pass rates in each component were: Morphology 61%, Coagulation 68%, Transfusion 76%, Oral 79%.
8 candidates (21%) failed a single component (4 Coagulation, 4 Morphology). 7 candidates (18%) failed 2 components (Morphology was a component in 5, Coagulation in 4). 4 candidates (11%) failed 3 components (Morphology and Transfusion were components in all 4). 2 candidates (5%) failed all 4 components.
Observations at the End of One Year
The Part 1 examination is providing a pass rate comparable to the written component of the old format Part 1 Essay Papers (68% ± 13 for 2004-6). It provides a test of a trainee’s knowledge across a broad sampling of the curriculum as well as a more in depth test of a trainee’s knowledge, experience and judgement in 4 major topics.
No component of the old format examinations is an appropriate comparison for the Part 2 Examination. The pass rate so far is below the range experienced in the old format Part 1 Practical Examination (57% ± 14 for 2004-2006) which was set at a lower standard and it is significantly below the pass rate for the old format Part 2 Oral examination (88 ± 14% from 2004-6) which simply tested candidates whose practical skills had been tested previously, on 8 pre-published topics.
Examiners reported that the morphology papers were particularly poorly answered in both these examinations and more candidates failed Morphology (7) than Coagulation (6) in the Spring examination. Coagulation had proved to be the major hurdle in recent Part 1 Practical Examinations. Interestingly, failure in Morphology and in Coagulation appears to be mutually exclusive in this group of candidates.
As a result of the introduction of the new format examinations, the range of candidates who sat the Part 2 Examination in 2007 was not typical of candidates coming through to either of the old format examinations. Candidates eligible for this examination include those who have failed the Part 2 Practical Examination on several occasions after passing the essay component. One third of candidates had failed that examination on 2 or more occasions and may not have developed the laboratory skills to the level necessary for the new ‘exit exam’ standard. Candidates in these two examinations who had never sat the old format practical component or had a single previous attempt at the old format practical examination had a pass rate of 53% which approaches that of the old test of practical skills.
The feedback from examiners is clear: many candidates for the examination are not achieving the skill level in morphological diagnosis or the investigation and interpretation of coagulation disorders required in this examination. Transfusion skills may be being ensured by attendance at prescribed transfusion training sessions. However trainees seem not to be acquiring the laboratory skills currently required of a Consultant Haematologist in the areas of morphology and coagulation.
The Specialty Advisory Committee is aware of the problem and the clinical pressures on Departments and trainees and is considering possible solutions. Nonetheless it is essential that trainers ensure that trainees receive adequate laboratory training to achieve the level of expertise expected in a Consultant Haematologist. Otherwise the future of the British Haematologist with both clinical and laboratory skills and responsibilities may be jeopardised and we may move towards the American model.
Revalidation
Professor Adrian Newland
Successive Governments of whatever flavour have over the last 25 years sought to decrease the perceived autonomy of professions, particularly doctors. Unfortunately, the actions of a minority of doctors have only served to increase this resolve. In these times of “no blame” culture, there’s still plenty of room to blame doctors - and the other healthcare professions are not far behind.
The endgame of this sequence of events began with the publication of the White Paper ‘Trust, Assurance and Safety – The Regulation of Health Professionals in the 21st Century (2007)’.covering the regulation of healthcare and associated professions. Initially, the focus will be on the regulation of medical doctors.
The two components of revalidation are relicensure and recertification. Both will be a positive affirmation of the doctor’s entitlement to practise and confirmation from the College to the GMC that a doctor has met appropriate standards. Common elements in the process will hopefully enable the implementation of one process with two outcomes.
The Royal Colleges will be responsible for recertification through setting standards, arranging comprehensive assessment of doctors against those standards and issuing statements of assurance to the GMC. It is suggested that recertification should occur on a 5-year cycle.
The process is being driven by the Academy of Medical Royal Colleges Revalidation Development (formerly Implementation) Group that meets every 8 to 10 weeks. The RCPath has representation on this group.
The College’s own short-life Revalidation Task Force had its first meeting on 6th March and considered revalidation work streams, funding and communication strategy.
The brief of the task force includes the development of fair and transparent standards and processes for revalidation that meet the needs of pathology specialties and the GMC. The task force will also commission tools that are developmental, supportive, and which assist doctors in recertification by solving problems early on. It will ensure consistency and share good practice across pathology specialties, and identify and develop appropriate systems for the delivery of revalidation.
The College work streams for revalidation are:
• Standard setting
• Assessment methodologies
o Multi source feedback
o CPD scheme development
o Clinical audit
o EQA
• E-portfolio
Thus the brief includes all the elements of the work of the Professional Standards Unit, and determines our development programme for the foreseeable future.
The Academy is starting to classify the work streams differently, though the above list enables us to focus on this College’s tasks rather than those more relevant to the Academy. To make matters slightly more confusing, the AoMRC’s Organisational Response to the White Paper on Medical Regulation - Trust, Assurance and Safety is entitled “Specialist Recertification Project Plan”.
Standard Setting
The GMC has reviewed the headings of Good Medical Practice (GMP)
Four domains are now proposed:
o Knowledge, skills and performance
o Safety and Quality
o Communication, partnership and team work
o Maintaining trust
The GMC have produced a Draft Framework for Appraisal and Assessment derived from Good Medical Practice as a foundation for revalidation.
Assessment Methodologies
The Academy now has a work stream entitled “Defining Standards and Criteria for Methods and Evidence used to demonstrate Specialist Practice” which includes these elements. There will be a CPD Working Group, an e-Portfolio Working Group, an MSF Working Group, a non-clinical Working Group and a Remediation Group (for “struggling doctors”).
Multi Source Feedback
The overarching aim is that any pathologist or trainee pathologist only needs to use a single MSF tool . Nevertheless, the work of the RCPath will conform to the principles set by the Academy, who are establishing a group to consider how Colleges can support MSF for revalidation including advising on content, usability and cost.
Continuing Professional Development
At the Academy level, there is a research project is funded by the GMC focusing on what motivates doctors to do CPD; how they engage with or access CPD; how/where do doctors find out about CPD; the educational and learning benefits of CPD; and whether CPD has any tangible impact on clinical practice.
Within the RCPath the PSU has also reviewed the scheme to ensure an appropriate spread of CPD credits between categories.
Interpretative External Quality Assurance
EQA is a good example of a continuous assessment process and enables the early identification of concerns about performance. It is intended to build on the success of these schemes by extending them to appropriate specialties.
To facilitate this links will need to be made between the schemes (where they exist) and the College speciality groups. Such links already exist in Histopathology. I have had contact with the NEQAS scheme organisers for Interpretative schemes in Haematology and Clinical Chemistry and intend to meet with both of them. I have also had correspondence with NQAAP Chairs and the Joint Working Group. The challenge is to construct a mechanism that enables both satisfactory and unsatisfactory performance to feed into the revalidation process.
e-Portfolio
The development of e-portfolios for medical professionals is a key topic at present and one that is very relevant to the College. The Academy group will focus on the details and specifications of e-portfolios for revalidation. There is recognition of the importance of developing an e-portfolio for revalidation from the outset. It was essential that such a system be in place if the roll out of revalidation is to be successful
Funding and Timing
The GMC have been funding the Academy to carry out work on revalidation for the last two years. The Department of Health are making further monies available to support the work. There is clear recognition of how challenging this work is, recognised in the more realistic timetables for revalidation hinted at by the consultative literature accompanying new legislation proposals. Thus the GMC wishes to start piloting work around revalidation in 2009, very different from the original roll out of 2010, and there may be need for further amendment legislation once the outcomes of the pilots are known. The Impact Assessment tells us the White Paper will be reviewed in 2011, when the costs of revalidation “may be known”. Beyond the projected costs of pilots in 2008/9 of £4m there's no mention of costs or how they will be recovered. The devil is in the detail so watch this space.
Pernicious Anaemia Society
The Pernicious Anaemia Society is the only organisation in the world that offers information, advice and support to sufferers of Pernicious Anaemia and their families. Information and support for Pernicious Anaemia patients is provided on the society website (www.pernicious-anaemia-society.org), most of the information and support being provided by fellow sufferers. The society is a registered charity (Charity No. 1115195) whose executive chairman is Martyn Hooper. The society can be contacted at PO Box 245, Bridgend, CF31 9FB Tel. 01656 720187/0113 815 3130.
The Pernicious Anaemia Society is keen to forge links with medical professionals and is currently in a tripartite investigation into the genetics of PA being undertaken at Manchester University.
BSH Annual Scientific Meeting, Glasgow, 2008
Professor Ken Mills, Scientific Secretary
The BSH Annual Scientific Meeting held in Glasgow in April 2008 incorporated the 6th Bi-Annual I-BFM Leukaemia Symposium. Over 1000 people attended the meeting from approximately 40 countries introducing a truly international flavour. The scientific, educational and clinically orientated sessions reflected the best of haematology across all disciplines and also highlighted common themes between adult and paediatric malignancy. The popular NEQAS and BCSH sessions also showed the need for the meeting to address the whole range of issues affecting clinical and scientific haematologists at all levels and locations. Three excellent plenary sessions across the meeting also reflected these different themes and disciplines.
The record number of submitted abstracts for the meeting reflected the high quality of the oral and poster presentations. The number of free communications sessions was increased and included the re-introduction of the Presidential symposium featuring the six highest scoring abstracts with the prize being awarded to Dr KM Debatin from Ulm, Germany for the presentation entitled “Time to leukaemia (TTL) assessed in NOD/SCID mice transplanted with primary ALL leukaemia cells determines early relapse in patients and is identified by a specific gene signature”. Other prize-winners were: A Hayes (Cardiff) for the Nursing Prize, Dr Izraeli (Israel) for the Paediatric Malignancy prize and Dr Jawad (Nottingham) won the Young Investigator Prize. The poster prize winners for Monday and Tuesday are available on the Annual Scientific Meeting section of the BSH website.
The social highlight of the meeting had to be the dinner that was held at the spectacular Kelvingrove Art Gallery, where the acoustics of the main hall were used to the full by the operatic “singing waiters”.
If you attended this year’s Annual Scientific Meeting, then you will already know that is was a great meeting with excellent scientific, translational and clinical presentations; if you didn’t attend then pencil it into your diary: April 27th – 29th 2009 in Brighton.
