About SP Performance Assessment
Privileges
Only users with Observer privilege can fill out the SP Performance Assessment (SPPA), and only if they are members of the Faculty group of the Activity.
What is SP Performance Assessment?
SP Performance Assessment offers a quantitative approach to measure the performance of your SPs.
Why Apply SP Performance Assessment?
To compare SPs’ performances more objectively to each other
To make sure that learners are training with SPs whose portrayals are adequately consistent and realistic
To identify those SPs who might need additional training to better their performance
How Is SP Performance Assessed?
Only a user granted the Observer privilege can conduct an SP Performance Assessment, which consists of two progressive phases:
Inter-rater Agreement
You can measure the accuracy of an SP's assessment skills with the Inter-rater Agreement Method.
Here, the user with the Observer privilege observes the SP in action while the SP portrays a scenario.
Meanwhile, the Observer fills out the SP checklist for the Case to set up the expected standard for the SP's data entry.
The Observer should complete the SP checklist as if they were portraying the scenario to create a benchmark for the SP data entry.
Setting the benchmark depends on the learner's performance who is currently featured in the scenario:
If a learner does not, e.g., use the correct technique, the Observer should remark it accurately on the SP checklist. From the observed SP's point of view, this is the correct answer, not the answers marked as correct in the Case Content editor.If the SP still provides positive feedback on the SP checklist despite a learner's poor performance, then the SP will receive a lower score on their Inter-rater Agreement.
After the scenario, the SP also completes the SP checklist for the same learner. The system then checks how the SP's data entry correlates with the Observer's data entry.
Conditions for Observer Inter-rater Agreement
Learn more about setting up an Inter-rater Agreement here.
SP Performance Assessment Checklist
Use an additional data collection method alongside the first one: the SP Performance Assessment Checklist.
Here, the Observer fills out a checklist to evaluate the SP's performance.Conditions for SP Performance Assessment Checklist
Find out more on the Conditions for the SP Performance Assessment Checklist page.
Setting Up Data Entry Tools for SPPA
IMPORTANT: Data Entry Tools for SPPA can be set up in Classic view, for which the diagonal arrows button at the right end of the header has to be clicked.
Expanding the Data Entry Tools tab in Classic view displays the SP Performance Assessment - Live Observation panel.
Select a running Activity from the drop-down menu, click Select,
then narrow it down using the Room, Activity, Case, and Learner drop-downs.
On the SP Performance Assessment - Post-Encounter Observations panel:
An Activity has to be selected from the Activity drop-down,
then clicking Select will navigate to the Post-Encounter Observation page.
Clicking Go next to the encounter intended to be observed will redirect to a page
where the video recording of the encounter can be watched, and the SP and/ or SPPA checklists, if any, can be filled out.
Setting Up SP Performance Assessment
IMPORTANT: When making a post-encounter observation, the system gives a randomly generated list of encounters that should be observed.
These encounters are selected so that, together with currently existing observations, they will satisfy the Observation Requirement, which refers to the number of encounters that should be observed.
On the Preset Reports tab of the Reports module, select the Activity you want to work with. Click the Activity Reporting Setup link at the bottom of the tab.
NOTE: The link only appears when an Activity has been selected.
On the SP Performance Assessment tab, select the desired preferences from the radio button options and specify the percentage/ number via the drop-down menu:
% from all encounters - the required number of observations calculated based on all the encounters
% from the encounters of each SP - the required number of observations calculated based on the number of encounters each SP has
encounter(s) per SP per day (event) - the required number of observations equal to the number of encounters each SP has a day/ event
Accessing SP Performance Assessment Report
IMPORTANT: The SP Performance Assessment Report is used to evaluate the performance of each SP for each assigned encounter and Case.
It shows a table listing all Cases in the Activity, with separate lines for each SP who participated in a given Case.
Each row shows the main SP-evaluating scores and includes tools to review observations for each observed encounter.
The report can be accessed by selecting the desired Activity on the Preset Reports tab of the Reports module, which activates the SP Performance Assessment button in the Test Activity Management column.
The SP Performance Assessment report is only available when the Activity has already been selected.
On the SP Performance Assessment Report page, the following scores are displayed:
Inter-rater Agreement: The accordance of answers given by the SP and the observer for the same SP checklist is expressed as a percentage.
SP Performance Evaluation: The result of the SPPA checklist, if any, expressed in percentage.
Total SP Performance Score: The mean of the Inter-rater Agreement and the SP Performance Evaluation results.
SP Observation Comparison
When you select an observation from the Observations drop-down, the SP Observation Comparison screen appears, where any differences between the SP's and the Observer's answers about the selected learner's performance are highlighted in yellow.
For further information, see the Case Manager pages and the Case Manager Training Videos.