SP onboarding process

1. Registration of the SP user in User Manager

2. Training - Practice tool: watching videos and SP assessment - SP Training

3. Observation - SP Performance Assessment (SPPA)

4. Evaluation - SP Performance Review report

In the User Manager, when the role is set to SP or the Privileges include SP, three additional settings surfaces are displayed (See also Additional Settings for SPs):

1. Personal Details settings: These are for entering customized data specific to and necessary for managing SP users.
   

   Two additional features reveal themselves when closing and reopening the User account/Edit user window.

2. Physical & Medical History settings: These are used to register the medical history and conditions of the SP user. 
   The conditions listed here are real conditions of the SP users based on their own medical history.
   
   

3. Activity settings: for recording Activities (with specific details) the SP user has participated in.
   
   

   

SPs can be registered as described in SP Enrollment by enabling the Standardized Patient Administration (SPA) module within LearningSpace.

SP Specific Search

A specific way to search for specified SPs is using the Search users, groups field of User Manager by pre-selecting “SP” in the respective dropdown menu and selecting the checkboxes for the desired SP sections under Which SP sections?:

       

You can search for SPs in the LearningSpace database in a more focused way based on the above added Personal and Physical data.

To upload this data, you can use the Batch User Create (BUC) feature of LearningSpace by exporting an Excel file of a specific SP group, including all the entered data in separate columns. (User manager > Batch User Create tab > Step 1 > Download template > Select SP group from dropdown > Download)

SP Privacy Mode

When Privacy Mode is enabled, no one can view the camera feeds in your room, allowing you to change clothes or participate in sensitive scenarios with complete privacy. The cameras will be blocked until you turn Privacy Mode OFF.

When you enable Privacy Mode, the camera feeds of the room are disabled:

SP Evaluation

The SP checklist (SP Part) you can set up in the Case Manager serves for the SP’s evaluation of the Learners' performance. To evaluate SPs, you can use the following tools (with templates you can customize to a limited degree) of LearningSpace:

SP Training part

This part serves for self-training/self-evaluation, as a practice for SPs, assessing other SPs' performance. It provides a means for analyzing and measuring how standardized patients perform compared to other (more experienced) SPs. This tool allows SPs to study web-based training material, view sample video streams easily, watch “gold standards” performances, practice checklist completion, and check understanding by completing the checklist and comparing their answers to others. You can save time, get equal or better results, and provide self-directed learning for your SPs.

General criteria for setting up and using the SP Training Part

1) The Activity used for SP Training has to be running or upcoming.

2) The Case in which the SP Training is set up has to be assigned to the Activity.

3) a completed SP part (checklist) and a video recording of the Case must be completed.

4) The SP doing the SP Training has to be a member of the Activity's SP group.

Setting up the SP Training part in the Case Manager

By selecting the SP Training tab from the Case part, you will find three sections: Practice, Evaluation, and Materials (these sections are different from the regular SP part sections). The order of these sections is strict: a Practice section has to be set up before an Evaluation section, and placing the Materials section is flexible. The SP Training section must include a Practice and an Evaluation section; optionally, it can also include a Materials section.

Setting up the Practice section

1. On the Practice section page, you can select the encounter for the Trainee.

2. From the 'Activity' drop-down, select the Activity the Trainee will do in the SP Training (all those activities, including the current case, are listed in this drop-down).

3. Then, you can select a learner from the 'Learner' drop-down (this drop-down includes only those learners who are members of the related activities and have a video recording for the Case. After saving this page, you can set up the Evaluation section.     

   

Setting up the Evaluation section

The Evaluation section shows the results of the Trainee for the selected encounter compared to this 'control' section. Upon submitting the Practice section checklist, this evaluation is displayed for the SP Trainee. On the Evaluation section page, you can select the Practice section you have set up before from the' Selected section' drop-down.

Setting up the Materials section

The Materials section is optional. You can use it to give the trainee some instruction or additional information. In this section, you can select only the Note item (or Multimedia and/or Message Field in the Classic Case editor).

By selecting the Message Field item, you can provide any textual information. By selecting the Multimedia item, you can provide further information for the Trainee.

1. Upcoming Activity: Normal SP Data Entry is unavailable, so SP Training will be accessible and displayed under Current Activities in the Your Activities panel.
   
   

2. Running Activity - SP Training access can be controlled via the 'Data entry only allowed from manually authorized workstations' setting in the Activities module.

   1. When an SP logs in from an unauthorized computer, they can access the SP Training but not the SP Data Entry.

   2. When an SP logs in from an authorized computer, they can access the SP Data Entry.

Reports

The SP Training results can be found in the SP Training Review on the Test Activity Management tab in the Reports.
 

Calculating the score for the Trainee is based on the original Case and Activity. When the SP Trainee (the SP doing the SP Training) submits responses for the checklist, their answers are compared to those of the SP originally filling in the SP part for the Case. When an SP selects an Activity for doing a training part, their submitted answers are always saved to the database of the selected event

SP Performance Assessment (SPPA)

This checklist serves as an evaluation of SP performance. 
Only users with Observer privilege can complete this part; the results affect the SP performance score. Only users with Observer privilege can do the SP Performance Assessment (SPPA), and only if they are members of the Faculty group of the Activity.

There are two methods of SP assessment in LearningSpace:

a. Filling out the same checklist the SP has completed, then comparing the Observer's and the SP's answers, thus getting a different value for the SP within a Case.

b. Filling out the SP Performance Assessment Case part of an Activity, in addition to the SP checklist. 

⚠️ Another important condition is to have the correct settings adjusted on the Activity Reporting setup page of the respective Activity. Here, the SP Performance Assessment table sets how the required number of observations should be calculated and what the SPPA report should look like.

Observers can access the SPPA module via Classic view > Data entry tools > SPPA Live Observation or SPPA Post-Encounter Observation.

SP Performance Assessment report

The SP Performance Assessment Report is used to evaluate the performance of each SP for each assigned encounter and Case. It is accessible under Preset Reports, Test Activity Management module.

It shows a table listing all the Cases of the Activity with separate lines for each SP who participated in a certain Case. Each row shows the main SP-evaluating scores, and there are also tools to review the observations for each encounter.

Inter-rater Agreement

The accordance of answers given by the SP and the observer for the same SP checklist is expressed in percentage. (this value is the absolute value of the difference between the value of the answer selected by the SP and the one selected by the Observer. If, for an item, they both select the answer with the same value, then the difference value for this item is 0. The difference value for an item is zero even if they selected different answers if those answers have the same point value. This is because the difference does not affect the learner's score: it would be the same if the SP did exactly as the Observer. (Of course, if the answer selection for an item is important and student scores depend on the SP’s choice, the answers should have different point values.)

The difference value calculation only uses responses to items with pre-defined answers. Responses to open-ended items like text boxes are NOT used, not even if they are scored.

SP Performance Evaluation: The result of the SPPA checklist, if any, expressed in percentage. It refers to one encounter and is calculated similarly to the learner scores. Here, we are scoring the SP, not the learner. The SP gets as many points for each section of a certain part of the Case as the answer selected by the observer is worth. Then, the system calculates the percentage score for each section of this part and gets the SP Performance Evaluation value for this observation by calculating the weighted average of the section percentage scores using the section weights, as set in the Case Manager.

Total SP Performance Score: The mean of the Inter-rater Agreement and the SP Performance Evaluation results. (the average of the inter-rater agreement value and the SP performance evaluation value.)

Section agreement Value: This percentage value refers to one section of the SP part and is the difference of the total achievable non-open-ended point value in the section and the difference value divided by the total achievable point value. 
 

SP Performance Review

This shows the results of the SP checklists by listing all the participating SPs along with the number of learners they have evaluated (in brackets) and the scores they have given to each section of the SP checklist (abbreviations in the headings of the respective columns) expressed in percentages and their Average value.

Name: (115)—the number of learners the SP has submitted Data entry for; SP part sections and averages.

What is it used for? If the average score is remarkably different (shows big discrepancies), it might be due to the SP being less efficient, having a bad day, or needing more training. It is NOT connected to the SP Performance Assessment.

It would be handy for checking which SPs handled a specific Case, but it only shows records for a specific Activity.

4. Which report shows how often a specific SP has worked with a specific student?

- The SP Activity feature of the User Manager can be used for this.

• “SP-s who did this “- feature of Case manager

(No specific report can be run in the Reports module of LS.)

SP Training

Set Data entry only allowed from manually authorized workstations and log in from a non-authorized workstation to access this:

AFTER filling in the same checklist the SP did (SP part of the Case), the Evaluation appears:

SP Training report shows the same Agreement value:

REPORTS SHOWING RESULT OF OBSERVATION:

SP Observation Comparison (SPPA Report > Observations dropdown)

The answers of SP (SP) vs. Observer (O). If different, a difference value is added ‘1’.

In the SP part, those items where the answers selected by the SP and the observer are different are highlighted. (The items of the SPPA part are filled in only by observers, so here you will not see "differences” but the answers of the observer.)