Basics

The new Checklist AI Assistant (aka ‘Prefill’) feature provides a checklist prefill solution for faculty using AI-based analysis of transcribed interactions (such as patient-provider simulations).
It helps teachers complete individual student checklists during video reviews.

Using the transcript text as a basis, the AI attempts to answer questions and provide recommendations - which teachers can accept or ignore.

Example:

If a student greets the SP during an encounter, this will be evident from the conversation transcript.
Let’s say the student greeted like:”Good morning Mr Wolf”.
If the FON checklist includes a question asking whether the student greeted the SP, the AI could automatically provide a "yes" response or suggestion.

This tool significantly

• reduces the workload of faculty

• reduces the time and effort needed for human reviewers

• automatically maps specific sentences from interactions to relevant checklist items

• maintains accuracy.

By structuring responses in this way, the application produces clear, consistent outputs for evaluation.

AI Prefill is available to admins and faculty members (all users who previously had access to data entry functions).

Pre-Conditions for Prefill Generation

The Checklist AI assistant feature only activates when the following preconditions are met:

• A video has an associated transcript.

• The video has an associated FON checklist.

• The Checklist AI assistant Availability setting is enabled in your LearningSpace instance (System Manager).

Enabling Checklist AI assistant

The Checklist AI assistant (Prefill) Availability setting can be enabled in the System Manager, Ehli tab, under ‘AI Functions’ by Admin or System Manager privileged users.

Ehli Tab - AI Functions toggles

AI Functions enabled

Once the admin user has enabled the AI Functions, the features become available to users (based on their permissions).

Use the toggle to enable Ehli AI Checklist Assistant for your account (same for OEQS Scoring).

How does the Checklist AI assistant work?

1. Once a video recording is completed (when the user clicks the stop button), the transcript begins generating.

2. Immediately afterward, the AI begins evaluating the answers.

3. Once this is done, both the Transcript and the AI assistant become available.

4. The AI assistant is represented in purple throughout the LearningSpace.

If a recording has an AI assistant, a purple box appears below the top row on the Video Review/Data Entry tab. This box contains a brief description and a toggle that the user can use to reveal the AI's answers.
If the AI has not yet run or the transcript has not been generated, the toggle will be disabled, and a hover message will indicate that the system still needs some time.

Faculty users can toggle the Prefill for a given checklist at the top of the checklist during Data Entry.

Enabling the Ehli AI Checklist Assistant: Toggle the switch at the top of the Data Entry field to make the AI visible.

Users can toggle the AI on and off at any time. If you manually accept the AI's answers, these answers will remain in the input field even if the AI is turned off. Users can still edit the answers, but all purple elements (indicating AI suggestions) disappear from the screen.

Jump to Source

The AI makes its recommendations based on the transcript, specifically on what was said in the video. These can be considered evidence that the user can verify. The Jump to Source function provides timestamps that the user can click on to view this evidence. This could be a single timestamp or multiple timestamps.

Confidence Level

The AI can score the confidence of its answers. This allows faculty members to know whether they need to double-check a particular question.
There are 3 categories, indicated by stars: ⭐️ = least confident, ⭐️⭐️⭐️ = most confident.

If you wish to accept the answers suggested by the AI (ChatGPT), click the Use AI Answer button.

You can withdraw the selected AI answer with the Undo button.

Supported Checklist Items

Users can apply the prefill service for the following types of Checklist Items:

• Radio Button

• Yes/ No

• Correct Technique - IT - ND

• Drop-down

• Checkbox

• Text

• Matrix

  
  

  For checkboxes and radio buttons: items are highlighted in purple with a shadow behind them.

  

  Radio button items highlighted

  
  

  
  

  

  Checkbox items highlighted

  
  

  
  

  For dropdown and free-text: A purple box containing the recommended answer appears. If the user hovers over a purple item, a tooltip appears with additional information.
  For dropdown questions, the purple box displays the correct answer from the dropdown, which the user can accept by clicking the button.

  
  

  
  
  

  
  

  Any text entered by the user in the free text or dropdown fields will be replaced by the AI-generated answer upon clicking the "Use AI Answer" button. A warning message highlights this behavior. “Clicking the button will overwrite your current response.”

  
  

  

  Items not numbered in the checklist (Item counter turned off in the Case Content)

  
  

  
  

If AI cannot generate answers, the following notice is displayed:

If the AI cannot find a suitable response to any of the questions asked, display this message to the user. “Unfortunately, the AI was unable to find an answer to any of the questions in the checklist. We are unable to assist you at this time.”

If LS cannot reach the AI server for any reason, an error message will appear: “AI server is currently not available. Please try again later.”

Best Practices for Writing Checklists for the AI Checklist Assistant

For the AI to perform its job more accurately and effectively, the checklist items themselves must be written clearly and intentionally. Our Best Practices for Writing Checklists for the AI Checklist Assistant guide explains how to write Questions and Answer options that the AI can reliably interpret and match to the transcript.

User Rights and Responsibilities

• Acceptance of Terms

By enabling the Checklist Prefill feature, you agree to these Terms of Use set forth by Elevate Healthcare.
A disclaimer below the submit button states that the AI may make mistakes. This is a legal requirement.

• Access to Checklist Prefill Feature

If you disable the Checklist Prefill Availability setting, LearningSpace will cease generating prefills for FON checklists.

• Data Privacy and Security

You retain ownership of all content provided to LearningSpace, including transcripts and FON checklists. You have the right to access, correct, and delete your data in accordance with Elevate Healthcare’s data management policies.

Limitation of Liability

Elevate Healthcare and its providers are not liable for unauthorized access to user data or indirect, incidental, or consequential damages associated with the Checklist Prefill feature. This limitation does not intend to restrict any legal rights you may have.

Amendments to Terms

Elevate Healthcare reserves the right to modify or update these Terms of Use to reflect changes in our services or compliance requirements. Users will be notified of any material changes, and continued use of the Checklist Prefill feature after such modifications constitutes acceptance of the revised terms.