Installation & Setup

This guide covers the one-time setup required after installing (or upgrading to) the Real Events package version that includes AI Scoring for Evaluations. These steps are performed by a Salesforce administrator. Once complete, scoring runs automatically and no further setup is needed.

Scoring will not work until all of the steps below are complete. Most "Failed" results on a fresh install trace to a missed step here — work through them in order.

Step 1 — Confirm an active Scoring Rubric

The package ships a Default Scoring Rubric so scoring works out of the box.

• Setup → Custom Metadata Types → Scoring Rubric → Manage Records.
• Confirm a record exists and Active is checked.
• To use your own criteria, create a new record with your prompt, mark it Active, and deactivate the Default (only one rubric may be Active at a time).

If no rubric is Active, scoring fails with "No active Scoring_Rubric__mdt found."

Step 2 — Enter your OpenAI API key

The package does not ship an API key — each org supplies its own.

• Setup → Security → Named Credentials → External Credentials tab → open OpenAI.
• In the Principals section, open the principal (named rie__OpenAI_…) → Authentication Parameters.
• Enter your values:
  • APIKey — your OpenAI secret key (begins with sk-)
  • Org — your OpenAI Organization ID (begins with org-)
• Save.

Get these from the OpenAI platform dashboard (platform.openai.com): the API key under API keys, and the Organization ID under Settings → Organization → General. The OpenAI account must have active API billing/credit. (API access is separate from a ChatGPT subscription.)

Step 3 — Set the endpoint reference in the Scoring Rubric correctly

This is a required, easily-missed step. In the active Scoring Rubric record, the Endpoint field must reference the packaged Named Credential by its fully namespaced API name:

Endpoint = rie__OpenAICompletions

Not OpenAICompletions. Because the Named Credential is a packaged component, scoring code must reference it with the rie__ namespace prefix. Using the unprefixed name causes the callout to fail to resolve the credential — it's blocked before reaching OpenAI, appearing as a "couldn't access the endpoint" error with status code 0.

Step 4 — Grant access to the External Credential

The user (or process) that runs scoring must be granted access to the OpenAI External Credential's principal. Assigning a permission set alone is not enough — the principal access binding must be explicitly enabled.

1. Create or choose a permission set for scoring users (Setup → Permission Sets).
2. In that permission set, open External Credential Principal Access, click Edit, and move the OpenAI principal (rie__OpenAI_…) into the Enabled list. Save.
3. Assign that permission set to every user who will trigger scoring — including any user who saves Evaluations into "Ready to Score" status, and the user under which any scheduled scoring runs.

Without this binding, the callout is blocked inside Salesforce and scoring records show "Failed" with a "couldn't access the endpoint" error.

Step 5 — Confirm the scoring trigger flow is active

• Setup → Flows → confirm ChatGPT Evaluation AI Scoring Trigger is Active.
• This flow fires when an Evaluation's AI Scoring Status changes to Ready to Score, and starts scoring automatically.

Verify the setup

1. Open an Evaluation, populate the answer fields and credibility inputs, and set AI Scoring Status to Ready to Score. Save.
2. Refresh after a few seconds (there may be a short delay from ChatGPT).
3. The status should change to Scored (or Verify Claims), and the score/rationale fields populate.

If the status changes to Failed, check the debug log and match the cause to the steps above:

Setup checklist

• An active Scoring Rubric exists (Default, or your own with Default deactivated)
• OpenAI APIKey and Org entered on the External Credential principal
• Scoring Rubric Endpoint = rie__OpenAICompletions (namespaced)
• OpenAI principal enabled under External Credential Principal Access on a permission set
• That permission set assigned to all scoring users
• ChatGPT Evaluation AI Scoring Trigger flow is Active
• A test Evaluation scores successfully end-to-end

Last updated 1 month ago