Watch the walkthrough
The English interface uses isolated example data. Changes and results in this walkthrough are demonstrations.What analysis produces
For each analyzed call the result can include:- Sentiment —
positive,neutral, ornegative(overall caller sentiment). - Success —
true/false(ornullwhen success evaluation is off), plus a short reason explaining the verdict. - Data — a map of the structured fields you defined, keyed by field name.
- Analysis time — when the result was produced.
GET /api/v1/calls?sentiment=&success= to find matching calls quickly.
Analysis runs after the call and never affects the live conversation. If analysis is unavailable, the call and its transcript remain unchanged and no result is shown.
Configuring analysis
Open Assistant Settings → Analysis and turn on the parts you need. Sentiment is enabled by default. To disable extraction, remove its fields; disable Sentiment and Success as well to turn off analysis.
Assistant Settings → Analysis: configure User Sentiment, Call Successful and Extracted fields. AI QA scorecards appear separately when Beta features are enabled.
1
Sentiment
Enabled by default. Turn it off if you don’t need per-call sentiment.
2
Success criterion
Enable Success and describe, in plain language, what a successful call looks like — e.g. “The caller booked an appointment” or “The caller confirmed their delivery address.” The result contains a boolean plus a reason.
3
Structured fields
Add up to 30 fields with a unique name, a type and a description (up to 500 characters). Every field is attempted; missing scalar values remain empty rather than being invented.
Structured extraction types
- Text, Number, Yes / No return a string, a number or a boolean.
- Choice (enum) uses your allowed values. Allow multiple values returns all matches as an array; off returns one value. Under When nothing matches, choose an allowed value to save as the fallback. Without a fallback, an unmatched single choice is empty and multiple choice is an empty array.
- Object defines fixed fields (up to 20). Every key is present; unavailable values are empty. Child fields support Text, Number, Yes / No and Choice, including optional allowed values for Text.
- List defines the same fixed fields for each item. The AI returns one object per entry it finds, up to 100 items, or an empty array when none are found.
- JSON lets the AI choose the structure. Use Object or List when your webhook needs fixed keys. Free-form JSON is limited to 16,000 characters and 8 nesting levels.
Example configuration
calls.analysis looks like:
Using analysis results
- History filters — filter the call list by sentiment and success to find, say, all negative calls that did not succeed.
- Public API — every call in
GET /api/v1/callsandGET /api/v1/calls/{id}carriesanalysis,sentiment, andsuccess. Filter the list with?sentiment=negativeand?success=false. - MCP — the same call fields are exposed through the MCP
list_calls/get_calltools.
Re-running analysis on a past call — History → Re-evaluate — overwrites the stored sentiment, success, and extracted fields with a fresh result. It costs extra credits at the workspace’s History re-evaluate rate; current rates are on the Usage page.
AI QA scorecards
AI QA scorecards (Beta) score every finished call against your own quality checklist — a separate, complementary result from the sentiment/success/fields analysis above. Configure it in the same Analysis section as post-call analysis.1
Turn scorecards on
Off by default. The card is only shown once Beta features are switched on for the workspace under Settings → Workspace, and enabling it needs a plan that includes AI QA scorecards.
2
Set a pass threshold
An overall score from 0–100; a call passes once it meets or exceeds this value.
3
Add up to 20 criteria
Each criterion has a name, a weight (how much it counts toward the overall score), and a source:
- LLM judge — the AI reviewer reads the conversation, including what the assistant did, and scores the criterion from a free-text description you write. All LLM judge criteria of a call are scored together in one review.
- Reuse success — reuses this call’s Analysis success flag (yes = 1, no = 0), without another review.
- Reuse sentiment — reuses this call’s Analysis sentiment (positive = 1, neutral = 0.5, negative = 0), without another review.
PATCH /api/v1/assistants/{id}, the MCP update_assistant tool, or famulor update-assistant <id> --qa-scorecard-config '<json>'.
What the judge sees
The judge reads the conversation in order, with a time stamp on every line; in very long conversations, the middle part is shortened. Besides what the caller and the assistant said, it sees what the assistant did: tool calls and their results, handoffs to another assistant, transfers, and how the call ended. The system logged these lines; the caller did not hear them. A criterion such as “Transfers only during service hours” is therefore met when the assistant checked the time with a tool and then transferred, even if it never said so out loud. This works the same for phone calls, web calls, WhatsApp calls and web chat. When Redact PII in transcript is on for the assistant, the judge receives the same redacted text as the stored transcript. If a criterion cannot apply to a conversation, for example “Offers a callback” in a call that was transferred right away, the judge marks it Not applicable. It is left out of the overall score instead of counting as 0.Scorecard results
A call without a score counts neither as passed nor as failed. It stays out of QA averages, pass rates and the Audience Call QA average. Calls without caller speech are never sent to the judge. In History, criteria without a score show Not applicable, Not scored or Unavailable instead of 0 %. To list calls without a score, filter History by QA result → Not scorable.
In the API, MCP and webhooks,
qa_scorecard.status is scored, not_scorable or judge_unavailable, reason says why a call has no score, and score and passed are null unless the call was scored. See Post-call webhooks for an example.
This is different from the cohort-level AI Quality Assurance tool, which runs a QA pack you choose against a batch of past calls on demand. Scorecards run automatically, per call, as it finishes.
Configuring via the API
UsePATCH /api/v1/assistants/{id} or the MCP update_assistant tool to update the same analysis options programmatically. An empty configuration keeps default sentiment analysis enabled. See the API reference for the request schema.
Turn a real call into a simulation test
On a finished call in History, use Create Simulation Test. Famulor prepares a persona, script, and success criteria from the transcript and analysis, then opens the assistant’s Simulations panel for review. The same action is available as:POST /api/v1/assistants/{id}/tests/from-callwith{ "call_id": "…" }(assistants:write; Simulations must be included in the plan)- MCP tool
create_assistant_test_from_call