Bots - Get Maker Evaluation Test Run
Get Maker Evaluation Test Run by ID.
Get the status and results of a specific maker evaluation test run.
GET https://api.powerplatform.com/copilotstudio/environments/{EnvironmentId}/bots/{BotId}/api/makerevaluation/testruns/{TestRunId}?api-version=2024-10-01URI Parameters
| Name | In | Required | Type | Description |
|---|---|---|---|---|
|
Bot
|
path | True |
string |
The bot ID. |
|
Environment
|
path | True |
string |
The environment ID. |
|
Test
|
path | True |
string (uuid) |
The test run ID. |
|
api-version
|
query | True |
string |
The API version. |
Responses
| Name | Type | Description |
|---|---|---|
| 200 OK |
Successful response. |
Security
oauth2
Microsoft Entra ID OAuth2
Type:
oauth2
Flow:
implicit
Authorization URL:
https://login.microsoftonline.com/common/oauth2/authorize?resource=https://api.powerplatform.com
Scopes
| Name | Description |
|---|---|
| .default | .default |
Definitions
| Name | Description |
|---|---|
|
Evaluation |
The state of an evaluation run describing the currently active stage. |
| Metric |
A grader metric associated with a test case in a maker evaluation run. |
|
Metric |
The error reason providing additional information about a metric evaluation result. |
|
Metric |
The result of a grader metric in a maker evaluation run. |
|
Metric |
The status of a grader metric result in a maker evaluation run. |
|
Metric |
The type of a grader metric in a maker evaluation run. |
|
Test |
The result of a test case in a maker evaluation run. |
|
Test |
The state of a test case in a maker evaluation run. |
|
Test |
The response payload for a get evaluation run details request. |
EvaluationRunState
The state of an evaluation run describing the currently active stage.
| Value | Description |
|---|---|
| Abandoned | |
| Cancelled | |
| Completed | |
| Deleted | |
| Failed | |
| InProgress | |
| Queued | |
| Unknown |
Metric
A grader metric associated with a test case in a maker evaluation run.
| Name | Type | Description |
|---|---|---|
| result |
The result of a grader metric in a maker evaluation run. |
|
| type |
The type of a grader metric in a maker evaluation run. |
MetricErrorReason
The error reason providing additional information about a metric evaluation result.
| Value | Description |
|---|---|
| AgentResponseIsNullOrEmpty | |
| EmptyOrInvalidModelResponse | |
| ExpectedInvocationStepsAreNullOrEmpty | |
| ExpectedKeywordsAreNullOrEmpty | |
| ExpectedOutputIsNullOrEmpty | |
| GraderCreationFailed | |
| InputOutputCountMismatch | |
| IntentMatchInvalidMatchType | |
| ModelLabelGraderInvalidLabel | |
| QueryIsNullOrEmpty | |
| RequestTokenLimitExceeded | |
| RetrievedKnowledgeSourcesTextsAreEmpty | |
| RetrievedKnowledgeTokenLimitExceeded | |
| UnexpectedInternalError |
MetricResult
The result of a grader metric in a maker evaluation run.
| Name | Type | Description |
|---|---|---|
| aiResultReason |
string |
The reason generated by LLM, providing additional context to the evaluation result. |
| data |
object |
The set of key-value pairs that defines this metric. Keys are normalized to lowercase. |
| errorReason |
The error reason providing additional information about a metric evaluation result. |
|
| status |
The status of a grader metric result in a maker evaluation run. |
MetricStatus
The status of a grader metric result in a maker evaluation run.
| Value | Description |
|---|---|
| Error | |
| Fail | |
| Pass | |
| Unknown |
MetricType
The type of a grader metric in a maker evaluation run.
| Value | Description |
|---|---|
| AllKeywordMatch | |
| AnyKeywordMatch | |
| CapabilityUse | |
| CompareMeaning | |
| CustomLabels | |
| ExactMatch | |
| GeneralQuality | |
| TextSimilarity | |
| Unknown |
TestCaseResult
The result of a test case in a maker evaluation run.
| Name | Type | Description |
|---|---|---|
| metricsResults |
Metric[] |
The list of grader metrics associated with the test case. |
| state |
The state of a test case in a maker evaluation run. |
|
| testCaseId |
string (uuid) |
The ID of the object model entity that the test case is sourced from. |
TestCaseState
The state of a test case in a maker evaluation run.
| Value | Description |
|---|---|
| Cancelled | |
| Completed | |
| Error | |
| Running | |
| Unknown |
TestRun
The response payload for a get evaluation run details request.
| Name | Type | Description |
|---|---|---|
| cdsBotId |
string (uuid) |
The CDS bot ID where the evaluation run is executed. |
| endTime |
string (date-time) |
The timestamp when the evaluation run ended. |
| environmentId |
string |
The environment ID where the evaluation run is executed. |
| id |
string (uuid) |
The unique identifier for this evaluation run. |
| mcsConnectionId |
string |
The MCS connection ID used in the evaluation run. |
| name |
string |
The name of the evaluation run. |
| ownerId |
string |
The ID of the owner who initiated the evaluation run. |
| startTime |
string (date-time) |
The timestamp when the evaluation run started. |
| state |
The state of an evaluation run describing the currently active stage. |
|
| testCasesResults |
The test cases and their results in the evaluation run. |
|
| testSetId |
string (uuid) |
The ID of the object model test set entity record that the run is associated with. |
| totalTestCases |
integer |
Number of test cases included in the evaluation run. |