1. Ledger
   1. Test execution type
      1. Automated integration testing
      2. Automated UI testing
      3. Manual testing
   2. Test execution status
      1. Pass
      2. Fail
2. Design Changes
   1. Verify the Color changes For
      1. Evaluation Header
      2. Comment Box
      3. Feedback Box
   2. Evaluation Header Verfication
   3. Remove the scroll bar
      1. Verify the scrollbar should only be visible when user is hovering on evaluation screen
      2. Verify the above flow from other screen
   4. Change in audio log colour
      1. Verify the colour of the audio log
   5. Verify the following components as per the new design
      1. Description
      2. Feedback
         1. Adding a feedback designs
         2. Editing a feedback design
         3. History feedback icon
         4. Original feedback text box
            1. Thumbs up and thumbs down button
         5. No feedback design
         6. Feedback draft state
         7. Feedback on emails
         8. Thumbs up design
         9. Thumb down design
         10. Thumb down text box design
      3. Comment
   6. Audio buttons
      1. Verify the timestamp
         1. Call
         2. chat
         3. email
3. Functional Flows
   1. Ability to Provide Feedback
      1. Thumbs Up/Down Functionality
         1. Verify users can give thumbs up on auto-QA’ed questions.
         2. Verify users can give thumbs down on auto-QA’ed questions.
         3. Login with different users for same conversation and verify the thumbs up and down flow
      2. Additional Feedback on Thumbs Down
         1. Verify users can provide additional feedback when giving thumbs down on auto-QA’ed questions.
         2. Verify users can select "Evidence drafted by Auto-QA is incorrect" when providing additional feedback on thumbs down.
         3. Verify users can select "Answer selected by Auto-QA is incorrect" when providing additional feedback on thumbs down.
         4. Verify users can enter additional details in the text field when providing feedback on thumbs down.
   2. Unified Evidence Experience
      1. Metric Tag Evidence
         1. Verify the presentation of metric tag evidence adheres to the specified format.
            1. When metric tag threshold was followed
            2. *Name of the metric tag*in this conversation was *below/above sort of operator* *duration*
            3. When metric tag threshold was not followed
            4. *Name of the metric tag* in this conversation was not *below/above sort of operator* *duration*
      2. Conversation Tag Evidence
         1. Verify the presentation of conversation tag evidence aligns with the defined structure.
            1. When conversation tag was followed (Agent tags)
            2. The agent followed *Name of the conversation tag* in this conversation. The agent said:
            3. *Timestamp* : *Utterances that we show in snippets
            4. When conversation tag was not followed (Agent tags)
            5. The agent did not follow *Name of the conversation tag* in this conversation. The agent said: *Timestamp* : *Utterances that we show in snippets*
            6. When conversation tag was followed (Dynamic tags) For ex Convo ID 91237
            7. The agent followed *Name of the conversation tag* in this conversation. The agent said: *Timestamp* : *Customer utterances that we show in snippets* *Timestamp* : *Agent utterances that we show in snippets*
            8. When conversation tag was not followed (Agent tags) For ex Convo ID 91251
            9. The agent did not follow *Name of the conversation tag* in this conversation when the customer said: *Timestamp* : *Customer utterances that we show in snippets*
            10. When conversation tag doesn’t exist (Dynamic tags without customer utterances) For ex, convo ID 91238
            11. The customer did not mention phrases or words related to *name of the conversation tag*.
      3. Guidelines Based Evidence Format
         1. Verify the breakdown of guidelines evidence in the specified format.
            1. Guidelines evidence should be broken down into this format : *Summary*. The agent said : *Timestamp* *Utterances* *Timestamp* *Utterances* *Timestamp* *Utterances*
      4. Timestamp Experience
         1. Verify clicking on play button or timestamp directs the user to the exact utterance in the transcript or call map.
            1. Call
            2. The exact utterance in the transcript (call, chat and email)
            3. The exact moment in the call map (in case of call)
            4. Verify clicking on a timestamp in call evidence takes the user to the specified time in the audio timeline and highlights the full utterance.
            5. Eg: Start time: - Clicking on timestamp (such as 01:35 - highlighted in yellow) takes the user to 01:35 min in the audio timeline, plays the audio and highlights the full utterance (yellow as per the evidence) or exact words in parallel (whichever requires less effort for V1)
            6. Verify the audio continues playing till the end of the highlighted utterance.
            7. End time - The audio should continue playing till the end of the utterance (end of yellow part)
            8. Chat
            9. The exact utterance in the transcript (call, chat and email)
            10. Verify clicking on a timestamp in chat & email evidence highlights the exact utterance in the transcript.
            11. Verify the entire utterance presented in the evidence is highlighted.
            12. Start time: - Clicking on timestamp (such as 01:35) highlights the exact utterance in the transcript (yellow as per the evidence) End time - The whole utterance (as per specific timestamp) presented in the evidence should be highlighted
            13. Email
            14. The exact utterance in the transcript (call, chat and email)
            15. Verify clicking on a timestamp in chat & email evidence highlights the exact utterance in the transcript.
            16. Verify the entire utterance presented in the evidence is highlighted.
            17. Verify how transcript identification works for email conversations when using timestamps.
            18. Start time: - Clicking on timestamp (such as 01:35) highlights the exact utterance in the transcript (yellow as per the evidence) End time - The whole utterance (as per specific timestamp) presented in the evidence should be highlighted
         2. Edit of time stamp
            1. Verify that users can edit timestamps within the specified format (e.g., 00:00).
            2. Verify that editing timestamps in the middle of the evidence line does not affect the timestamp format.
      5. Metabase verification
         1. Verify the following data are stored in metabase
            1. Tenant/Org
            2. Question
            3. Convo ID
            4. Original Answer (auto-QA’ed)
            5. Updated Answer
            6. Date & time of changes
         2. Verify that the Metabase table captures changes in custom form answers and evidences, and confirm the workflow from suggestion to final suggestion to user-edited
      6. Draft comment flows
         1. . Verify if users can save feedback drafts before submitting the edited feedback.
         2. Verify the state management for draft feedback, ensuring the last feedback is retained for the submitted evaluation.
         3. Verify other draft feedback flows
   3. Add/ Edit Evidence Experience
      1. Edit Auto-QA’ed Evidences
         1. Verify users can edit auto-QA’ed evidences using a rich text editor.
         2. Verify users can rephrase words/phrases, modify timestamps, and save changes.
         3. Verify the flow after clearing the evidence, and saving it,
      2. User Detail on Manual Evidence Edit
         1. Verify the title changes to “Evidence - [name of the user]” when evidence is manually edited.
      3. Add Evidence for Manual Question
         1. Verify users can add evidence to manual questions in the manual rubric, including timestamp conversion.
         2. Verify users can go to the exact place in the transcript and call map using added timestamp.
      4. Overwriting of Feedback
         1. . Verify that if multiple users provide feedback, the final version, based on permissions, supersedes.
            1. Verify the permissions chart for editing and confirm that the final version is shown before submission.
      5. Multiple Feedback
         1. Verify that multiple feedback entries are not allowed for a single evaluation.
   4. Comment Button Visiblity
      1. Verify the comment button is enabled only after evaluation submission.
      2. Verify all the previous comment flows
      3. Verify the accept and dispute flows for comments
   5. Export Feature
      1. Verify the Evaluation Comments should be downloaded in CSV
      2. Verify the Evaluation Feedback should be downloaded separately in CSV
      3. Verify the export file
      4. Check with plat team for any export functionality issues
   6. Calibration Flow Changes
      1. Calibration Review Screen Changes
         1. Verify all users can view, edit, and add AI responses and evidences in the calibration review screen, with applicable UI changes.
            1. View
            2. Edit
            3. Add
         2. Verify remaining same flow as evaluation
         3. Verify that the new UI elements (e.g., header) are visible in the calibration review screen.
      2. : Calibration Report View Changes
         1. Verify only user-edited evidences and manual evidences are visible in the calibration report view.
         2. Verify that comments are replaced by user-edited evidences in the calibration report view.
   7. Transcript/Snippet Highlighting
      1. Verify that when switching between transcript/snippet tabs, evidence for conversation tags is always directed to the transcript.
         1. Verify that the connectivity is shown even when users switch between tabs.
         2. Verify the handling of metric tag "dead air" where it points to the snippet with no connectivity to transcript.
         3. Verify that when data is not available, it connects to the snippet.
   8. Post-Evaluation Actions
      1. Verify if users can provide thumbs up or thumbs down after the evaluation
      2. Verify if evidence editing is allowed post-evaluation.
   9. Edge cases
      1. Evaluation delete
         1. In case of evaluation deletion, manually added feedback/evidence will keep back or will delete? We need to retain the auto-QA feedback and keep the rest of the behaviour same as comment
      2. Page refresh
         1. if a user does thumbs up or thumbs down and he refreshes the page then on hover blue thumbs up/down should be visible or white thumbs up/down
      3. Thumbs down are independent of users
         1. Login with different users and verfy
      4. Different Versions of Edited Evidence
         1. Confirm that different users can have different versions of edited evidence and verify the support for drafts.
      5. Rubric Changes
         1. Verify that changes in evidence remain safe and visible when users choose back the same rubric after a rubric change.
   10. Audit Log, Notification & Email related changes
       1. Verify the regression for these flows
4. Permission cases for feedback
   1. Adding Evidence
      1. Verify QA Auditor can add evidence.
      2. Verify Admins/SA can add evidence.
      3. Verify Agent cannot add evidence.
      4. Verify Manager cannot add evidence.
   2. Editing Evidence
      1. Verify QA Auditor can edit their team's evidence.
      2. Verify Admins/SA can edit any evidence.
      3. Verify Agent cannot edit evidence.
      4. Verify Manager cannot edit evidence
   3. Feedback Provision
      1. Verify Agent cannot provide feedback.
      2. Verify Manager cannot provide feedback.
      3. Verify QA Auditor can provide feedback on evidence.
      4. Verify Admins/SA can provide feedback on any evidence.
   4. Evidence Review
      1. Verify Agent can view edited evidence.
      2. Verify Manager can view edited evidence.
      3. Verify QA Auditor can view edited evidence.
      4. Verify Admins/SA can view edited evidence.
   5. Original Evidence Visibility
      1. Verify Agent cannot view original evidence once edited.
      2. Verify Manager cannot view original evidence once edited.
      3. Verify QA Auditor can view original evidence once edited.
      4. Verify Admins/SA can view original evidence once edited.
   6. Access to Unedited Evidence
      1. Verify Agent can view original unedited evidence.
      2. Verify Manager can view original unedited evidence.
      3. Verify QA Auditor can view original unedited evidence.
      4. Verify Admins/SA can view original unedited evidence.
   7. Extras
      1. States for user when they dont have permission to evaluate
         1. Dont show empty feedback boxes
         2. Dont show the lower state of the feedback boxes
   8. Multiroles
      1. Verify that users with multi-role permissions have the expected access.