Skip to main content
Blackboard Help

Evidence Collection Process

In Outcomes Assessment, evidence is the collected sample of student assignments. This evidence is analyzed for the evaluation of programs, courses, research projects, faculty and student achievements, and more.

Assessment Managers and Assessment Administrators can create evidence collections, sample the collections, and assign evaluators.


Create Evidence Collections

You create evidence collections to gather evidence from courses that is relevant for a specific project or assessment activity. For example, create an evidence collection to gather all assignment submissions from courses within a particular program or Business Unit that are aligned to a specific goal.

You can set up an evidence collection by defining the criteria used to collect artifacts from Blackboard Learn courses. You define an evidence collection by:

  1. Access Goals and Assessments on the Outcomes tab or the link on the My Institution tab. In addition, administrators can access it on the administrator panel under Tools and Utilities.
  2. Click View Collected Evidence to see a list of existing collections.
  3. Click +Collect Evidence to create a new collection.
  4. On the Collect Evidence page, click Find Goals.
  5. On the Discover Goals pop-up window, select the goals the evidence must be aligned to for inclusion in the collection. Click Submit.
  6. In the Define Collection Options section, select where evidence will be collected from: the entire system, a limited scope, or from specific places.  Within each of those options, you can further narrow the criteria by restricting dates.
  7. Give the collection a name. You can specify start and end dates which can be used to help you manage the timelines of various collection projects and understand when projects have been completed.
  8. Click Submit if you are finished or Collect More Evidence if you need to create another evidence collection.

In order to collect evidence, content within courses must be aligned to the relevant goals. See Goals for details about creating and aligning content to goals. 

Sample a Collection to Create an Evidence Set

An evidence set is the subset of artifacts, which are student assignment submissions that meet the collection criteria, that were randomly drawn from the evidence collection. You create an evidence set by sampling the evidence collection to reduce the number of artifacts that will be evaluated for institutional assessment activities. After sampling, evaluation can occur.

  1. On the View Evidence Collections page, select a collection name.
  2. On the collection's page, you can access the:
    • Status of the collection at the top of the page
    • Buttons for your next available action
    • Collect tab to display the items that match the evidence collection criteria
    • Sample tab to display the randomly selected subset of artifacts from the collection. This tab appears after sampling has occurred.
  3. Click Sample Artifacts. If the collection was already sampled, you can click Sample Again.
  4. In the pop-up window, choose to Include All Artifacts or Randomly Select Artifacts. If you choose to randomly select, type the total number of artifacts needed or the minimum number of artifacts needed from each assignment.
  5. Click Submit. The sampling process runs immediately and only takes a few seconds to complete.

Evaluate Artifacts in an Evidence Set

You select one or more evaluators to review artifacts in an evidence set. They use a rubric to evaluate the artifacts that have been assigned to them. Any user in the system can be added to an evaluation session. They do not require any special permission or access. Evaluators are emailed invitations to review an assigned selection of student submissions within an evidence set.

  1. On the View Collected Evidence page, select a collection name. The collection must be sampled before you can create an evaluation session.
  2. Click Create Evaluation Session.
  3. In the pop-up window, under Select Evaluators, browse for one or more evaluators.
  4. For Evaluation Setting, select the number of evaluators that need to evaluate each artifact.
  5. For Grading Rubrics, browse for a rubric for evaluators to use.
  6. Select a Due Date.
  7. Check the Privacy box to enable anonymous evaluation.
  8. Click Start Session. Users are automatically notified by email that you have selected them as evaluators for this session. A notification is also posted in the Updates section of My Blackboard. Both the email and the notification contain a link to the Evaluator View.

About Evaluation Sessions

Evaluators see only the artifacts assigned to him or her. This enables the assessment manager to more effectively compare the scores of different evaluators of the same artifact and better understand inter-rater reliability.

The assessment manager can return to the evaluation session page to monitor the progress of various evaluators. They can opt to check the artifact evaluations themselves, drop or add evaluators, and close the session. If evaluators are added or removed, the system automatically redistributes the remaining non-evaluated artifacts across the updated list of evaluators.

After an evaluation is complete, the assessment manager can run a report to understand the results. If they do not choose to perform a separate outcomes evaluation on collected evidence, but instead want to report on the rubrics scores collected from assignments in Course Delivery, this can be accomplished immediately after the evidence is sampled.