Skip to main content
Blackboard Help

Migrate from Command Line to SIS Framework Snapshot Flat File

This topic is for Blackboard Learn administrators who have implemented and maintain an automated batch integration using the Snapshot Tool in the Blackboard Learn product.  The intent of this topic is to provide direction and guidance for updating the integration from using the command-line interface (CLI) Snapshot tools to the new SIS Integration Framework.  Thus the primary focus of this document is on replicating as closely as possible existing CLI integration strategies using the SIS Integration framework.

The content of the topic is based on some important assumptions:

  • The information only applies to Blackboard Learn 9.1 SP8 or later releases. 
  • The information assumes you are moderately familiar and experienced with the CLI Snapshot tools and data management in Learn.
  • The information assumes assumes you are using text-delimited, “flat” files for your Snapshot integration.
  • Deployment assumptions are based on typical implementations facilitated by Blackboard Consulting and include certain effective practices, which may not be exactly the same for your deployment.

What’s not covered in this document?

  • Information about implementing Real-Time Integration
  • Detailed or extensive deployment guidance
  • Training in the complete use of the SIS Integration Framework
  • Details about new or extended features like:

How to manage course availability through “Terms.”

How to setup and use Institutional Hierarchy Nodes.

How to build and use custom field mapping scripts

A Note on Deprecation

In September 2012, Blackboard officially announced that the Command-Line Interface (CLI) Snapshot Tools had been deprecated and started encouraging customers to consider migrating existing integrations to the new SIS Integration Framework, introduced in the summer of 2011.

It is important to note that deprecation doesn’t mean unsupported or removed.  The CLI Snapshot Tools continue to be supported interfaces for integration with Learn in 9.1 SP10 and the foreseeable future.  The CLI Snapshot tools will also receive ‘critical’ maintenance until they are removed from the product. As such Deprecation should be considered an announcement to begin the process of preparation for the end of something not an end in itself.

Specifically, deprecation is the announcement that a product or capability has been replaced or is no longer necessary and will be removed at some point in the future. This allows planning for change management. Additionally, announcing deprecation effectively ‘freezes’ development of the effected product – in the case of the snapshot command-line tools this means they will remain at release 9.1 SP7 capabilities – and that the product will receive full support and only ‘critical’ maintenance until it is removed from the product.

Given that CLI Snapshot Tools are deprecated and ‘frozen’ at 9.1 SP7 capabilities it is important to understand that this means that new platform features (for example, Terms, Course Merge, and Hierarchy) have not and will not be added to the capabilities of the CLI Snapshot.

When Blackboard determines the release in which Snapshot will be fully removed from the product, the customer community will be given extensive advanced notice so that you may plan your transition accordingly.

Official information and FAQ are presented on the Blackboard Support site in KB Article 43468.

How will deprecation affect my Snapshot integration?

Currently, not at all. However, the best practice of announcing deprecation allows you to begin to proactively assess your integration approach moving forward and identify the necessary changes you will need to make to support the new Framework Snapshot functionality. Note that the Framework Snapshot XML provides a path for those who currently use the XML file format in their Snapshot integration. It does not and will not support new capabilities post 9.1 SP7.

Snapshot Integration History

For years, thousands of customers have used the command-line interface (CLI) called Snapshot to implement data integration with external systems and Blackboard Learn.  Snapshot in the command-line interface context collectively refers to the command line tools called snapshot, snapshot_override, and dsm. These tools are to be replaced by the SIS Integration Framework Snapshot integration type.

With Release 9.1 Service Pack (“SP”) 6, Blackboard enhanced the integration capabilities of the Learn platform substantially.  The Student Information System (SIS) Integration Framework we’ve built is designed to provide, “out-of-the-box” tools and an extensible infrastructure to support robust, standards-based, flexible integrations with a variety of Student Information Systems.   In summary, with the SIS Framework, administrators have GUI access to everything necessary to create, configure and monitor several simultaneous integrations, including:

  • Custom mapping of fields from the SIS data to attributes in Learn
  • Use of JavaScript (via the Rhino interpreter embedded in Learn) to manipulate the data and support custom business practices
  • GUI interface for manually uploading files on demand
  • Web interface for automating file and event processing
  • Access to detailed logs with variable logging verbosity settings

In SP6, we included with the new framework three “integration types” based on IMS standard data formats.  In SP8 we added two more types based on the formats and capabilities of the old CLI Snapshot Tool.  They are:

  • Snapshot XML: Loosely based on IMS 1.0 with several Blackboard specific extensions.
  • Snapshot Flat File: Character delimited text files.  (This is the only type discussed below)

9.1 SP6 and SP8 also expanded Learn product features that have data integration implications.  These features include the Institutional Hierarchy, Course Enrollment Merging, and Terms.

Given the complexities of working cooperatively in standards organizations like IMS, newer Learn product features may take extra time to fully implement using such open standards. Since Blackboard defines the Snapshot Flat File format, new Learn features can be immediately implemented using this well-known data format. The Snapshot Flat File format, being a simple text file, also provides the added advantage of being the easiest to create.


Four significant changes occurred between the new SIS Framework Snapshot Flat File and CLI Snapshot. Specifically:

  • GUI-based Implementation Configuration and Management
  • File Delivery
  • Data Source Keys and Data Ownership
  • Field Mapping and Scripting

Each of these must be taken into consideration as you move from CLI Snapshot to the SIS Framework Snapshot Flat File and will be covered in depth in the below Details section.

GUI-based Implementation Configuration and Management

Installing, configuring and managing SIS integrations are significantly easier when using the SIS Framework.

Past: With CLI Snapshot, all the configuration files, logs and the tools were only accessible at the command line – either on the application server or through the Snapshot client install. With the Framework, all interfaces for deployment, configuration and troubleshooting are now GUI based. 

Present: Now the people who administer the integration don’t need command line access to the Learn servers to implement, execute or maintain the integration.  This new configuration overcomes many constraints related to security, hosting server location, and the end-user skills that limited customer’s adoption of automated integration.

File Delivery

Providing data to Learn when using Snapshot is done via files. How this is done is different than with previous snapshot based integrations and will require some adjustment to existing processes.

Blackboard Consulting Controller:  For customers who have been using the Blackboard Consulting Snapshot Controller, moving to the Framework will make that software obsolete.  We will discuss how you replace it and the implications for ICM (maintenance services) in a later section.

Past: When using the CLI Snapshot, there were generally two steps involved in moving and processing the data files from the SIS.   First, the files had to be moved to the Learn server - usually by FTP - then the Snapshot tool had to be called to process the files using JDBC or SOAP to submit the data to the Learn database.

Present: With the Framework, the data is introduced directly to Learn by POSTing files to defined, secure HTTPs “end points”. Files POSTed to Learn are immediately processed by the integration, so all automation of the integration process is external to the Learn server. 

Also important to note, the SIS Framework supports manual uploading of files too.  Whereas with snapshot you had to run the command line, now a Learn administrator can simply upload a batch file in the GUI.  Again, this gives control to GUI administrators who, due to security rules or system skills, may not have been able to execute one-off command-line runs.

Data Source Keys and Data Ownership

For more information and history on Data Source Keys follow the Data Source Keys Overview knowedge path.

Past: The CLI Snapshot tools used the Data Source Key (DSK) attribute of records to group them together for certain bulk operations.  In particular, when running CLI Snapshot in the SNPSHT mode, the DSK allowed you to implicitly disable records associated with a specific Data Source Key without disabling all the records in the system. 

Present: The SIS Framework version of the SNPSHT mode is called “Complete Refresh”, and rather than using the Data Source Key to identify the group, it uses the integration deployment membership. This can be a difficult concept to understand, even if you’ve completely internalized how the DSKs work with the CLI Snapshot.  This will be covered in more detail below.

Field Mapping and Scripting

Previous snapshot integrations required that all data be prepared before creating the files used with the CLI. This required adjustment to SIS systems, middle-tier software, or pre-processing of the data to properly generate the snapshot files.

Past: With CLI Snapshot, virtually all of the business logic resided in the “generator” software, which extracted the data from the SIS.  In order to make even small changes to the integration, Learn administrators would need to go back to the SIS programmers and request changes to the underlying queries. 

Present: The SIS Framework provides the Blackboard administrator in-system tools to take control of the logic by enabling remapping fields and/or using simple, custom scripts through the admin GUI to provide the proper data to Learn.

When you transition from CLI Snapshot to Framework, you may not require this feature immediately, but you should build familiarity over time to make the most use of the capabilities later.

Known Issues

Blackboard is continuing work on enhancements to the SIS Framework and the Snapshot Flat File integration type. Know problem points focus on reporting of processing success and the ability to operate on a set of data using multiple integration types.

Only simple HTTP status codes are provided to indicate the success/failure of POSTing files.  Framework currently has no detailed response to external systems regarding the success or failure of data processing.  Self-hosted customers may implement methods to query logs, but these methods can be complex to implement and aren’t supported/documented by Blackboard.

There is no supported method in the Framework to change the Integration “ownership” of a record from one integration to another.  Making this change requires delicate updates to certain database records.  Again, this process isn’t currently documented or supported by Blackboard.

Review the version specific known issues documentation for up-to-date notes regarding product issues.

Summary of Significant Changes

  • SIS Framework is 100% GUI.  No need for back-end access to implement or manage.

  • All configuration is completed via the GUI – the file is no longer used.

  • Snapshot doesn’t support data elements and capabilities introduced in the Learn platform since 9.1 SP6.  (E.g. Terms, Course Merge, and Institutional Hierarchy)

  • HTTP POST delivers files to ULRs on Learn secured with a username and password defined when setting up the Integration.

  • All scheduling and automation is external to Learn.  (No more Bb Consulting Controller)

  • Posting of Files: cURL is one popular option for scripting procedures to POST files, but any technology that supports HTTPS POST of text files can be used.

  • Snapshot modes (SNAPSHT, MANUAL, REMOVE) have been replaced with Framework operations (“Full Replace”, “Store”, and “Delete”).

  • CRS_COPY_INTO operations have been incorporated into the standard processing of course data in the Framework. (Applies to Organizations too.)

  • (Complex) Whereas Snapshot used the Data Source Key to determine on which records to work, the Framework uses “membership” in the particular Integration deployment.  Thus, during “Full Replace” operations, the SIS Framework will only disable records that were created by the integration you’re working in currently.

  • Field “Ownership” which was determined by the file is now determined by a checkbox that controls whether the field should be subject to “Change on Update?”

  • File field headers are no longer hard-coded.  You can name fields anything you want and map them in the GUI to the standard Learn fields.  You can even add extra fields for use with custom scripting – giving you control of the business logic of the integration without (always) going back to the file generating scripts.

  • File footer is not utilized in the Snapshot Flat File format and will cause an error if included in the submitted data file.

  • You can run the Framework integration in “testing” mode, which will evaluate the data for errors, but not commit data/changes to Learn.

Making the Move

It is necessary to understand the differences between the CLI Snapshot and SIS Framework Snapshot in order to address changes required for a successful transition from CLI to Framework-based Snapshot.

Object Types and Files

All of the CLI Snapshot object types (files) are supported in the Snapshot Flat File type in the SIS Framework in Learn 9.1 SP8 and later.  Also, certain new features in Learn have added new data object types that are not supported by the CLI Snapshot but are supported by the Framework.

Not all the naming conventions are the same as CLI Snapshot nor are they entirely consistent within the SIS Framework interfaces.  Most everything is obvious, but we’ve included some notes in this section to help navigate the differences.

For the Course Delivery license the supported Object Types are:

  • Users (aka Person)
  • Courses
  • Enrollments (aka Course Membership or just Membership)

    The feature “Standards, Objectives and Goals” was only available to K12 licenses before SP8, but is now available to all customers.  In the CLI Snapshot these were called Learning Standard Alignments.

The following object types have been added to Course Delivery and are not available to the CLI Snapshot tool:

  • Terms

With the Community Engagement license:

  • Secondary Institution Role Associations  (aka User Secondary Institution Role)
  • Organizations
  • Organization Enrollments

    As you review the SIS Integration Framework interface, you should explore the features under Advanced Configuration to understand what the Header values should be and which attributes are required.  Since this is all displayed in the GUI, the online product documentation doesn’t describe these items explicitly.  (This is in contrast to the old Advanced Data Integration Guides that listed out all the fields for CLI Snapshot.)

    For more detailed information on Headers and data see the Examples and Data Format knowledge paths.


    Figure 1: Custom Field Headers and Field Mapping

    By selecting the Field Mapping interface you can see the fields that are required by looking at the “Required for Insert” column. By selecting Custom Headers you can see the full list of default header values.

    If you are very familiar with the CLI Snapshot formats you may realize that we have exposed even more attributes for the objects you can manage through the Framework.  Some, but not all of them are related to the new features (eg. Terms).

    While it is also unlikely that you will use this feature immediately, the SIS Framework allows you do define the data header names to suit your needs.  Not only can you rename an existing header attribute, but also you can add additional headers. You will be able to select these headers from the Source Field drop downs on the Field Mapping page and in custom scripting. For more detailed information on Field Mapping and Custom Scripting see the Examples knowledge path.

    Setup of Snapshot Flat File

    Detailed instructions and options for setting up the Snapshot Flat File integration type may be found in the Snapshot Flat File Configuration Knowledge Path.

    Past: The CLI Snapshot uses the configuration file to get a large amount of information about how it should operate.

    Present: With the SIS Framework, virtually all of that information is now implemented through the GUI.


    Figure 2: Portion of the Snapshot Flat File properties page. Accessible when creating a new integration or by selecting 'edit' from the integration's contextual menu.

    As you can see in Figure 2, this properties page captures configuration information like the file delimiter and log verbosity that were in the file.   Other elements of will be configured elsewhere – described in later sections.

    You should also notice this is the interface that maintains the user/password for the integration; as well as controls the integration status.  The status allows you to run your integration in testing mode, where you can see the results in the log without actually committing the data changes to Learn.

    Snapshot Flat File Operation Modes

    In this section we are going to expand on how the Framework relies less on the use of Data Source Keys for managing groups of data. The various modes for processing data relating those to known CLI modes and function are also covered.

    Operation Modes

    Similar to the CLI Snapshot, the Framework allows you to execute the processing of data files in three modes:

    Snapshot Flat File Data Ownership vs SNPSHT and Data Source Keys

    While Data Source Keys are still important to the management of records in the system, they are not as critical to the operation of the Framework as they were with the CLI Snapshot.

    The big change comes from the fact that the CLI Snapshot tool required the Data Source Key to do a SNAPSHT operation, because that's how it sets the boundaries of what was safe to disable from the system, since SNAPSHT operations implicitly delete all records from the data source that are not found in the feed.

    With the analogous SIS Framework operation (Complete Refresh), the Data Source Key does NOT control the boundaries of what’s safe to disable, these boundaries are controlled by whether the record was created by a specific integration.  We may state that a record is “owned” by the integration that created it.

    In both cases, the reason for the safety boundaries is to prevent a SNAPSHT operation or a SIS Complete Refresh from deleting too many records, including potentially system-created user accounts and so forth.  Note, it remains possible to run disables and purges in Learn by Data Source Key using the Data Sources Admin Panel tool.

    Like with CLI Snapshot, the SIS Framework can change the DSK of a record.  However, there is no currently supported method to change a record’s integration ownership.

    How to Use the New Methods

    The implication for your CLI integration will depend on your use of multiple Data Source Keys and the SNAPSHT method.  You will have to figure out the best configuration for your institution, but we will use a common deployment configuration as an example to help your planning.

    Because it was one of the features of the Blackboard Consulting Snapshot Controller, it is common for customers to use the name of the data file to define the Data Source Key of the records. A typical set of integration files and corresponding DSKs might look like this:

    Integration Files and Corresponding DSKs
    Data Type Contents Filename
    Users Active user accounts Users.txt
    Secondary Institution Roles 2ndary role Assignments 2ndary_roles.txt
    Courses Fall 2012 Courses FA2012_CRS.txt
    Student Enrollments Fall 2012 Enrollments FA2012_ENR.txt
    Instructor Assignments Fall 2012 Instructor Assignments FA2012_STAFF.txt
    Courses Winter 2013 Courses WI2013_CRS.txt
    Student Enrollments Winter 2013 Enrollments WI2013_ENR.txt
    Instructor Assignments Winter 2013 Instructor Assignments WI2013_STAFF.txt

    Table 1: Example DSKs and Files

    If you want to use the “Full Replace” method in the SIS Framework, you will need to define separate integration deployments that segregate groups of data of the same type.   With the example above, we would recommend that you deploy three integrations - one for the data types that are not term based and one for all the files that belong to a particular term.  The result might look like this:


    Figure 3: Example Integration Deployments

    This would allow you execute a “Full Replace” on the Fall 2012 enrollment records without inadvertently disabling any other enrollments on the system.  And note, do to this, you don’t NEED to have separate integrations for every file.

    Data/Field “Ownership” Control

    The CLI Snapshot used properties in the files to identify the attributes in Learn that would be controlled or “owned” by Learn.  By listing an attribute, the CLI Snapshot would ignore that field when performing data updates.  Hence, users could change these values in the Learn GUI and not have the integration process overwrite them.

    In the Framework, this same control is maintained at the attribute level by a checkbox in the Field Mapping interface.  In the capture below, the green arrow indicates the column “Change on Update?”  When the box is checked, the values in the file will overwrite whatever is in Learn.  With the box unchecked, the attribute will be populated during an INSERT, but ignored for all subsequent UPDATEs.


    Figure 4: Ownership Checkbox

    For example: Let’s assume that you pre-populate Learn with users’ passwords based on some known convention.  Your data file includes this initial value for each user.  However, you encourage users to change their password when they first log-in and give them the option to change their password whenever they want.  If you un-check the box, subsequent runs of the integration will NOT reset the password.

    Course Copy/Template Changes

    The new framework supports the use of Course Copy and Course Templates just as does CLI. The key difference is operational and configuration.

    Course Copy

    Past: Course Copy configuration is done using the file. Course copies (COPYINTO) are done using the CLI tools CRS_SNPSHT or CRS_MANUAL with the –t option and the TEMPLATE_COURSE_KEY field in the feed file.

    COPYINTO operations use the Template_Course_Key to specify the source of the copy operation based on the ownership setting in the file.

    Present: Course Copy configuration is done using the SIS Framework admin GUI. Course Copies are completed as part of the processing of STORE or FULL REPLACE requests. The feed files must contain the TEMPLATE_COURSE_KEY (Source Course Copy Key in the UI).

    In all cases using the COPYINTO is only meant for populating new Courses. Using this method with existing Courses will not copy the data. Settings for Course Copy are found in the Advanced Configuration Settings for the integration under Courses:Course Copy Settings.

    For more information on Course COPYINTO and using Templates see the Examples knowledge path.


    As discussed, the SIS Framework includes HTTP “end points” which are the programmatic interfaces for delivering the data files to the Learn system.  To automate batch integration, you will need to establish your own method for securely posting your files to these endpoints that does not require human intervention.  Customers are encouraged to work with technology that is familiar and easily supported at your institution.

    Also, you will have to decide the exact location from which the files are going to be posted.  Consider both the security and stability of the system(s) to ensure that your integration runs and that no unauthorized people have access to the data.

    Figure 5: Shows the endpoints for a fictitious integration.  Notice how each entity/mode has a unique URL.  Also, it is important to recognize that it is the integration Username that distinctly identifies this integration from others.  All integrations of the type Snapshot Flat File use the same URLs – the username determines which integration is executed.


    Figure 5: HTTP End Points

    Automation Scripting: A cURL Example

    To give you an example of how this can be done, we are providing some instructions for how to use cURL, a command line tool for transferring data with URL syntax. You can download the version of cURL you want here: Make sure to use a version that supports SSL.

    <p class="note>The following example is for a Windows system. cURL must be downloaded and installed. The appropriate binary may be acquired at the curl project website: <a class=" external"="" data-cke-saved-href="" href="">

    Linux systems may or may not require installation of cURL. Downloads are available from the curl project website:

    Select the latest stable version (highlighted) for your version of Linux.

    See the Snapshot Flat File Automation Knowledge Path for more information regarding automating a Snapshot integration type.

    With the curl.exe file on the PC, we execute a post using the following command syntax:

    curl -k -H "Content-Type:text/plain" -u 2d2f8833-90b7-444e-8bea-514111d5be70:password --url --data-bin @courses_fall2012.txt

    The syntax is explained below:

    At it's most simple, you can build a batch or shell script file that contains the commands you want to execute. Make sure you follow the logical order of creating data objects before you try to assign them to something. (For example, run your Users and Courses files before you run Enrollments).

    This is the most basic example of a batch file that would execute the upload of three files.   The assumption is that you would name this something like integration_post.bat and then create a scheduled job to run automatically at times when you know the files will be there.  Clearly, you can be more sophisticated in your scripting and scheduling based on your skills and requirements. For shell script (Linux) and other automation strategies see the Examples knowledge path.

    • “Store”: performs explicit add and update operations on the data records that appear in the file.  This is analogous to the CLI Snapshot mode MANUAL.
    • “Full Replace”: performs add and update operations on the data records in the file AND it implicitly will execute a remove operation on records that are not in the file but are members of the integration.  This is like the SNPSHT mode with the CLI.
    • “Delete”: executes explicit remove operations on the records included in the file.  This is just like CLI Snapshot REMOVE mode.
    • -k = allow connection without verifying the SSL certificate.
    • -H = determines this a required HTTP header line for Learn
      • "Content-Type:text/plain" tells Learn that we are sending a plain text file.
      • If you were using XML, it would be "Content-Type:text/xml"
    • -u = username and password for authentication.
      • 2d2f8833-90b7-444e-8bea-514111d5be70:password is from our setup example (see Figure 1)
    • --url = the page to which you will post the file.
      • End-point URLs are listed in Learn on the “HTTP Information” page for all integrations.  For flat files this is specific to the Data Type (users, course, enrollments, etc.) and mode (store, complete refresh or delete) we want the data
    • --data-binary = HTTP post of binary data
    • @courses_fall2012.ext = the path to the file that is being sent to Learn.  In this example the command is being run while in the directory where the file is located.


REM.-- Simple script to run three integration files to Learn

REM. --USERS posted in STORE mode

curl -k -H "Content-Type:text/plain" -U 2d2f8833-90b7-444e-8bea-514111d5be70:password --url webapps/bb-data-integration-flatfile-BBLEARN/endpoint/person/store --data-bin @users.txt

REM. --COURSES posted in STORE mode

curl -k -H "Content-Type:text/plain" -U 2d2f8833-90b7-444e-8bea-514111d5be70:password --url webapps/bb-data-integration-flatfile-BBLEARN/endpoint/course/store --data-bin @courses_fall2012.txt

REM. --ENROLLMENTS posted in STORE mode

curl -k -H "Content-Type:text/plain" -U 2d2f8833-90b7-444e-8bea-514111d5be70:password --url webapps/bb-data-integration-flatfile-BBLEARN/endpoint/membership/store --data-bin @enrollments_fall2012.txt


The cURL commands will only return standard HTTP status codes from Learn as an indication of the success or failure of the file post. Code 200 is the only one that indicates a fully successful file post. You may reference for a list of HTTP status codes.

The successful post of the file does not indicate the successful processing of the data.  For that you will have to examine the log entries on the server – in the GUI.

Strategies for monitoring integration processing progress are outlined in the Examples knowledge path.