Sizing is a three-step process consisting of two modeling exercises and performance testing. The modeling exercises are used to gather statistical evidence about how users interact with the system. The data generated from these exercises is subsequently used in a series of performance tests. Performance tests help quantify what the system will look like under hypothesized workloads and scenarios.
The process begins with understanding how Blackboard clients have used the product in the past. This form of sampling is called behavior modeling. The objective of this form of sampling is to gather meaningful data representing the following:
- Who is using the system?
- What is being done?
- Where are they performing their operations?
- When are they performing their operations?
- How long are users spending performing their operations?
Predictive modeling is used for new performance testing new features. Little information can be collected about a feature that has not been built. Because of this, Blackboard hypothesizes user interactions with these new features.
The data collected from both modeling exercises is then used for performance testing and benchmarking. Performance benchmarking is conducted by Blackboard with a selected partner of choice (such as Sun Microsystems, Dell, Microsoft, or Oracle) at the Blackboard Performance Laboratory using a combination of purchased and donated equipment from a partner. Performance testing and benchmark activities focus primarily on the performance (response times exhibited by users) and scalability of the system (utilization of system resources such as CPU, memory, and I/O). HP/Mercury LoadRunner is the simulation tool of choice for generating workload.