Performance testing in a scaled down test environment

The blog considers performance testing in a traditional full size performance environment compared to a scaled down test environment.

Introduction

• There is a perception that scaled down test environments do not deliver the same testing value as a full sized environment
• This blog provides a brief overview of performance testing approaches using full and scaled down test environments
• The author is currently managing the relationships with out strategic accounts having previously delivered performance testing engagements for our clients

Full Sized vs. Scaled Down

Large sums of money are spent on creating test environments to accurately test the capacity and performance of a system. Costs associated with performance testing is mostly driven by the size of the environment or the number of test cycles. 

In general, there are two different testing approaches, both with pros and cons.

How can a scaled down test environments give more representative results?

• Modeling can bridge the gap by accurately forecast capacity and performance even with results from a scaled down test environment.
• Modeling is a systematic approach to breaking down and understanding the relationship between transactions and resource usage
  • A demand model will take results from performance tests and model the impact of different levels of demand on system resources
• By analyzing performance data from your tests you can create advanced yet comprehensible models even with data from a scaled down environment
  
  • Modeling can exposure performance issues which can be masked by noisy test results

Summary

• Testing in a full sized environment is less cost effective however can produce initially more accurate results
• Testing in a scaled down environment can be more cost effective if accompanied by performance modeling
• Often production sized environments are not practical or possible, modeling can be used to account for any discrepancies between production and your test environments.