Sauce Labs works hard to make our service function like a bigger, better, drop-in replacement for your local grid. For a test, session creation and communication on Sauce are very similar to what happens locally. Under the hood, however, running a test on Sauce Labs is very different. Some of the potential differences are listed in the table below.
| Local Grid | Sauce Labs | Impact | |
|---|---|---|---|
| Point of Difference | |||
| Network Distance | Short - same physical machine or data center | Potentially hundreds of thousands of times longer (and thus slower) | Selenium commands and responses take longer to transmit and receive |
| Network Quality | Dedicated networking hardware | Public internet | Speed and routing depends on the provider, usage, and other users on the same network |
| Host Machine State | Reused between tests | New, unused copy for every test | No cached data and no inconsistent startup states |
| Host Machine Hardware | May vary between test hosts. Almost certainly different from Sauce Labs. | Identical for every test on a given platform. May differ substantially from local tests. | Sauce Labs platform instances perform identically. They may be more or less performant than local hardware. |
| Software Versions | Changed manually. May not be consistent across hosts. | Defaults change across every host at once. Specific versions can be requested. | Sauce Labs platforms are configured consistently at any point in time. |
| Browser Setup | Potentially highly customized. | As close to "Stock" as possible. | Sauce Labs browsers provide a test environment as close to a real user as possible |
| Concurrency | Limited by your grid capacity. Costly and time consuming to increase | Limited by your account administrator. Easy and cost-effective to increase. | Sauce Labs can provide much higher concurrency on demand. |
Fundamentally, we consider comparisons of test duration to be spurious. The differences in environments are complex, variable, and mostly unavoidable. By loading tested products over the internet, using a stock browser, we provide a test environment much more realistic than a local grid. Our service provides advantages in stability, flexibility, and concurrency, and our analytics and experience help customers get better quality outcomes.
There are some steps you can take to ensure your tests are running optimally:
- Check out this and this on our wiki to help ensure your tests are lean and optimized
- Use a test framework to run your tests concurrently, taking full advantage of your Sauce Labs account
- Check the Efficiency metrics in our Analytics to identify optimization opportunities