Software Testing Lesson 2: Overview of Test Automation

Overview of Test Automation

How do you know that your software still works after you make changes?

Did you remember to test EVERYTHING?

Are you sure?

There is a way to be more confident that your app still works every time you deploy. Testing!

Automated tests prove that your code still works after you make changes. It’s an extra safe guard that helps ensure that your app stays reliable. This course will walk you through the creation of an agile software development process where test automation is both a first-class citizen and the hero of the story.

Let’s look at an example:

PayPal processes a huge number of cash transactions every day, all around the globe. If PayPal breaks, it hurts a lot of customers in the worst way — financially.

Imagine that you’re working at PayPal, and you have to refactor some core piece of functionality in order to enable a new currency — say, BitCoin. Imagine that each locale supports exactly one currency, but now, with digital currencies popping up, every locale should be able to support any number of currencies.

That would be a pretty scary change. A tiny error could cause transactions to post in the wrong currency, and that could be a HUGE PROBLEM for customers. (In the real world, many locales already do business in multiple currencies, so tying a single currency to a location is probably a bad idea to begin with).

With automated tests, you don’t have to fear small changes, or even big refactors. Every time you make an update, your changes are passed through the test suite. If the tests fail, the new deploy doesn’t proceed.

That’s pretty great, but that’s not even the best thing about tests.

Benefits of Testing

Improve application design

  • User-focused design – write tests first!

Product Validation & QA

  • Prove that the implementation is feature complete
  • Prove that the code does what it’s supposed to do
  • Prove that bug fixes actually fix the bug

Streamlined process

  • Automated testing & continuous integration. Dramatically speeds up deployment because you’ll have a lot more confidence that your software won’t break.

As you can see, tests provide a number of benefits that go beyond the obvious quality assurance benefits. Probably the most underrated is design perspective. Writing tests first forces you to think about the problem from the perspective of the interface user before you think about the problem from the perspective of the developer. This has immediate design benefits, because you’re more likely to come up with a simpler, more convenient interface when you focus on the user first. Remember, the only thing that matters in development is that users love the product.

Benefits of Sauce Labs

We’re working with Sauce Labs to be sure that we’re teaching current suggested practices (nobody knows more about software testing than they do!), and in order to supply you with access to the best tooling available. I’ve tried the competition, and Sauce Labs has all the right ingredients for good testing:

  • Faster development. Run your tests in parallel on an impressive array of available platforms and browsers. You’re tests will run in a fraction of the time, so your team can spend more time building a great product.
  • Less infrastructure. Sauce Labs manages all your test infrastructure in the cloud, so you don’t have to buy and configure all the machines, or juggle hundreds of devices and configurations, which means you can do more with less money. Leave testing infrastructure to the experts so you can focus on your products.
  • Reliable tests. Sauce Labs fires up clean browsers and virtual machines for each test run. I’ve tried competing services where it was obvious that the environment had been recycled and polluted by other customers, and that can impact the reliability of your tests.
  • See and share test runs. All the test sessions are recorded in video, so when things go wrong, you can take a look at the session, see what’s failing, and get error messages.
  • Manual control. There’s no substitute for hands-on testing. If you suspect there might be a problem on a specific platform, you don’t have to go searching for some obscure physical device — just jump on Sauce Labs and fire up a manual session. Easy and convenient!
  • Security. Sauce Connect creates a secure tunnel between your app and Sauce Labs, so your IP is safe.
  • Automated Process. Sauce Labs works with several continuous integration servers, so your test suites can run as part of your automated build and deploy process.
  • Expertise. Sauce Labs has the talent. They know the platforms in and out, and they’re important contributors to the open source testing community, including Selenium, Selenium Builder, and Appium.
  • Free for Open Source. Sauce Labs offers free accounts for open source projects, so if you’re giving back to the community, Sauce Labs won’t charge you for it. Check out their Open Sauce program.

Sauce Labs vs In-House Testing

That’s all nice, but why shouldn’t you just run your own testing server? I’ve been at a company or two that tried to do that. Here’s how that worked out:

  • $10k per machine for super-fast testing servers

  • Tens of thousands of dollars paying developers to maintain said servers, and to maintain a whole bunch of old computers and mobile devices so that we could test platforms that were hard to simulate on the super-fast testing server. This happened frequently. For a long time we basically had a developer dedicated to nothing but tweaking the test configurations.

  • Incalculable opportunity cost while developers got pulled off of development to mess with the test process. That time could have been spent building something that shipped to users.

In other words, Sauce Labs does a better job with a smaller bugdet, and your team is available to get more work done.