Striking a Balance with UI Tests

UI-tests

Everyone wants well-tested apps, and I know I’ve been drawn to User Interface testing since Xcode added UI Recording. Not only is it really fun to watch the app move so fast, but testing also offers a lot of potential benefits. While unit tests help validate the code we write, UI tests automate user-level functionality.

It may seem easy to record UI tests, but there are challenges associated with getting them to run consistently. It is difficult to create and maintain a suite of UI tests that grow with your app. I’ve tried two different approaches to adding UI tests and, in doing so, I struck a balance that doesn’t bog down a project too much.

Light Side – Black Box

In the beginning, I was in love with the concept of testing. I was working on a project that was going along well, nearly complete, and I wanted to automate the test steps I had been doing on a regular basis. The tests would guard against regressions during the final phase of the project. However, I discovered several challenges when trying to add tests late in the project.

One of the challenges involved the app architecture not having the dependencies set up in a way that would make testing easy. The test suite could run against production, staging and development servers, and it didn’t rely on any fake or mocked behavior. This black box approach to UI tests forced me to remain at a user’s understanding of how the app works. I recorded functionality and edited the resulting code to abstract away certain details, which can be quite difficult. Here are a few helpful tips:

  • Create accounts using a formatted string that has a timestamp.
  • Create a new project, then assert the projects list has a project with the new name.
  • Share some string resources between the app and UI test target.
    • Share Localizable.strings across UI test target and target app. This allows you to use concepts in UI tests rather than hard-coded strings. As long as the keys do not change, base translation changes will only have to be maintained in one place.
class UIStrings {

    static func getString(key: String) -> String {
        return NSLocalizedString(key, bundle: Bundle(for: UIStrings.self), comment: "")
    }

    static var loginButton: String {
        return getString(key: "login-button")
    }

    static var settingsNavBar: String {
        return getString(key: "settings")
    }

    static var signOutButton: String {
        return getString(key: "sign-out")
    }

}

The tests proved to be very valuable. We had control over the full stack and would use the UI tests to run the app against development, staging and production as we rolled out server changes. The API had a test suite as well, but we increased confidence in our changes by ensuring the app still worked via UI tests. They were harder and harder to maintain as time went on, and I settled on creating one big test that served as an integration test for the MVP. I rerecorded and refactored parts as they needed to change.

Heavy Side – Developer Style

I tried a different approach on another project where the motivation for writing tests was much more theory and statistic driven. I tried to keep to the spirit of TDD and still write the tests while finishing up a feature. These tests were valuable in different ways. There was a lot more craft put into the app architecture and the test architecture. The tests were more rigorous, maximizing code coverage in the ViewControllers and squeezing out anticipated state variations. This was a very motivating goal, but it misaligned with UI tests because it required a deeper understanding of the code than a user would have. Some techniques to open the black box were:

  • Set up schemes to assist in recording the tests. UI tests have some ability to manipulate the Process of the target app, but it only takes effect while running tests, not recording them.
  • Place mocks in the target app, injecting them as necessary based on Process data.
    • ie: Set up some test credentials in a mock, then use the test credentials as inputs to the UI tests to trigger certain login flows.
  • Write data files from your test target.
    • This works well if you use some sort of interceptor pattern for your web requests. You can inject mock network behavior that reads data from the filesystem to fulfill requests.
    • Note this solution only works in the simulator due to Entitlements.

This particular approach was really satisfying as a developer, but ended up slowing the development speed down a lot. In prying open the black box, the test target was getting more and more coupled with the application.

Related: Patience with Test-Driven Development (TDD)

A New Hope

Both approaches were done with good intentions in supporting the overall goals of the project. However, I learned that the attitude and perspective toward testing could use some improvement. Both methods of testing were great learning adventures in how to UI test, but they added unwanted stress at regular intervals. I suggest trying to strike a balance between the two approaches, but be sure to shift the focus to showcasing your work at regular intervals. Based on these results, we write UI tests that run through our sprint demo plans; it has added a level of focus to finishing up a sprint and has minimized late nights leading up to demo day.

There is a time trap in getting UI tests to do things you could have tested easily by unit testing ViewControllers, so be sure to stick to black box testing as often as possible. Only add programmer-level knowledge, when necessary, to achieve high-value tests. My advice: it is better practice to carry forward a suite of maintainable tests than to wrangle every possible assertion into your test target.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *