git bisect run xcodebuild

This blog post explains how you can easily find a commit that introduced a bug in an iOS application.

Motivation

The following technique is helpful if you work with many developers on a large repository with daily complex commits. Once you find the commit, you can analyze the file changes to determine the root cause of the bug.

Technique

Git, as source-control management (SCM) system, provides a helpful command called git bisect.

This command uses a binary search algorithm to find which commit in your project’s history introduced a bug. You use it by first telling it a "bad" commit that is known to contain the bug, and a "good" commit that is known to be before the bug was introduced. Then git bisect picks a commit between those two endpoints and asks you whether the selected commit is "good" or "bad". It continues narrowing down the range until it finds the exact commit that introduced the change.

You could perform a manual test to check if it is a "good" or "bad" commit for every iteration, but this is very tedious. Instead, can you write a test or script to detect the bug? Often this is possible even if you don't know what code caused the problem.

If you have a script that can tell if the current source code is good or bad, you can bisect by issuing the command $ git bisect run my_script arguments

You can use git bisect run xcodebuild test ... once you enhanced an existing XCTestCase to detect the bug through UI tests. Then Git can perform every iteration automatically and will spill out the information which was the first "bad" commit.

I recommend outsourcing the xcodebuild command in a Makefile for readability.

Example for a Makefile:

uitest:
    xcodebuild test \
      -project FlakyApp.xcodeproj \
      -scheme FlakyAppUITests \
      -sdk iphonesimulator \
      -destination 'platform=iOS Simulator,name=iPhone 13'

Now you can call git bisect run make uitest :)

Another advantage of outsourcing is that you can tweak the xcodebuild command and still stably trigger git bisect run. For example, let's run only the one test to detect the bug by specifying -only-testing option.

uitest:
    xcodebuild test \
      -project FlakyApp.xcodeproj \
      -scheme FlakyAppUITests \
      -sdk iphonesimulator \
      -destination 'platform=iOS Simulator,name=iPhone 13' \
      -only-testing "FlakyAppUITests/FlakyAppUITests/testExample"

This change will make the bisection execution faster and you still trigger it with git bisect run make uitest.

Example

The following simplified repository contains an iOS application, and a bug was introduced at some point.

https://github.com/MarcoEidinger/GitBisectXcodebuild

We only know the last "good" commit (2de359e) and one "bad" commit for which the app does not work as expected (6e02ff9).

In this example the bug is that the value of a SwiftUI Toggle is no longer true. We can update the UI Test in FlakyAppUITests.swift to check for that.

    func testExample() throws {
        let app = XCUIApplication()
        app.launch()

        let isValidSwitch = XCUIApplication().switches["isValidSwitch"]
        XCTAssertTrue(isValidSwitch.value as? String == "1")
    }

Now let's start bisecting the commits.

git bisect start 6e02ff9f1e319fe095a9dcfec092f76c3005355c 2de359e2ce524d9d4e8276595cc3c70ae9311996

Git tells us that roughly 2 steps are necessary to find the culprit.

Bisecting: 3 revisions left to test after this (roughly 2 steps)
[7e68c4102f3d68310285fe58acbcf9d21503ff51] maybe

Then we tell Git to run make uitest for every iteration.

git bisect run make uitest

Git starts processing ...

** TEST SUCCEEDED ** [9.950 sec]

Bisecting: 1 revision left to test after this (roughly 1 step)
[65c0d1ac4489241306692be4a7addbb38703aff3] maybe !

...

** TEST FAILED **

make: *** [uitest] Error 65
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[c8da8ee9fa48736b63bdbec0e1e1a179612396b3] maybe

...

65c0d1ac4489241306692be4a7addbb38703aff3 is the first bad commit
commit 65c0d1ac4489241306692be4a7addbb38703aff3
Author: Marco Eidinger <eidingermarco@gmail.com>
Date:   Wed Apr 20 08:45:34 2022 +0200

    maybe !

 FlakyApp/ContentView.swift | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Voila, we know the first bad commit is 65c0d1a !

Now that we know which commit is the culprit, we call the following commit to return to the original HEAD.

git bisect reset

Note: In this simplified example, we could have used git blame to identify the commit & root cause because there are not so many commits and the commits are very small. But imagine there are fifty commits with each for more than ten file changes of hundred lines!

Summary

git bisect is perfect when you don't know what code caused a problem and are working on a large and frequently updated project.
Leverage tests to automate git bisect.
- For iOS app use xcodebuild test.
- For a Swift Package you can use xcodebuild as well or swift test.

git bisect run xcodebuild

Comments

More from this blog

All new frameworks presented at WWDC26

All new frameworks presented at WWDC25

My WWDC25 Week: Events, Meetups, and Where to Find Me

You better increase URLCache size

All new frameworks presented at WWDC24

Motivation

Technique

Example

Summary

Command Palette

Comments

More from this blog

Motivation

Technique

Example

Summary