[Appium] Prevent building WDA with useXctestrunFile or usePrebuiltWDA

Building WDA every time takes much time to initialise Appium sessions for iOS. I assume many users would like to reduce the initialisation time as possible.

xcodebuild can run xcuitest process without building them. The command is xcodebuild test-without-build. If you already have a build module, xcodebuild can use it without additional building process.

Appium also has the ability. By default, Appium calls test-without-build after build-for-testing for WDA [link]. They can be build and test with useSimpleBuildTest capability.

Appium can run only test-without-build if the capability has useXctestrunFile or usePrebuiltWDA. It should raise an error if there is no built module. It can install WDA binary and run it as xctestrun process.

Benefit

The benefit is we can make the initialising phase faster. I would share an example as below. The test host machine was high spec, so build time was much faster. But you can see almost twice the difference.

evaluation

• library: https://github.com/appium/ruby_lib_core
• Appium version: 1.11.1
• code

@core = ::Appium::Core.for desired_capabilities: {....}

time = Time.now
@driver = @@core.start_driver
puts(Time.now - time) # puts the time diff

• machine spec:
  • Model Name: MacBook Pro
  • Model Identifier: MacBookPro15,1
  • Processor Name: Intel Core i9
  • Processor Speed: 2.9 GHz
  • Memory: 32 GB
• Test device
  • iPhone SE (real device)
    

Result

Tried five times. Below result is approximately range.

• full build (no useXctestrunFile nor usePrebuiltWDA)
  • 34 ~ 38 sec
• skipping build (with useXctestrunFile or usePrebuiltWDA)
  • 15 ~ 18 sec
• Installing WDA to real device occupied the time
  The installation time does not depends on host machine spec than building time. The time should be slower if the host machine is cheaper.
  

Test scenario

1. Uninstall WDA from the device
2. Start Appium server process
3. Run the above test code
4. See the result of puts(Time.now - time)

Capability

Below is the basic capability.

platformName: :ios,
automationName: 'XCUITest',
bundleId: 'com.kazucocoa.apple-samplecode.UICatalog', # https://github.com/appium/ios-uicatalog
udid: 'auto',
platformVersion: '12.1',
deviceName: 'iPhone SE',
useNewWDA: true, # Uninstall installed WDA app everytime
wdaLocalPort: 9000, # to make sure if process argument worked
xcodeSigningId: 'iPhone Developer',
xcodeOrgId: 'XXXXXXXXXX',
updatedWDABundleId: 'com.kazucocoa.WebDriverAgentRunner'

How to skip the build

You can choose either below way.

1. useXctestrunFile
   • bootstrapPath: '/Users/kazu/tmp/Build/Products', useXctestrunFile: true
2. usePrebuiltWDA
   • derivedDataPath: '/Users/kazu/tmp/', usePrebuiltWDA: true
     

The above are addressed in https://github.com/appium/appium-xcuitest-driver

About useXctestrunFil

xcodebuild creates a directory as below.

Build / Intermediates.noindex
      / Products / Debug-iphoneos
               / WebDriverAgentRunner_iphoneos12.1-arm64.xctestrun
Index
info.plist
Logs
ModuleCache.noindex

derivedDataPath capability can specify the output directory. Without them, xcodebuild generates like /Users/kazu/Library/Developer/Xcode/DerivedData/WebDriverAgent-afhjvleufqpvajasmhorbgksryeg/.

WDA for real device is generated in Build/Products. The naming rule for xctestrun file is WebDriverAgentRunner_iphoneos<platformVersion>-arm64.xctestrun.

The file and binary signed properly are necessary to install and run WDA.If you have them, you do not need to build WDA every time. You can make them portable as well. If you copy them to another machine and specify them with bootstrapPath and useXctestrunFile, you can run them without building on the machine against iPhone OS 12.1. (If WebDriverAgentRunner_iphoneos<platformVersion>-arm64.xctestrun is WebDriverAgentRunner_iphoneos12.1-arm64.xctestrun case. )

This feature should help your test environment.

Process arguments

I put wdaLocalPort as an example of the process arguments. The capability handles the port number to communicate with WDA. Appium provides the argument as a process argument via xcodebuild.

It means you can set process arguments for WDA by bootstrapPath and useXctestrunFile.You can run tests as same as full build version.

Error case

Below error is thrown if the path by bootstrapPath does not have proper xctestrun file.

[debug] [XCUITest] 'pgrep -nif iproxy 9000' didn't detect any matching processes. Return code: 1
[debug] [XCUITest] Starting iproxy to forward traffic from local port 9000 to device port 8100 over USB for the device 242bfcc998fd156df0bd0ad1dde8ab8e0a032114
[XCUITest] Error: if you are using useXctestrunFile capability then you need to have /Users/kazu/tmp/Build/Products/WebDriverAgentRunner_iphoneos12.1-arm64.xctestrun file
[XCUITest]     at Object.wrappedLogger.errorAndThrow (/Users/kazu/GitHub/appium/node_modules/appium-support/lib/logging.js:78:13)
[XCUITest]     at errorAndThrow (/Users/kazu/GitHub/appium/node_modules/appium-xcuitest-driver/lib/wda/utils.js:219:11)
[debug] [BaseDriver] Event 'wdaStartFailed' logged at 1552569001665 (22:10:01 GMT+0900 (Japan Standard Time))
[debug] [XCUITest] Unable to launch WebDriverAgent because of xcodebuild failure: "if you are using useXctestrunFile capability then you need to have /Users/kazu/tmp/Build/Products/WebDriverAgentRunner_iphoneos12.1-arm64.xctestrun file". Make sure you follow the tutorial at https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md. Try to remove the WebDriverAgentRunner application from the device if it is installed and reboot the device.
[XCUITest] Quitting and uninstalling WebDriverAgent, then retrying
[XCUITest] Shutting down sub-processes

You can configure your environment dynamically

Effective case

If you can build below environment, this capability works very well.

• Do not need to change provisioning file every test run
• We can re-use test devices which can run applications built by the same Xcode (SDK) versions
  • e.g.
    • A xctestrun is built by Xcode 10.2 (iPhone SDK 12.2)
    • The module can work on iOS 12.2, 12.1, 12.0, 11.4, 11.3, 11.2, 10.4
    • Then, we can re-use one xctestrun preference for ALL devices
  • WebDriverAgentRunner_iphoneos<sdk version>-arm64.xctestrun in Build/Product
    • https://github.com/appium/appium-xcuitest-driver/pull/903

This can reduce xcodebuild error cases by signing, for example.

Example

Below is one example. At first, iOS capability has no useXctestrunFile capability since there is no .xctestrun file. But the file will be generated after the test running. Following test cases can use useXctestrunFile with bootstrapPath then.

https://github.com/appium/ruby_lib_core/blob/master/test/test_helper.rb#L127

Conclusion

We can reduce a number of building if we can restrict test environment. This tip is such example.