[Appium][maestro] compare running speed in one scenario for Android

I was curious about how Maestro is faster than Appium UIA2. I picked up a scenario from https://github.com/appium/ruby_lib_core/blob/3de46abaa933e6fa813b78192650e1f9ead5a238/test/functional/android/patch_test.rb#L38-L47, which had multiple taps and sending keys. Maestro doesn’t support Unicode, so I switched the keyword to ASCII.

appId: io.appium.android.apis
---
- launchApp
- tapOn: "App"
- tapOn: "Activity"
- tapOn: "Custom Title"
- tapOn:
id: "io.appium.android.apis:id/left_text_edit"
- inputText: "Pokemon"
- assertVisible:
id: "io.appium.android.apis:id/left_text_edit"
text: "Left is bestPokemon"

It took a total of 30 seconds, as shown below. This ran on an emulator on my local laptop.

 
Running on emulator-5554
 ║  > Flow: maestro
 ║
 ║    ✅   Launch app "io.appium.android.apis"
 ║    ✅   Tap on "App"
 ║    ✅   Tap on "Activity"
 ║    ✅   Tap on "Custom Title"
 ║    ✅   Tap on id: io.appium.android.apis:id/left_text_edit
 ║    ✅   Input text Pokemon
 ║    ✅   Assert that "Left is bestPokemon", id: io.appium.android.apis:id/left_text_edit is visible
 ║
maestro test maestro.yaml  5.09s user 1.05s system 20% cpu 30.291 total


The maestro version was 1.40.3.

The Appium’s result was below. I ran the scenario after running appium driver run uiautomator2 reset command to uninstall the uia2 driver-related apks from the device to start Appium preparation from scratch. I have commented out unrelated scenarios to run only test_type. The target device was the same emulator. The Appim server was already running on the host. Appium version was v2.19.0 and uia2 driver was v4.2.3.

$ time rake test:func:android TEST=test/functional/android/patch_test.rb
Started with run options --seed 65017
1/1: [=========================================================================================] 100% Time: 00:00:12, Time: 00:00:12
Finished in 12.96819s
1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
Coverage report generated for Functional Tests to /Users/kazu/github/ruby_lib_core/coverage.
Line Coverage: 64.28% (871 / 1355)
rake test:func:android TEST=test/functional/android/patch_test.rb 0.31s user 0.17s system 3% cpu 13.654 total

My local laptop was a MacBook Pro, Mac15,3, Apple M3 chip with 8GB of memory.

They might have configuration tuning gaps, so the comparison may not be apples-to-apples. However, it appears that the Appium scenario (around 14 seconds) has cases that are faster than Maestro (around 30 seconds) as well.

[update] This post is for iOS.

[update] With Maestro’s sample Wikipedia scenario. I have removed some steps to simplify the flow. The Appium capabilities have [Appium] Reduce Animation Flakiness in Appium UIA2 driver here.

# maestro
$ time maestro test android-advanced-flow.yaml
Running on emulator-5554
║ > Flow: android-advanced-flow
║ ✅ Run subflows/onboarding-android.yaml
║ ✅ Tap on id: org.wikipedia:id/search_container
║ ✅ Run scripts/getSearchQuery.js
║ ✅ Input text qwerty
║ ✅ Assert that "qwerty" is visible
maestro test android-advanced-flow.yaml 5.53s user 1.33s system 23% cpu 29.683 total
# appium
$ time ruby appium_ruby.rb
ruby appium_ruby.rb 1.69s user 0.78s system 28% cpu 8.616 total
# Appium v3.2.0
# uiautomator2@6.8.0
require 'appium_lib_core'
require 'timeout'
require 'socket'
appium_log = '/tmp/appium.log'
appium_err = '/tmp/appium.err'
appium_pid = spawn('appium', out: appium_log, err: appium_err, pgroup: true)
begin
Timeout.timeout(30) do
loop do
begin
socket = TCPSocket.new('127.0.0.1', 4723)
socket.close
break
rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, IOError
sleep 0.2
end
end
end
rescue Timeout::Error
warn "timed out waiting for Appium to listen on 127.0.0.1:4723"
end
opts = {
capabilities: {
'platformName': 'android',
'platformVersion': '15',
'appium:deviceName': 'android',
'appium:appPackage': 'org.wikipedia',
'appium:automationName': 'uiautomator2',
'appium:udid': 'emulator-5554',
'appium:forceAppLaunch': true,
'appium:appWaitActivity': '*',
'appium:autoGrantPermissions': true,
'appium:disableWindowAnimation': true,
'appium:settings[actionAcknowledgmentTimeout]': 0,
'appium:settings[waitForIdleTimeout]': 0,
'appium:settings[waitForSelectorTimeout]': 0
}
}
begin
core = Appium::Core.for(opts)
driver = core.start_driver
driver.wait_until {|d| d.find_element(:id, 'org.wikipedia:id/fragment_onboarding_forward_button')}.click
driver.wait_until {|d| d.find_element(:id, 'org.wikipedia:id/fragment_onboarding_forward_button')}.click
driver.wait_until {|d| d.find_element(:id, 'org.wikipedia:id/fragment_onboarding_forward_button')}.click
driver.wait_until {|d| d.find_element(:id, 'org.wikipedia:id/fragment_onboarding_done_button')}.click
driver.wait_until {|d| d.find_element(:id, 'org.wikipedia:id/search_container')}.click
el = driver.wait { |d| d.switch_to.active_element }
el.send_keys 'qwerty'
if driver.switch_to.active_element.text != 'qwerty'
raise 'no element'
end
ensure
begin
driver.quit if defined?(driver) && driver
rescue StandardError => e
warn "failed to quit driver: #{e}"
end
if appium_pid
begin
Process.kill('TERM', appium_pid)
begin
Timeout.timeout(5) { Process.wait(appium_pid) }
rescue StandardError
Process.kill('-9', appium_pid) rescue nil
end
rescue Errno::ESRCH, Errno::ECHILD
end
end
end

1 Comment

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.