Skip to content

ci: create xcode simulators when running tests #1392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 22, 2025
Merged

Conversation

fewerner
Copy link
Contributor

@fewerner fewerner commented Aug 21, 2025

What's new in this PR

With this PR ios tests with device specific xcode build destinations create a dedicated device for test first and remove it after finishing the test. This should avoid any CI errors related to non existent simulator devices when using xcodebuild.


PR Submission Checklist for internal contributors
  • The PR Title
    • conforms to the style of semantic commits messages¹ supported in Wire's Github Workflow²
    • contains a reference JIRA issue number like SQPIT-764
    • answers the question: If merged, this PR will: ... ³
  1. https://sparkbox.com/foundry/semantic_commit_messages
  2. https://github.com/wireapp/.github#usage
  3. E.g. feat(conversation-list): Sort conversations by most emojis in the title #SQPIT-764.

Copy link

github-actions bot commented Aug 21, 2025

🐰 Bencher Report

Branchfelix/ci/test-simulators
Testbedubuntu-latest
Click to view all benchmark results
BenchmarkLatencymicroseconds (µs)
Commit add f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold
17,880.00 µs
Commit add f(group size)/cs1/mem/2📈 view plot
🚷 view threshold
737.94 µs
Commit add f(group size)/cs1/mem/202📈 view plot
🚷 view threshold
4,317.50 µs
Commit add f(group size)/cs1/mem/402📈 view plot
🚷 view threshold
7,826.00 µs
Commit add f(group size)/cs1/mem/602📈 view plot
🚷 view threshold
11,912.00 µs
Commit add f(group size)/cs1/mem/802📈 view plot
🚷 view threshold
14,771.00 µs
Commit add f(number clients)/cs1/mem/1002📈 view plot
🚷 view threshold
983,470.00 µs
Commit add f(number clients)/cs1/mem/2📈 view plot
🚷 view threshold
727.36 µs
Commit add f(number clients)/cs1/mem/202📈 view plot
🚷 view threshold
79,383.00 µs
Commit add f(number clients)/cs1/mem/402📈 view plot
🚷 view threshold
216,290.00 µs
Commit add f(number clients)/cs1/mem/602📈 view plot
🚷 view threshold
423,830.00 µs
Commit add f(number clients)/cs1/mem/802📈 view plot
🚷 view threshold
674,380.00 µs
Commit pending proposals f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold
116,820.00 µs
Commit pending proposals f(group size)/cs1/mem/2📈 view plot
🚷 view threshold
22,926.00 µs
Commit pending proposals f(group size)/cs1/mem/202📈 view plot
🚷 view threshold
41,728.00 µs
Commit pending proposals f(group size)/cs1/mem/402📈 view plot
🚷 view threshold
57,707.00 µs
Commit pending proposals f(group size)/cs1/mem/602📈 view plot
🚷 view threshold
77,399.00 µs
Commit pending proposals f(group size)/cs1/mem/802📈 view plot
🚷 view threshold
98,922.00 µs
Commit pending proposals f(pending size)/cs1/mem/1📈 view plot
🚷 view threshold
17,640.00 µs
Commit pending proposals f(pending size)/cs1/mem/101📈 view plot
🚷 view threshold
115,550.00 µs
Commit pending proposals f(pending size)/cs1/mem/21📈 view plot
🚷 view threshold
34,848.00 µs
Commit pending proposals f(pending size)/cs1/mem/41📈 view plot
🚷 view threshold
56,200.00 µs
Commit pending proposals f(pending size)/cs1/mem/61📈 view plot
🚷 view threshold
75,233.00 µs
Commit pending proposals f(pending size)/cs1/mem/81📈 view plot
🚷 view threshold
95,454.00 µs
Commit remove f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold
11,305.00 µs
Commit remove f(group size)/cs1/mem/2📈 view plot
🚷 view threshold
584.63 µs
Commit remove f(group size)/cs1/mem/202📈 view plot
🚷 view threshold
2,282.20 µs
Commit remove f(group size)/cs1/mem/402📈 view plot
🚷 view threshold
4,170.90 µs
Commit remove f(group size)/cs1/mem/602📈 view plot
🚷 view threshold
6,333.70 µs
Commit remove f(group size)/cs1/mem/802📈 view plot
🚷 view threshold
8,437.60 µs
Commit remove f(number clients)/cs1/mem/1002📈 view plot
🚷 view threshold
14,356.00 µs
Commit remove f(number clients)/cs1/mem/2📈 view plot
🚷 view threshold
135,710.00 µs
Commit remove f(number clients)/cs1/mem/202📈 view plot
🚷 view threshold
111,400.00 µs
Commit remove f(number clients)/cs1/mem/402📈 view plot
🚷 view threshold
87,341.00 µs
Commit remove f(number clients)/cs1/mem/602📈 view plot
🚷 view threshold
62,808.00 µs
Commit remove f(number clients)/cs1/mem/802📈 view plot
🚷 view threshold
38,506.00 µs
Commit update f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold
135,630.00 µs
Commit update f(group size)/cs1/mem/2📈 view plot
🚷 view threshold
753.99 µs
Commit update f(group size)/cs1/mem/202📈 view plot
🚷 view threshold
28,070.00 µs
Commit update f(group size)/cs1/mem/402📈 view plot
🚷 view threshold
55,082.00 µs
Commit update f(group size)/cs1/mem/602📈 view plot
🚷 view threshold
83,117.00 µs
Commit update f(group size)/cs1/mem/802📈 view plot
🚷 view threshold
109,170.00 µs
🐰 View full continuous benchmarking report in Bencher

@fewerner fewerner force-pushed the felix/ci/test-simulators branch 2 times, most recently from eecdf27 to eff35c0 Compare August 21, 2025 09:10
@fewerner fewerner marked this pull request as ready for review August 21, 2025 09:26
@fewerner fewerner requested a review from a team as a code owner August 21, 2025 09:26
@fewerner fewerner force-pushed the felix/ci/test-simulators branch from eff35c0 to 2733eb6 Compare August 21, 2025 09:32
@fewerner fewerner force-pushed the felix/ci/test-simulators branch from 2733eb6 to 99da20b Compare August 21, 2025 09:38
Copy link
Member

@istankovic istankovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start, but we need to avoid creating multiple devices for the interop test and move device creation/removal to standalone scripts.

@fewerner fewerner force-pushed the felix/ci/test-simulators branch 6 times, most recently from 4a87e94 to d89522c Compare August 21, 2025 15:21
We used to create a device for building the App used in the test and
another device to actually run the test. We can do both with the same
simulator device.
Note that a device deletion will shutdown the device if it is booted.
Thus, we remove the manual shutdown of the device.
@fewerner fewerner force-pushed the felix/ci/test-simulators branch from d89522c to 5fb5bb3 Compare August 22, 2025 07:36
Copy link
Member

@istankovic istankovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@fewerner fewerner merged commit 5fb5bb3 into main Aug 22, 2025
42 checks passed
@fewerner fewerner deleted the felix/ci/test-simulators branch August 22, 2025 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants