Skip to content

[Testing] Fix for flaky UITests in CI - 2 #31763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jfversluis merged 8 commits into from
Dec 17, 2025

added mac and windows snapshots

b9c0296
Select commit
Loading
Failed to load commit list.
Merged

[Testing] Fix for flaky UITests in CI - 2 #31763

added mac and windows snapshots
b9c0296
Select commit
Loading
Failed to load commit list.
Build Analysis / Build Analysis failed Dec 17, 2025 in 0s

.NET Result Analysis

Details

Tip

To unconditionally bypass the build analysis check (turn it green), you can use the escape mechanism feature. The completion time may vary, potentially taking several minutes, depending on the build analysis workload at the moment.

Build Failures

maui-pr-uitests / Android UITests / Controls (API 30) / Controls (API 30) Layout / Controls (API 30) Layout

[ 🚧 Report infrastructure issue] [ 📄 Report repository issue]
    ❌PowerShell exited with code '1'.

maui-pr-uitests / Android UITests / Controls (API 30) / Controls (API 30) Layout / Publish the android_ui_tests_controls_30 test results

[ 🚧 Report infrastructure issue] [ 📄 Report repository issue]
    ❌There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.

maui-pr-uitests / Android UITests CoreClr / Controls (API 30) / Controls (API 30) CarouselView / Controls (API 30) CarouselView

[ 🚧 Report infrastructure issue] [ 📄 Report repository issue]
    ❌PowerShell exited with code '1'.

11 other jobs are failing and were not displayed due to space limitations.
Builds with failures: [maui-pr-uitests]

Test Failures (8 tests failed)

🔹 [All failing tests from maui-pr-uitests]

    ⚠️ 3 failing tests not displayed for this pipeline due to space limitations

    ❌ Microsoft.Maui.TestCases.Tests.CarouselViewFeatureTests(iOS).VerifyCarouselViewWithKeepItemInViewAndPreviousItem [Details] [4.53% failure rate]
    [ 🚧 Report test infrastructure issue] [ 📄 Report test repository issue]

    Failing Configuration
    Exception Message
    System.TimeoutException : Timed out waiting for element...
    CallStack
       at UITest.Appium.HelperExtensions.Wait(Func`1 query, Func`2 satisfactory, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 2575
   at UITest.Appium.HelperExtensions.WaitForElement(IApp app, String marked, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency, Nullable`1 postTimeout) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 745
   at Microsoft.Maui.TestCases.Tests.CarouselViewFeatureTests.VerifyCarouselViewWithKeepItemInViewAndPreviousItem() in /_/src/Controls/tests/TestCases.Shared.Tests/Tests/FeatureMatrix/CarouselViewFeatureTests.cs:line 105
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    ❌ Microsoft.Maui.TestCases.Tests.CarouselViewFeatureTests(iOS).VerifyCarouselViewWithScrollTo [Details] [4.54% failure rate]
    [ 🚧 Report test infrastructure issue] [ 📄 Report test repository issue]

    Failing Configurations (2)
    Exception Message
    System.TimeoutException : Timed out waiting for element...
    CallStack
       at UITest.Appium.HelperExtensions.Wait(Func`1 query, Func`2 satisfactory, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 2575
   at UITest.Appium.HelperExtensions.WaitForElement(IApp app, String marked, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency, Nullable`1 postTimeout) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 745
   at Microsoft.Maui.TestCases.Tests.CarouselViewFeatureTests.VerifyCarouselViewWithScrollTo() in /_/src/Controls/tests/TestCases.Shared.Tests/Tests/FeatureMatrix/CarouselViewFeatureTests.cs:line 838
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    ❌ Microsoft.Maui.TestCases.Tests.Issues.Issue12134(Android).CookiesCorrectlyLoadWithMultipleWebViews [Details] [0.17% failure rate]
    [ 🚧 Report test infrastructure issue] [ 📄 Report test repository issue]

    Failing Configuration
    Exception Message
    System.TimeoutException : Failied on: 0
    CallStack
       at UITest.Appium.HelperExtensions.Wait(Func`1 query, Func`2 satisfactory, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 2575
   at UITest.Appium.HelperExtensions.WaitForAtLeastOne(Func`1 query, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 2590
   at UITest.Appium.HelperExtensions.WaitForElement(IApp app, String marked, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency, Nullable`1 postTimeout) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 743
   at Microsoft.Maui.TestCases.Tests.Issues.Issue12134.CookiesCorrectlyLoadWithMultipleWebViews() in /_/src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue12134.cs:line 23
   at System.RuntimeMethodHandle.InvokeMethod(ObjectHandleOnStack target, Void** arguments, ObjectHandleOnStack sig, BOOL isConstructor, ObjectHandleOnStack result)
   at System.RuntimeMethodHandle.InvokeMethod(ObjectHandleOnStack target, Void** arguments, ObjectHandleOnStack sig, BOOL isConstructor, ObjectHandleOnStack result)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    ❌ Microsoft.Maui.TestCases.Tests.Issues.Issue18961(Android).ModalPageMarginCorrectAfterKeyboardOpens [Details] [52.21% failure rate]
    [ 🚧 Report test infrastructure issue] [ 📄 Report test repository issue]

    Failing Configuration
    Exception Message
    VisualTestUtils.VisualTestFailedException : 
Snapshot different than baseline: ModalPageMarginCorrectAfterKeyboardOpens.png (1.34% difference)
If the correct baseline has changed (this isn't a a bug), then update the baseline image.
See test attachment or download the build artifacts to get the new snapshot file.

    More info: https://aka.ms/visual-test-workflow

    CallStack
       at VisualTestUtils.VisualRegressionTester.Fail(String message) in /_/src/TestUtils/src/VisualTestUtils/VisualRegressionTester.cs:line 162
   at VisualTestUtils.VisualRegressionTester.VerifyMatchesSnapshot(String name, ImageSnapshot actualImage, String environmentName, ITestContext testContext) in /_/src/TestUtils/src/VisualTestUtils/VisualRegressionTester.cs:line 123
   at Microsoft.Maui.TestCases.Tests.UITest.<VerifyScreenshot>g__Verify|9_0(String name, <>c__DisplayClass9_0&) in /_/src/Controls/tests/TestCases.Shared.Tests/UITest.cs:line 367
   at Microsoft.Maui.TestCases.Tests.UITest.VerifyScreenshot(String name, Nullable`1 retryDelay, Int32 cropLeft, Int32 cropRight, Int32 cropTop, Int32 cropBottom, Double tolerance) in /_/src/Controls/tests/TestCases.Shared.Tests/UITest.cs:line 209
   at Microsoft.Maui.TestCases.Tests.Issues.Issue18961.ModalPageMarginCorrectAfterKeyboardOpens() in /_/src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue18961.cs:line 40
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](Func`1 invoke)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)
    ❌ Microsoft.Maui.TestCases.Tests.Issues.Issue24574(iOS).TapThenDoubleTap [Details] [17.83% failure rate]
    [ 🚧 Report test infrastructure issue] [ 📄 Report test repository issue]

    Failing Configuration
    Exception Message
    System.TimeoutException : Timed out waiting for element...
    CallStack
       at UITest.Appium.HelperExtensions.Wait(Func`1 query, Func`2 satisfactory, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 2575
   at UITest.Appium.HelperExtensions.WaitForElement(IApp app, String marked, String timeoutMessage, Nullable`1 timeout, Nullable`1 retryFrequency, Nullable`1 postTimeout) in /_/src/TestUtils/src/UITest.Appium/HelperExtensions.cs:line 745
   at Microsoft.Maui.TestCases.Tests.Issues.Issue24574.TapThenDoubleTap() in /_/src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue24574.cs:line 41
   at System.RuntimeMethodHandle.InvokeMethod(ObjectHandleOnStack target, Void** arguments, ObjectHandleOnStack sig, BOOL isConstructor, ObjectHandleOnStack result)
   at System.RuntimeMethodHandle.InvokeMethod(ObjectHandleOnStack target, Void** arguments, ObjectHandleOnStack sig, BOOL isConstructor, ObjectHandleOnStack result)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

Was this helpful? Yes No

View more details on Build Analysis