Skip to content

Commit fd17cfb

Browse files
authored
docs(contributing): add info about skip and fail (#1944)
When contributing some test cases, I was a bit confused about if I should use `skip` and `fail`, the difference between the two, and how they interact with CI/CD tests passing or failing: #1939 (comment). This is my attempt at explaining the difference, but feel free to edit to make it more clear and concise (and correct any inaccuracies).
1 parent 05f0797 commit fd17cfb

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

CONTRIBUTING.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,26 @@ HEADLESS=false SLOW_MO=500 npm run wtest
198198
BROWSER=chromium node --inspect-brk test/test.js
199199
```
200200

201+
- When should a test be marked with `skip` or `fail`?
202+
203+
- **`skip(condition)`**: This test *should ***never*** work* for `condition`
204+
where `condition` is usually a certain browser like `FFOX` (for Firefox),
205+
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
206+
207+
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
208+
with `skip(FFOX)` since an alt-click in Firefox will not produce a download
209+
even if a person was driving the browser.
210+
211+
212+
- **`fail(condition)`**: This test *should ***eventually*** work* for `condition`
213+
where `condition` is usually a certain browser like `FFOX` (for Firefox),
214+
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
215+
216+
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
217+
with `fail(CHROMIUM || WEBKIT)` since Playwright performing these actions
218+
currently diverges from what a user would experience driving a Chromium or
219+
WebKit.
220+
201221
## Public API Coverage
202222

203223
Every public API method or event should be called at least once in tests. To ensure this, there's a `coverage` command which tracks calls to public API and reports back if some methods/events were not called.

0 commit comments

Comments
 (0)