You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(contributing): add info about skip and fail (#1944)
When contributing some test cases, I was a bit confused about if I should use `skip` and `fail`, the difference between the two, and how they interact with CI/CD tests passing or failing: #1939 (comment).
This is my attempt at explaining the difference, but feel free to edit to make it more clear and concise (and correct any inaccuracies).
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+20Lines changed: 20 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -198,6 +198,26 @@ HEADLESS=false SLOW_MO=500 npm run wtest
198
198
BROWSER=chromium node --inspect-brk test/test.js
199
199
```
200
200
201
+
- When should a test be marked with `skip` or `fail`?
202
+
203
+
-**`skip(condition)`**: This test *should ***never*** work* for `condition`
204
+
where `condition` is usually a certain browser like `FFOX` (for Firefox),
205
+
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
206
+
207
+
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
208
+
with `skip(FFOX)` since an alt-click in Firefox will not produce a download
209
+
even if a person was driving the browser.
210
+
211
+
212
+
-**`fail(condition)`**: This test *should ***eventually*** work* for `condition`
213
+
where `condition` is usually a certain browser like `FFOX` (for Firefox),
214
+
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
215
+
216
+
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
217
+
with `fail(CHROMIUM || WEBKIT)` since Playwright performing these actions
218
+
currently diverges from what a user would experience driving a Chromium or
219
+
WebKit.
220
+
201
221
## Public API Coverage
202
222
203
223
Every public API method or event should be called at least once in tests. To ensure this, there's a `coverage` command which tracks calls to public API and reports back if some methods/events were not called.
0 commit comments