Cross-browser testing on CI

[YoranBrondsema]

One of the main goals of Mobiledoc-Kit is to provide cross-browser rich-text editing. With the latest releases, regressions have crept in on namely IE11. Currently, the builds on CI are performed on phantomjs only. However, I think that extending this to multiple browsers (or at least IE11) can prevent such regressions in the future.

I was therefore wondering whether it makes sense to move to a cross-browser testing platform. I have no experience with them but I had a quick look and it looks like Sauce Labs could be interesting. Most interestingly, it is free for Open Source projects (see https://saucelabs.com/opensauce/).

This is just an idea but I think this project could really benefit from this, especially since its goal is to abstract all the cross-browser contenteditable quirks.

[mixonic]

Yeah I'd love help with this. SL integration just takes some time banging against configs to get right.

since its goal is to abstract all the cross-browser contenteditable quirks

Indeed we support IE11. I would not say cross-brower contenteditable quirk-fixing is a "priority" though- In fact if Mobiledoc-Kit used canvas rendering instead of contenteditable that should just be an implementation quirk.

But big 👍 for SL on IE11, Edge, FF, Safari, Chrome. Regressions are bugs.

[YoranBrondsema]

Cool, happy you're for it.

I assume you'd want to keep the accounts under your control. Do you mind creating an account on https://saucelabs.com/beta/signup/OSS/None? I don't mind doing the configuration and stuff.

[bantic]

@YoranBrondsema I've created the account. Feel free to ping me here or on gitter for more info. I'd be very happy to see SL integration going for mobiledoc-kit.

[YoranBrondsema]

@bantic Cool thanks. I'm still working on getting all tests to pass on IE11. As far as I could see there are two remaining "sets" of tests failing:

• Node.childNodes is interpreted differently in IE11. If you have <p>\tY</p>, then on Chrome childNodes of the paragraph contains a single TextNode containing the tab and the 'Y' character. On IE11, childNodes contains two TextNodes: one for the tab and one for the 'Y'.
• There's 5-6 tests failing related to the atoms. Haven't looked into those in detail yet.

What do you prefer:

1. that we get a perfect test suite on all browsers before getting SL to work, or
2. selectively disable failing tests on certain browsers and get SL to work before making them all pass on all browsers. This way we can intercept eventual regressions earlier.

[bantic]

Thanks for looking into all of this, @YoranBrondsema. I think it makes the most sense to get SL up and running first, before all the tests are passing.
I've been looking at some of these tests on IE locally today, and I'll continue to smoke-test new work in IE locally after today, but having SL running the tests as well would be great!

[YoranBrondsema]

Nice, I prefer 2 as well. Can you send me the SL info to my email address yoran@hstry.co? This way I can start setting it up. Thanks!