Regarding JavaScript testing

rubyonrails-core
1

Hi, I was happy to read the new features for Rails 5.1 here:

Specially as I have already been using webpack,npm and yarn for a while (well, yarn just recently, of course) and I'm pretty happy with the results. My main motivations were client-side performance improvements and source-maps support, but there were many more reasons I chose webpack as stated in the Goals section of this article:

http://rosenfeld.herokuapp.com/en/articles/2016-02-29-getting-an-spa-to-load-the-fastest-possible-way-and-how-webpack-can-help-you

As you can guess, I'm pretty happy with the direction Rails is taking as an opinionated system (even though I'm moving towards a discrete system I still see value in frameworks like Rails that are full featured and opinionated, specially for new or small-sized applications).

There's just one thing I found missing in this release notes and I don't want to sound critical. It's the other way around. As I said, I'm pretty happy with the direction Rails is taking, so this is a suggestion to improve it further if someone is interested on working or discussing this topic. I don't have time for that so if anyone is interested on making it happen feel free to disregard this suggestion.

I've been writing Single Page Applications since 2009 and even though I've tried testing them through solutions such as Capybara (and similar solutions for other languages such as Selenium) it doesn't really scale. At some point I completely dropped Capybara because I found that those specs were hardly executed since they were disabled by default because they were simply too slow.

I'm not saying adding Capybara to Rails was a bad idea. I do see value in it and I'm happy Rails adds it by default. I'm just saying that I feel like Rails isn't really aware of how SPA applications are developed in the wide. While browser tests allow you to test JavaScript they are not the only way. Using JavaScript test frameworks is what actually enable SPA testing to succeed. Since Rails 5.1 is supporting SPA I think it would be awesome if it also integrated some testing framework, maybe on top of Karma.js, for fulfilling that purpose and stablishing good practices just like it does for Ruby code.

https://karma-runner.github.io/1.0/index.html

Karma already exists for a while and looks like a mature solution to me, and could be integrated to several JavaScript frameworks, and tests can be run from the console, making it easier to integrate to CIs. Maybe Rails could adopt some well known framework such as Jasmine or Mocha for creating the stubs for new JS files.

Well, this is just an idea. I use my own JS test framework in our application, that's why I don't have a strong opinion on those test runners mentioned above, but I had some experience with them and they should be good enough for most applications.

Once again, thanks a lot for this 5.1 release. It seems like Rails is getting exciting again :slight_smile:

Best,

Rodrigo.

2

Thank you for the feedback Rodrigo.

I think it is worth to explore possibilities for JavaScript tests. I don’t see why Rails could not integrate better with the test ecosystem in the JavaScript community and I think it is a valuable direction to take. We didn’t included anything in this line in 5.1 because we didn’t have someone the champion this feature but we would be happy to have something for a next release.

So, now that the ground base is set for better relationship with the JavaScript community it is just a matter of someone willing to explore how the test history will be.

Looking forward to see what is coming on this front!

3

Great to hear that :slight_smile: I'm looking forward to see some Mocha integration in Rails 5.2 :wink:

Cheers, Rodrigo.

4

Couple of data points:

We are moving to Elm for all our client code, so Karma or Mocha wouldn’t help us much, as we use elm-test.

Also, we have multiple bundles for different SPAs living within the same Rails app, so I also wouldn’t want to see the hardcoded assumption that all tests have to be run as part of a single suite.

So, when considering a feature like this, it would be nice if it were designed in some pluggable way that let you choose different test frameworks and strategies, but still integrate well with built-in webpack/etc support.

– Chad

5

For our particular application, I have created my own test runner on top of Buster.js. One of the reasons is that I wanted more than just unit tests, as I wanted integration tests for my JS bundles. In order to ensure all dependencies are declared correctly, we run each top-level context in the suite in a separate iframe. It slows things a bit but at least each top context starts with a clean environment.

Also, differently from Mocha, Buster.js (and Jasmine too if I recall correctly) doesn’t support a beforeAll/afterAll hooks, which is very useful for integration tests, and if I recall correctly, Buster.js, being focused on unit tests, doesn’t allow one to specify the order of the tests (at least this is how it worked by the time I created OOJSpec).

Our application is quite old and still relies on a lot of jQuery code to these days, including live events, that are registered once and never unregistered since the application doesn’t need to unload some modules. That means some code must be ran just once in a beforeAll block for a given iframe.

Also, changing the DOM, can be a bit slow, so I split some features in several tests, to check a dialog was created, that some bindings are working correctly, that some actions happen when the ok button is clicked in the dialog and so on. There’s a separate test for each part but some tests will depend on the state set by previous tests. This speeds up the test suite a lot, but since this is considered bad testing practice, I had to come up with my own runner since I’m more interested in running my suites fast than on good practices.

Anyway, Karma is not really a test framework or runner and could be used with any runner. I’ve even integrated my own runner to Karma.

But it should be possible to adapt Karma to run each top context (Mocha, Jasmine, elm-test) in a separate iframe and I guess this would help with your scenario.

But I agree it should be pluggable at some point. But I just think that it’s important to integrate one of them by default. It doesn’t matter much which one, as long users would write tests for SPA properly rather than using Capybara for JS tests… When that happens we can actually say Rails is embracing SPA too. But the way I see, currently Rails is moving towards this goal, but it’s not there yet because JS is still a second class citizen in Rails apps, since Rails generates all kind of tests stubs for Ruby code, but not for JavaScript code. Often JavaScript code needs more tests than server-side code in SPA.