Rails-7 / Capybara speedup possible?

I really love the new rails-7 way, and together with vite, svelte and stimulus, see.

But, in the end, there remains one problem: painfully slow Capybara, especially the loading/initialisation time for each test (the running time for the test itself is OK).

I opened a issue on github/capybara too.

I work on a MacMini 2023. Before bying I also tested a MacStudio, but doubling the cores makes no difference because a single test can only use one core (except when using gems like parallel_tests, but that does not help for a single test). A single Capybara test with nothing more than a content like “visit root_path” takes 9 seconds, running only 9 tests at once takes 33 seconds! For comparison, by rspec I currently have 1,738 tests running for the same app, including these 9 Capybara specs, taking 3 minutes!

Working with much more interactive front-ends it is necessary to implement much more tests: For each stimulus or svelte component, you want … you have to! let the bot do a click and check the effect. That would mean implementing a lot more system tests! By writing a test you have to run it often!

While Capybara itself, with its intuitive commands, is really great it is the loading time of each test that is frustrating.

I am aware that this might be hard to solve, as it would require a solution like this: The browser and Rails environment would have to be started once in the background after opening the IDE, for example, to be initialised when I run one or more tests.

But, on the way for writing more front end interactive apps which is our or DHH’s vision I really see this slowdown of front end testing as the remaining problem.

Are there any ideas?

Best regards, Chris

2 Likes

Indeed, end-to-end tests that rely on a dynamic, JS-driven interfaces sure are “slow.” In my own suites, I run Capybara using the rack driver, which is only effective for me because I try to be vigilant about “graceful degradation” of JS in the client, allowing interactions to work without JS.

I know, this can only go so far.

I thought there were other web drivers for Capybara that eliminated subsequent test initialization time for system / end-to-end tests? I don’t recall…

Thanks, ybakos, and yes, i heard from rack that its fast. But Javascript is the only reason for me to use Capybara.

The Selenium Webdriver should be able to work in a similar way to a normal developer: Browser is started once, and then for each test you do nothing but reload the browser. It depends on the technology: Importmaps is always fast. But, I work with vite: it runs an additional development server, which only rebuilds junks of changed javascript and pushes them to the browser. Such a technique would be hard to maintain. But maybe, with that it could be as fast as rspec :slight_smile:

Or, run cores in parallel: One core for starting the backend, one for building the javascript and one for starting the selenium-driver. And after they are finished run the test on one core.

I think the Rails page’s non local resource (like webfont or tracking js file) broke the loading.

You can clone and testing my system file, it should less than 2 second, the repository didn’t refer any outside resource.

1 Like

Thank you Eric Guo, this is very interesting and a great help! I downloaded your project and tested it: with the stopwatch I measured 3 seconds and the test shows 1.1 seconds, but all this is very fast compared to my project. And I added a javascript and a test for it on your project and it is as fast as. Your project has a very different setup compared to mine:

  • Yours is on traditional asset pipeline, i am on Vite
  • Your are working by headless Chrome, i on headless Firefox
  • You have set puma to silent: true
  • You are on Minitest, i am on rspec

Based on Eric Guo’s project i made a test project. See Results on the README

Vite does not make a difference. Biggest difference ist Firefox: headless_firefox is aproximately 1.3 seconds slower than headless_chrome But all results are fast! For Chrome the slowest result was 2.6 seconds.

And i added a bash-script that measures the time for running a test more exactly then my above mentioned manually-stopwatch and i have to correct the times there:

  • For a smaller project, (now switched to headless_chrome) a minimal test is now taking aprox. 4.5 seconds
  • For the larger (with much more JS and CSS), above mentioned project («9 seconds») now switched to headless_chrome it takes between 6.9 and 8.6 seconds.

So the test project has almost the same setup (rspec/vite) but the difference to my projects is big. For now, I would assume that the number of packages (JS/Styles) makes Capybara slow. But that is just a first glance. All the fonts/styles in my projects are installed in the app, none of them are dragged remotely when the app is running.

Have you tried switching to playwright? Selenium is possibly the slowest of all the options out there.

I haven’t played with it a lot, since I rarely do system/js tests, but from what I’ve heard playwright eliminates a lot of flakiness in system tests because it’s not slow like selenium.

1 Like

@chris.covington Thanks! I played around with playright on a test project: Without Capybara DSL it is faster: 1.5 seconds compared to 1.9 seconds with Selenium. And the tests look cleaner. It is a good replacement for Selenium.

It does not solve my 8.9 second problem, but I now have a path and hope to be able to report soon.

I have now installed playwright in my above mentioned fat and slow project, on a spearate branch, to be able to test it alongside selenium, result:

  • First run after changing javascript:
    • Selenium: 11.5 seconds.
    • Playwright: 6.4 seconds
  • 2+ run:
    • Selenium: 6.5 seconds
    • Playwright: 2.6 seconds

For comparison, a simple request-spec (with only one get … step) takes 2.2 seconds (all results measured by this runner). Seems like Playwright is getting the maximum out of it.

1 Like