Spring fails to notice changes and reload frequently enough that it is a constant hindrance

For the last number of years every Rails project I have worked on has the following setup:

  • developers on mac and linux
  • so we standardize on a Docker setup
  • but it’s slow on Mac, so some people use Docker for the service dependencies not the Rails part
  • as an attempt at mitigating the slowness in Docker either NFS with aggressive caching, or something like the docker volume cache mount options are used Change Docker Desktop preferences on Mac | Docker Documentation

In that set-up Spring causes one of a few problems:

  • doesn’t load or reload changes promptly (or at all)
  • consumes so much CPU as to significantly shorten battery life.

Some workarounds are possible, globally disable Spring and suffer (usually) some issues that actually your auto-loading doesn’t actually work in day-to-day without Spring. Using an fsevents gem to cut down the amount of polling the listen gem is doing (doesn’t work for for Docker on any platform except Linux)

So, maybe this is an autoloading+spring+docker+network filesystem issue exacerbated by teams using different OSs, but this is a common set-up in most all dev teams I’ve worked in.

My biggest gripe is that Spring hides errors all the time, it’s entirely possible to have passing local tests, and push to CI and have them fail (something worked differently with autoloading), or to lose hours in a day to wondering why changes aren’t showing, or diagnosing load ordering issues.

(Note: my most relevant experience is on Rails <6, I believe a lot of this improved a lot with Zeitwerk, but non the less Spring and the loading/setup issues (autoloading aside) is a constant headache.

6 Likes

I have a similar impression in Rails on Mac. No heavy apps running but fans spinning? Check if spring is up.

I’ve been meaning to try CRIU on linux to improve startup. Does anyone have any experience?

I am excluding spring from my Rails projects for 8(?) or so years now. And the few times it was included, it cost me more time to detect an issue solvable by spring stop than it saved me. In my opinion spring should be opt-in when running rails new to not only newbies some headaches.

3 Likes

DHH has indicated that the Rails defaults are unlikely to change, but he’s supportive of a rails new --minimal preset.

People seem to be coordinating the work around that in Interactive “rails new” if you’d like to participate?

Spring has been nothing but a nuisance for me over the years, to the extent that I now have DISABLE_SPRING=1 in my bash profile. It has cost me far more hours trying to diagnose problems that never really existed than it has saved, including excessive CPU use, not reloading files that have changed and (recently in CI): The user limit on the total number of inotify watches was reached.

I am currently working with an app that has somehow integrated Spring as a dependency and so my DISABLE_SPRING=1 config doesn’t work, and the only way I can reliably get any Ruby/Rails commands to work is by killing Spring first:

pkill -l spring; bin/rspec …

The thing is, I’m a relatively competent developer and have been doing Rails for years, and this still sometimes bites me, and when it does, it’s really confusing. Spring issues for new developers must be very confusing indeed.

edit I should point out, I don’t use docker or networked filesystems, these issues all happen on the local machine.

3 Likes

Have you ever restarted spring, when it wasn’t needed, just to be on the safe side?

Spring trades cpu cycles for developer mental overhead, because every single time something’s not working right you need to consider the impact of running stale data. It’s rarely a trade worth doing.

I never been vocal about spring because I would always remove it. But my pet peeve is that it sets up new programmers, especially rails first timers for failure by default. And they don’t even get much benefit to begin with because newer apps are light in dependencies and will load pretty fast.

It would be nice if spring were opt in. For example, when you start a rails server, or console print Rails booted in 2.3s. Add spring to your gemfile to speed this up.

3 Likes

There are plans to add a --minimal version of the Rails setup that (among other things) automatically skips spring.

I like the Rails booted in 2.3s. Add spring to your gemfile to speed this up. idea.

Is there any consensus on how long “too” long is? It would not be hard to code this feature, I just don’t know what the threshold should be.

Walter

@Betsy_Haibel That doesn’t really fix the issue though. I already skip spring on any new Rails app I create, and have DISABLE_SPRING=1 in my bash profile, but --minimal has two problems:

  1. Should Rails really default to installing such a hated and broken piece of software?
  2. New users are unlikely to use --minimal, or even know what the problems with spring are, so the chances are they are going to continue to get hit by spring problems.

To paraphrase one user I talked to last night:

“When I was beginning Rails I would re-boot my machine whenever I had problems with the Rails app not seeing my code changes because I didn’t know there was a process you could restart.”

The fact this person is still a Rails user after this experience is pretty remarkable.

2 Likes

@will_j I hear what you’re saying. I’ve definitely lost time to Spring issues before and I agree that that aspect of the beginner experience isn’t acceptable.

I’ve noticed a tendency among Spring haters to assume that everyone else hates Spring as well. They therefore assume that disabling it is an easy decision! But I personally really like it, flaws and all. My experience of using Rails is that before Spring became a default, slow Rails startup times were one of the biggest drags on people using TDD. The test quality of the apps I worked on just kind of …magically went up… after Spring was introduced. I think that slow Rails boot times are a bigger issue than a lot of people credit, because people either don’t notice or don’t want to admit the ways that they change their behavior to avoid waiting for a boot.

So I think the situation is complex.

I think it is very unlikely that DHH will straight-up remove Spring from the Rails defaults. So to me, “Spring is widely-hated and should be removed” is a reasonable sentiment, but not one that moves the ball forward. Even though I personally like Spring, I respect that others don’t and that it can complicate the newbie experience a lot. So I’d love it if we could find a solution that improved that experience other than “just remove it from the defaults.” To be pretentious for a moment: something that moves us past thesis and antithesis into synthesis.

I like the idea of having a Rails booted in 2.3s. Add spring to your gemfile to speed this up. message. Elsewhere I think I’ve seen @dhh float the idea of only turning Spring on after application boot reached a 3s threshhold.

What are some other ideas that would keep Rails boot fast for large apps, but not force newbies to learn Spring’s idiosyncracies straight off?

4 Likes

Nowadays I don’t feel the need to restart Rails often, outside of when I’m writing tests, Then again I don’t practice TDD. I usually spend some time shaping and refactoring before I’m ready to lock the solution into place with tests. Long running rails s and rails c processes cover most of my needs.

I removed Spring ages ago some time past the beginner stage. Think my mental model of Ruby and Rails then wasn’t sufficient to cope with debugging code around the idiosyncrasies of Spring.

Now that my understanding of Rails is a bit more mature, all this discussion makes me inclined to give Spring another try. It’d be nice to have a clear way to toggle ‘Spring mode’ on and off (there might be, I can’t tell from scanning the docs).

Speaking from the other side, I’ve appreciated Spring ever since it came out. It’s not without issues, but it’s been on the balance positive.

The benefit is exactly that: I can run rspec a bunch of times without waiting for whole seconds on each single test run. This makes the write-run-debug cycle way smoother, it’s day and night. And I wouldn’t want to include that manually on each rails new.

I’m now well informed of the caveats. I tend to do a spring stop every time I change app configurations or anything that a code reloading might not pick up. If I’m making a lot of changes or do anything special (eg bisecting a bug with git and rspec), I’ll prefix DISABLE_SPRING=1.

However, it reminds me that I find the experience around restarting Rails to be kind of clunky (it’s also not a trivial problem, mind you). When I should I restart? If I have a console and a server running, what do I do? Just ^C and restart? Spring stop in between? What if I change gems? and so on. That might benefit from some improvement.

My main point being: there are indeed appreciating users of Spring. While I empathize that some people don’t like it, there really is a bias of availability: people who are appreciative or slightly appreciative of Spring are just not that motivated to show up in “I dislike Spring” threads. Sorry if that’s a bummer :wink:

3 Likes

Agreed - I’ve removed Spring every time it gets added because not only are there edge cases where it doesn’t reload correctly, I wasn’t able to find a good way to manually restart it. I remember struggling with it for a few days and dealing with it was just too hard to manage.

The problem is that if the size of your app is large enough that Spring is helpful, it’s also likely so large that Spring will get bitten by the complications of the app.

1 Like

I’m curious how recently a lot of folks who routinely disable Spring last tried it. My experience is that now that “auto-restart if specific files have changed” has landed, a lot of my previous Spring WTFs have melted away.

Unfortunately Spring got in my way just the other day with a brand new Rails 6 application. I didn’t look into it because my response was of course “ugh Spring again” followed by immediately getting it out of my app.

Symptoms were failing tests, which passed when run without Spring, and rogue Ruby processes that I had to kill.

I have recently reenabled Spring in an old Rails app that was updated from 3 to 6 and so far haven’t noticed any problems. It significantly improves my workflow (fast test on file save).

I’d like to re-state a point I made among the rest when I opened this topic, but as good as Spring may be in environments where you run Ruby directly on your host, as soon as you introduce Docker, or VirtualBox or any kind of shared network filesystem, then the guarantees break down, because the listen gem stops noticing changes promptly|properly.

Most of the problems with Spring missing changes/etc I think are actually problems with the listen gem (and the fact that cross-platform, cross-filesystem, cross-technology filesystem events are really hard to get right, and software like NFS makes no effort to get it right because of the performance implications and fact that it may be nearly impossible to get right with fsattr caching)

I never looked into it, but I spoke with Will J (in this thread) a bit about it off-line, and we lamented the lack of some “debugging” pipe where Spring can notify about changes it has noticed, and how it’s responding to those. That prompted me to go check the docs (they don’t have a diagnostics debug channel/socket that shows what changes it noticed) they do however have “support” for sharing the Socket with Docker, but I don’t really see how that helps if you edit in your host, and run the entire Ruby stack in the Container - GitHub - rails/spring: Rails application preloader

If someone more familiar with Spring knows if the “listener” instance is exposed, building a diagnostics callback should be possible to just print the changes that were noticed (or not noticed) GitHub - guard/listen: The Listen gem listens to file modifications and notifies you about the changes.

I wonder what would happen if we flipped things around and had spring (or something else) define a listening socket that other tools could write file paths to on changes.

I mean, when do files change? When an editor writes data out or when doing certain git actions. Probably some docker stuff too. If we could define hooks and get buy in from the tools we could get something that’s more reliable and cross-platform. Maybe!

Worth reshaping the problem at least, because I think listen is doing a good job, well, as good as it can do.

1 Like

The listen gem has a Socket option that they moved to a peer gem at the latest big release because it was overcomplex and not that widely used (according to the release notes)

so we’re definitely not that far away.

Also, as Docker (on mac, principally, where you always have two levels of filesystem mount mapping) is currently looking at implementing a new filesystem mounting platform based on mutagen which claims to be amazing for this stuff - there’s a beta build of Docker4Mac here File system performance improvements · Issue #1592 · docker/for-mac · GitHub that people are discussing.

If I can wrestle my macbook back from my wife, I’d like to do some testing on how fsevents do, or do not propagate over various share mediums.

2 Likes

Update to add that the Listen gem does claim to provide a debug env var option:

-GitHub - guard/listen: The Listen gem listens to file modifications and notifies you about the changes.

I will look into that shortly.

1 Like