Rails 7 javascript and asset pipeline documentation not really complete and fully comprehensible

rubyonrails-docs
1

Hi,

I just ported two rails apps from rails 5.2 to rails 7.0, and it caused me some headache and cost me some time to get the asset pipeline and a small ajax function (which just updates a table in a web page) working again. I had to do a lot of trial-and-error and googling other web pages. Since you ask for feedback on the guides pages, I’ll give some in order to improve the pages.

A general problem is that the pages – I admit that many docs have this problem nowadays – tends to tell “do this if you want that”, but never gives a comprehensible overview, picture of what is going on.

One thing is, that with 5.2 I had the javascript in app/assets/javascripts , while newly created and updated 7.0 apps do have app/javascripts. The Update guide does not mention this in particular.

I have not yet really understood wether app/assets/javascripts has completely been moved to app/javascripts, or whether they coexist and what’s the difference between app/javascripts/application.js and app/assets/javascripts/application.js

Working with JavaScript in Rails — Ruby on Rails Guides gives some javascript code examples, but does not mention at all, where to put them and how to include them.

I’m also not sure what is the current syntax of including things, whether

//= require rails-ujs

or

import Rails from “@rails/ujs” Rails.start()

The latter worked, but I found this on some blog page somewhere in the internet, not in the guide.

The rails guide mentions the UJS framework, but does not tell how to get this running or whether it takes additional steps to do so. It took me some hours testing and trying to figure out, why my form does not trigger the javascript to just fetch a table and replace it on the page, and instead always loaded the table as a new page, i.e. did the regular form action and not the ajax action.

On thing was that I had to learn that I had to turn turbolinks off, in order to get things work again, and that the guide was simply wrong in that detail, because it still talks about turbolinks, while rails 7 uses the newer framework turbo instead of turbolinks. Therefore, it takes a

data-turbo=“false”

attribute, and not a

data-turbolinks=“false”

as told in the guide, to turn turbo off and keep it from spoiling the ajax function. Given the fact, that turbo seems to heavily impact the page behaviour, the guide tells very little about this. And it does not tell how to get completely rid of it.

And I had to do a

yarn add @rails/ujs

and to change the application.js in order to get things running (while not yet really understanding sufficiently, what’s going on).

Similar difficulties with the asset pipeline.

Both guides do not really match rails 7, and they are not enabling a reader to understand what’s going on. As mentioned before, they are a collection of tips and tricks of the style „if you want to achieve behaviour x, then write line y into file z“, without ever explaining, what file z is, what syntax it uses, where it goes to, or – e.g. application.js – where it is, in app/assets/javascripts or app/javascripts (in both directories there is one).

For both the asset pipepline in common, and the javascript thing in particular, some overview is needed. Which parts, which files of the rails app do belong to this functionality (e.g. what role do package.json and yarn.lock play here), what are the rake/rails commands to steer that thing, and what is the meaning and syntax of all those files in that game. I see several files where I simply cannot tell whether they are relevant for the asset pipeline and javascript, or not.

E.g. it took me some time to understand, that this is done in two steps, first create things in products, and then pack them together.

And I did not get the pipeline working in development mode. Even in development mode I have to run assets:precompile to get things running. For the shere fact that I do not see a description about how things are supposed to work in development mode, and thus do not know where to start debugging.

regards

11 Likes
2

never gives a comprehensible overview, picture of what is going on.

When in doubt, what I do is I run rails new myapp and have a look how things look, then compare with the app that I want to upgrade.

I have not yet really understood wether app/assets/javascripts has completely been moved to app/javascripts

I upgraded my app to Rails 7 and now have all JS in app/javascript. I have an entry point app/javascript/application.js and another app/javascript/admin.js. They have regular ES6 syntax, no //= require directives.

//= require rails-ujs

In the beginning I also still used Rails UJS and included it like import Rails from “@rails/ujs”; Rails.start() from the application.js. Then I progressively converted all the forms to Turbo (import * as Turbo from "@hotwired/turbo" + Turbo.session.drive = false; in application.js + adding data: { turbo: true } on the individual forms), then I finally removed Rails UJS completely and am now in the process of enabling Turbo Drive application-wide.

Similar difficulties with the asset pipeline.

The Sprockets asset pipeline is there like before, by default. Although the longer term plan is to replace it with Propshaft. My app still uses Sprockets and I don’t remember particular difficulties on that front when upgrading to Rails 7. It is mainly configured with app/assets/config/manifest.js.

first create things in products, and then pack them together.

I’m not sure what “products” and “pack” means here. The default in Rails 7 is not to bundle JS files at all, the asset pipeline only minifies, zips and fingerprints the files. Then those JS files are referenced from the HTML head using an import map. In our own app we got some issues with import maps and decided to use jsbundling-rails with esbuild, which we are happy with. This way the JS is bundled and it is much faster than webpack that we also used at some point. Anyway, whether using importmaps, jsbundling-rails or webpack (via webpacker or shakapacker), in all cases Sprockets doesn’t take care of bundling the JS any more. And relatively soon (even now if you want to try), it should disappear as propshaft should take care of the remaining responsibilities.

(Someone please comment me if I’m saying something wrong…)

Even in development mode I have to run assets:precompile to get things running.

What does it mean “to get things running”? I would try to decide if you want importmaps, jsbundling-rails or webpacker/shakapacker and see where that leads you. Then you might be able to ask more specific questions. You could also compare with a new Rails app (using the various options to choose between importmap, jsbundling, etc) to see how things should be organised.

2 Likes
3

Yes, that’s what I did and often do. Crating a new app and compare.

But in this case, it does not provide much help.

I would try to decide if you want importmaps, jsbundling-rails or webpacker/shakapacker and see where that leads you.

Shouldn’t this be part of the guide, to tell, that these options exist and how to decide and configure?

3 Likes
4

Sûre, please feel free to improve the guides :slightly_smiling_face: hopefully someone else could say what is the best process to submit such a change proposal.

5

Unfortunately I cannot improve the guides before having completely understood what is going on in the rails 7 asset and javascript pipeline. And currently I’m not even close to that.

1 Like
6

importmaps: requires either an internet connection or having the library in the vendor/ directory locally. also not widely used so you are kind of on your own (will certainly change)

jsbundling-rails: (especially with esbuild) very fast, compatible with the wider node community, can use react with .jsx files

webpacker: deprecated, should use jsbundling-rails with webpack, has more features than esbuild but also at least 10x slower

1 Like
7

I’ve said this before on this forum, so I don’t mean to sound like a broken record. But I think the frontend solutions offered with Rails 7 are interesting, but ultimately less than ideal. I think Webpacker was completely fine conceptually, but suffered from Webpack itself being a messy and config-heavy solution.

I think the well-maintained vite_rails project is currently the best solution to assets/javascript/stylesheets today. Think of it as Webpacker, only simpler and faster. Like jsbundling it uses esbuild under the hood, but it also has hot module reloading, so you can develop css/js without refreshing the page, which is a big time-saver. And the opinionated setup uses rollup for production builds, which is more mature than esbuild, and has a huge plugin ecosystem if you have specific needs. (I use it for brotli sidecar assets, for instance.)

I tried to use vanilla Rails 7 for a new project, but after an hour of frontend setup, I went back to vite_rails. Strangely, Vite feels closer to the Rails Way than Rails 7 itself, since it’s opinionated about tooling and asset management in a way that jsbundling or import maps isn’t.

1 Like
8

webpacker: deprecated

As announced on the webpacker repo, it is superseded by shakapacker. Webpack can also be used with jsbundling-rails.

I’ve never tried vite. I’m quite happy with jsbundling-rails although indeed at this point I don’t care much about HMR (I work across the entire stack so don’t spend most of my time on the CSS/JS).

9

Hi hadmut,

thank you for your post. I am currently struggling with Rails 7 too. I have created a Rails 3 inhouse app (later migrated to Rails 4) for touch screen terminals and handheld barcode scanner in our warehouse which is now running for 10+ years. It was easy and fun to develop this app back then.

But todays Rails 6/7 with sprockets / webpacker / webpack / npm_modules / yarn / propshaft / et al and different config files and syntaxes seems way to complex for a beginner like me.

I have created a new rails app from scratch, added a welcome controller, added a default.scss and a view specific welcome.scss and added <%= stylesheet_link_tag params[:controller] %> to application.html and already failing with Asset abholung.css was not declared to be precompiled in production.

I added a little gem flag-icons-rails which provides some nice country flag images. When adding @import "flag-icon"; to app/assets/stylesheets/application.bootstrap.scss I’m getting app/assets/stylesheets/application.bootstrap.scss when running yarn build:css --watch (btw yarn is not mentioned at all in RailsGuides )

These are only two examples I’m struggling with in creating a simple app. And as you have already stated: the current docs are not really helpful (and I haven’t even started to use javascript…)

1 Like
10

Klaus,

I confirm. I started with Rails 2, and this was a robust and comprehensible tool at that time.

RoR has not just become far too complex for beginners. RoR became too complex for it’s own maintainers. Somewhere around Rails 5 the documentation was not able to pace with the changes and could not cover RoR anymore. It was not possible to understand RoR by reading the documentation anymore. The documentation could not explain the tool anymore, and converted into a collection of hints and recommendations of the form „if you want this, try that“. When I criticized that rails apps got bloated from around 1MB to around 100MB from Rails 5 to Rails 6, and most of the stuff didn’t even make sense (I found one thing where it’s own developer had warned to not use it anymore, because not maintained anymore), I was blocked for a week from reporting bugs.

Maybe RoR is beyond it’s peak.

Although I have been using RoR for years and started around 2008, I’m currently far from understanding what is going on there, or from just beeing able to tell what every file in the app is good for. I’m not really able to debug if something is not working as expected. It’s more like guess, trial, and error.

RoR is currently polluted by additions that do not conform to basic requirements of software engineering. It’s not even possible to clearly describe all function groups and to tell, what belongs to what.

I’m not even sure which gems required by earlier versions have been superseded or are still required.

We are living in a decade where the “code of conduct” is far more important than the function and the documentation of the software.

5 Likes
11

Rails 6/7 with sprockets / webpacker / webpack / npm_modules / yarn / propshaft / et al and different config files and syntaxes seems way to complex for a beginner like me

I’m not even sure which gems required by earlier versions have been superseded or are still required

Rails 7 aims to make front-end development simpler to start than with Rails 6 (*). There is no need for webpacker, webpack, npde, yarn, and instead JS dependencies are included with an import map. By default sprockets is used, although propshaft can be tried as a replacement.

For a beginner and when in doublt, install the latest Rails (currently 7.0.2.4) and run rails new myapp without customization flags. This will show the default recommended experience and gems to be used.

(*) “The default path without Node is dramatically simpler, surprisingly capable, and skating to where the puck of future front-end development is likely to go anyway. The alternate path of integration with Node is lighter, less prescriptive, and less brittle than ever before.”

1 Like
12

But this is just a forum comment. Nobody will systematically find it.

Why isn’t there a guides web page with a big table, containing a list of all those gems, webpacker, webpack, npde, yarn, …etc. and a column telling what it is, what it is/was needed for, in which version of RoR it was needed, and which one replaces or supersedes which.

And while the method of just trying a new app and compare the Gemfiles is poor and not a documentation, it is just a list of recommended gems. It does not tell in any way, what happened with the gems from older versions. E.g. when having an app from RoR 5 or 6, you can see that the Gemfile differs from RoR 7, but you don’t know why and what it means.

This is not a professional approach.

3 Likes
13

If you are talking about documentation for upgrades, you can find it here. I combine that with the release announcements on the blog, e.g the one I linked in my previous post, GitHub repo activity, some of DHH’s writings, etc. All in all I manage to learn that sprockets is no longer actively developed and propshaft is intended as replacement, that webpacker is no longer the default, that import maps are the new thing, etc.

If you have proposals to improve the documentation I think it would be welcomed, although I can’t speak on behalf of Rails maintainers. There might also be more reception if you propose concrete code/doc changes instead of pointing out what’s wrong.

14

I just registered to add another thought/2c to this discussion.

Because I’m shocked somehow. Frist, I was reading through @hadmut’s post and I was like: “YEAH! Your’re right! I had exactly the same problems!”. Honestly, I thought I need to grab popcorn, because someone will tell @hadmut where he (and me) can find everything documented - something like a “RTFM”. But the answer that he got was just incredible incomprehensible. @hadmut should contribute to the Ruby on Rails Guides? Wow. He even clearly stated, that he has/had no idea what’s going on. And I can relate to that, so, so much. I’m somehow angry and somehow sad about this kind of response and the whole situation in general. Having a project in mind and knowing that there is no fast help out of that stuff. Because, further more, as far as I read, no one just accepted that this is just a (big) gap in RoR documentation and a RoR problem in general. And it somehow was always like that…

But let me clear that out. I’m not a fulltime developer. But I’m in IT security - so, somehow I’m at least not new to technical stuff. And: I have to deal with a lot of web developers at my company mostly everyday. At least every web developer that I know of - I mean, every web developer that I have to work with over the last 15 years or so - wanted every… single…time the newest and hottest stuff. The more bleeding edge, the better. And I needed to discuss a lot of web projects in terms of security. I understand that this type of developing is evolving very fast - probably faster than any other development. But after I found someone (@hadmut), hat had (or have) the same issues as I have, I really started to think if this is always the right way of doing stuff. Of course I know that, if RoR wouldn’t build in the lastest web sh*t, it probably wouldn’t be used that much. But on the other hand, RoR is also “responsible” (I don’t know a better word for that right now) for guys like me. A web developer n00b that just want to start somewhere. And you know what? You catched me already years ago with your concept… and with ruby (gosh, I love ruby). I already tried the first project in Rails 2-3. Then something in between (Rails 3-4?) until now, where I try to build something in Rails 6-7. Why I’m writing 2-3? Because I started with 3.anything.verynew. And I brought the newest book available that time: Agile Web Development with Rails (Third Edition) - from Sam Ruby, Dave Thomas and David Heinemeier Hansson. Great book which really pushed me into web developing as a hobby. But already then it didn’t cover RoR 3.x Javascript stuff (can’t remember what was going on there and what problem’s I had). After a couple of years, I started again in a time where RoR switched from somehting to UJS and I couldn’t find any good documentation. You know it’s bad, when even Influencers like Ryan Bates (railscasts) or Chris Oliver (gorails) can’t keep up with the changes. And now I’m again at exactly the same spot. Stimulus, Hotwire, Turbolink (Ah! Sorry, just Turbo… I JUST read that this is not a synonym), Importmaps. Yarn - not yarn? People are discussion a way of doing stuff right - but I can’t find consent… And honestly, even your explanation, @sdubois, couldn’t help me. I just have not even the slightest clue how to switch to jsbundling with my already developed code. This all feels like I’m in some kind of a Groundhog Day movie or something. The worst about all this: I somehow crashed my production instance yesterday and I always think/thought it’s just my bad because I just don’t understand javascript (and Ruby on Rails in that part). It got even that far, that I began to hate javascript to the fullest. Let me tell you what: I get goosebumps so bad, when someone is trying to tell me something about javascript… It really is a nightmare already.

So this story shouldn’t be an hate speech. I wouldn’t be here and take the time, if I was just angry and wanting to bi*** around. But maybe my story show’s that there are people out there that love your stuff. That love your Framework. That appreciate your work. And that don’t think (or at least hope), that Rails is not beyond it’s peak. And maybe you can help us to get an even better start with RoR - including fu**ing javascript.

My coworker are always laughing at me, because I stick to Rails that long. Some of them tried to convince me to switch to Symfony (a few even tried django). But even with better documentation, more support, more people that developing symfony… I would love to stay with rails. But if I have sleepless night’s because of adding one single and “simple” selectable-dropdown (first with select2+jQuery, then with tom-select which now just broke my production), it’s very hard to find benefits of using Rails before everything else.

I’m here waiting for anyone to help me with my broken web app in rails 7.0.4 over at Import Maps not using precompiled JavaScript assets in pipeline in production environment only (Rails 7) - Stack Overflow

Regards, Jens

15

@d33pjs I’m sorry to hear your pain. I’ve shared above what I could from my limited experience, just in case there is something helpful in there.

For lack of a better idea (as I know very little about importmaps and the asset pipeline), you could try bisection, i.e. compare an app that works (a new rails app built following the guide) and one that doesn’t and converge them one towards the other until the issue gets narrowed down. It’s time-consuming but quite systematic.

NB: After trying importmaps for a month, we stopped using them for a few reasons:

  • difficulties with importmap --download
  • no distinction between top-level dependencies and transitive dependencies, no straightforward mechanism to upgrade dependencies
  • ga.jspm.io CDN sometimes not available and generally slow
  • noisy JS warnings which I didn’t manage to silence

Instead we switched to jsbundling-rails & esbuild and I’m happy with that decision so far, except for source maps not working.

16

Thanks, @sdubois for your feedback.

Yes, I could try bisection (which I often do, in lack knowledge how things are put together). But as you said by yourself, it is time-consuming and - let’s face it - not ideal. Especially if there is a Newbe like me that even didn’t get the full picture (in some areas).

Also it doesn’t help me with adding (in my example) tom-select to a new Rails7 App, because I couldn’t find a public repo that did it (in newer Rails versions). And of course no one did it “the rails way” (because, as I said, some people say “yarn” some people say “don’t yarn! use importmaps!”) - of course I mean: probably there are people out there that did it, but maybe not publicly.

Another thing that makes it even more complicated: My current problem is, that PROD environment behave different to the TEST and DEV environments (as stated on Stackoverflow, DEV+TEST are working fine, but PROD somehow doesn’t precompile Javascripts that are under another folder like app/javascript/anotherfolder/my_self_made.js - I just haven’t any clue what’s wrong here and where to begin to debug). I don’t know if it would help to switch from importmaps to jsbundling-rails.

But thanks, @sdubois for clarifying and giving another couple of hints.

Regards, Jens

17

In our app using jsbunlding-rails and stimulus we could use tom-select (which I didn’t know) using:

  • yarn add tom-select
  • add <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/tom-select@2.2.1/dist/css/tom-select.css"> in application.html.erb in the head
  • create a stimulus controller
// app/javascript/controllers/tom_select_controller.js
import { Controller } from "@hotwired/stimulus"
import TomSelect from "tom-select"
export default class extends Controller {
  static get targets() {
    return ["myInput"];
  }

  connect() {
    new TomSelect(this.myInputTarget, {});
  }
}
  • register the stimulus controller
// app/javascript/controllers/index.js
import TomSelectController from "./tom_select_controller.js"
application.register("tom-select", TomSelectController)
  • create an html page:
  <div data-controller="tom-select">
    <input data-tom-select-target="myInput" value="awesome,neat" autocomplete="off" placeholder="How cool is this?"/>
  </div>
  • render the page and try out the select form, it works

Screenshot 2022-09-30 at 11.55.18
Screenshot 2022-09-30 at 11.55.182348×278 72.4 KB

Stimulus is part of Hotwire, which I believe now comes by default with Rails 7 apps.

18

Thank you very much, @sdubois! Even though it is not the initial topic, but maybe I can fix my current problem with it (keep you updated, @sdubois). But for now (can’t test right now), I see, you’re not putting tom_select_controller.js into a separat folder (like Replacing Select2 with Tom Select + Stimulus · Cool Request did here: GitHub - CoolRequest/tom-select-demo) and your adding a stylesheet in plain HTML into application.html.erb.

Getting back to topic: From my point of view, the biggest problem, is not to know enough of how all comes together - like @hadmut mentioned - especially in part that involves javascript, hotwire, stimulus, turbo(-links)… I would just wish to have RoR Guides/Docs covered my back.

19

I combine that with the release announcements on the blog, e.g the one I linked in my previous post, GitHub repo activity, some of DHH’s writings, etc. All in all I manage to learn that sprockets is no longer actively developed and propshaft is intended as replacement, that webpacker is no longer the default, that import maps are the new thing, etc.

That’s exactly the situation. Rails 7 might be on the bleeding edge of frontend technologies, but it’s a nightmare to use since there’s (almost) no documentation.

How on earth would you know whether you need sprockets, yarn, webpack, webpacker, propshaft, importmap, jsbundling, cssbundling, esbuild, rollup, sassc, dartsass, or any combination of those? It’s just ridiculous.

2 Likes
20

There is an effort to upgrade the guides on this: Guide to Rails 7 and the Asset Pipeline - #12 by benoittgt

1 Like