JavaScript Documentation Coordination

I’d love to see some guidelines on sprockets JS and Webpacker JS co-existing in older code bases, especially during a migration. I touched on this in the Big Ball of Sprockets Frustration thread and have since done a bit more digging.

As I understand it now, it’s quite feasible to leave existing (let’s call it legacy) javascript as-is, asset-packed by sprockets as it traditionally has been, and add a separate pack file for Webpacker based JS. You can then make some decisions in your layout or wherever makes sense to include one or the other (or both maybe, in weird cases).

Then all that’d be needed would be a migration path.

Specifically what I think would be a great thing to walk through in some detail is migrating some jQuery based code. I know that’s probably ancient history in Javascript terms but as a mostly back-end guy last time I checked (which seems only weeks ago but realistically was several years) jQuery based rails-ujs was the way to do things.

From what I understand the reason jQuery has receded is because ES6 more or less covers the ground it used to cover. It might well be that covering JS migration from legacy style to modern is well out of scope for Rails documentation (a reasonable perspective), but even if that’s a given I’d say some kind of structural advice might be worthwhile. Maybe something like what I laid out above - keep both pipelines working, include one or the other and consider using [external resource A, B or C] to learn to migrate your legacy JS across to modern ES6, Webpacker stuff.

edit: I didn’t feel right putting this into a pull request because I don’t actually have enough detail to write this myself. I considered a GitHub issue but didn’t want to spread the discussion across too many platforms. Happy to engage there if that’s what is preferred.

2 Likes

Yes, you can leave legacy JS in sprockets and write new stuff in Webpacker, there might be some problems if they needed to talk to each other, but including them separately should work.

I’m fine to engage here for as long as the WTF May is going on, but I’m also open to issues on that repo as being a good way to track requests.

(FWIW, my recollection of moving a jQuery-based site from Sprockets to Webpack is that it was less trouble than I anticipated, but I don’t have access to the repo anymore to go back and check my steps. I think the really non-intuitive part was making the jQuery global object visible…)

1 Like

Thanks! I’ll keep an eye on the repo and if I get some more valuable understanding of the process as I’m doing it I’ll aim to contribute. This area more than anything else is my major pain point in Rails and if it can be made better and easier for others, that’d make my frustration with it a bit more useful in hindsight. :slight_smile:

I wrote up some detailed thoughts on migrating from Sprockets to Webpacker that may be helpful to you in the mean time and perhaps a source of info to adapt for the guide. How we switched from Sprockets to Webpacker - rossta.net

2 Likes

Thanks for taking on this effort!

I don’t want to pull too many things in but would you think this effort should coordinate with whether to add Typescript types to Rails Javascript? They’re not exactly the same but I thought I’d mention it.

I would be completely up for adding TypeScript types, but I think that’s probably a separate effort. There are unofficial type definitions for some parts of the Rails JavaScript Expanded Universe. For the book I found types for ActionCable, ActiveStorage, and Turbolinks, but not UJS. But if UJS is going away in Turbolinks 6, that might not be an issue.

I’ve lived through this sort of sea change twice now – first with the abandonment of Prototype.js, and now with Sprockets. In both cases, the cusp time was the very worst.

It’s impossible to overstate how confusing it is for me to try to use code that currently works (oh, let’s just name it: bootstrap.js) in a new module-preferring world where jQuery doesn’t exist as a window object. The quantity and severity of code-smells I contributed in just trying to get a modal to be reusable by multiple controllers was quite astounding. And the amount of really bad advice I encountered in the SO-verse and Blog-land made this very difficult to climb as a learning curve. Gems that I have used to inject bits of magic into the front-end and relied on for years just don’t work any more, and their README is no longer authoritative.

Eventually, this will all work itself out, as gems are updated, and Bootstrap 5 brings us a new, modular JS without a jQuery requirement. Rails is a big tent, and it casts a long shadow, and all the sideshow folks will want to come along in whichever direction the train is heading. (Sorry to mangle the metaphor.)

Right now, because I want to ship something this month, I resorted to injecting external script references to jQuery, the Bootstrap CDN, and my tiny little file of extensions to that code (written in what I like to call Ye Olde jQuerie) – all 55 lines of it, including comments and whitespace – into application.html.erb, and called it a night. This resulted in the removal of hundreds of NPM bundles (yay!) and a halving of the size of my application.js (yes, I know it all went in other files, which all load from CDNs, so I’ve just moved the junk drawer).

It was a humbling defeat for someone who has been writing JavaScript as a side job since 1997, who still is the List Mom for the prototype-scriptaculous mailing list. I’d really like to believe that this is the way forward, but it is not for me, at least not this year.

So much of this tooling is built around the very real needs of front-end developers who don’t have the luxury of a mature framework like Rails, who have to build each thing from scratch. It all feels like so much Astronaut Architecture when you bring it into the context of wanting to handle an Ajax update from a modal form. I use Rails because I don’t build SPAs. I don’t need a whole SPA factory full of custom tools designed around solving a problem that I don’t have.

I can see the appeal of using the same tooling that JS-heavy front-end devs are experienced with, so that Rails devs who manage the deployment of multi-tiered JS-heavy applications can fit right in. In my mind, that smacks of the same false syllogism that gave us “run JS on the server, what could go wrong?” and “one language everywhere – your devs can be twice as versatile”. I would love if this could be optional, for those who need it, and that something more fit to a JavaScript-lite approach was still layered in there somewhere.

My two pfennigs, anyway.

Walter

8 Likes

The first PR in the Webpacker guide is at Moving toward a first pass by noelrappin · Pull Request #1 · noelrappin/rails_javascript_documention_drafts · GitHub – it’s just a start, please comment. More updates coming.

I’ve lived through this sort of sea change twice now – first with the abandonment of Prototype.js, and now with Sprockets. In both cases, the cusp time was the very worst.

It’s impossible to overstate how confusing it is for me to try to use code that currently works (oh, let’s just name it: bootstrap.js) in a new module-preferring world where jQuery doesn’t exist as a window object. The quantity and severity of code-smells I contributed in just trying to get a modal to be reusable by multiple controllers was quite astounding. And the amount of really bad advice I encountered in the SO-verse and Blog-land made this very difficult to climb as a learning curve. Gems that I have used to inject bits of magic into the front-end and relied on for years just don’t work any more, and their README is no longer authoritative.

Eventually, this will all work itself out, as gems are updated, and Bootstrap 5 brings us a new, modular JS without a jQuery requirement. Rails is a big tent, and it casts a long shadow, and all the sideshow folks will want to come along in whichever direction the train is heading. (Sorry to mangle the metaphor.)

Right now, because I want to ship something this month, I resorted to injecting external script references to jQuery, the Bootstrap CDN, and my tiny little file of extensions to that code (written in what I like to call Ye Olde jQuerie) – all 55 lines of it, including comments and whitespace – into application.html.erb, and called it a night. This resulted in the removal of hundreds of NPM bundles (yay!) and a halving of the size of my application.js (yes, I know it all went in other files, which all load from CDNs, so I’ve just moved the junk drawer).

It was a humbling defeat for someone who has been writing JavaScript as a side job since 1997, who still is the List Mom for the prototype-scriptaculous mailing list. I’d really like to believe that this is the way forward, but it is not for me, at least not this year.

So much of this tooling is built around the very real needs of front-end developers who don’t have the luxury of a mature framework like Rails, who have to build each thing from scratch. It all feels like so much Astronaut Architecture when you bring it into the context of wanting to handle an Ajax update from a modal form. I use Rails because I don’t build SPAs. I don’t need a whole SPA factory full of custom tools designed around solving a problem that I don’t have.

I can see the appeal of using the same tooling that JS-heavy front-end devs are experienced with, so that Rails devs who manage the deployment of multi-tiered JS-heavy applications can fit right in. In my mind, that smacks of the same false syllogism that gave us “run JS on the server, what could go wrong?” and “one language everywhere – your devs can be twice as versatile”. I would love if this could be optional, for those who need it, and that something more fit to a JavaScript-lite approach was still layered in there somewhere.

My two pfennigs, anyway.

Walter

1 Like

I applaud this effort and hope to (read: will - I promise!) make contributions to it.

However: it’s going to be very frustrating to develop documentation for features that are soon to be dramatically modified - especially when we don’t get told what will replace them.

For that reason, I’ve logged this request: Turbolinks and Stimulus are developed behind a wall

Your comments and support are appreciated.

1 Like

Yep. It is also frustrating to be writing a book for those features :slight_smile:

3 Likes

Silver lining: it gives you a proper opportunity to fully flesh out your chapters on CableReady and StimulusReflex. Only half-joking!

Thanks for taking this on. I have tried a couple of times to run rails 6, and the whole webpacker thing ended up tying me in knots. There just seems like a whole load of new stuff to learn and manage.

I ended up deciding that I’d just wait for the guides on ‘Webpacker for rails 5 users’. I had assumed they would be quickly produced and numerous which hasn’t proven to be correct.

I’m delighted that you’re giving this a shot.

One comment: It might be worth splitting out separate sections for

  • Upgrading Rails 5 app to Rails 6 and
  • New Rails 6 project

thanks.

Should a note be added to guides.rubyonrails.org to this effect and maybe pointing to edgerails too? This may just add too much maintenance, but when The Asset Pipeline — Ruby on Rails Guides is so out of date?

And thanks to all for working to make these improvements

I’ve got a new update at GitHub - noelrappin/rails_javascript_documention_drafts that includes some feedback, some pull requests, and moves forward a bit. Comments and PRs welcome.

4 Likes

I’ve got no specific comments on anything in the current draft except to say this is already probably the most useful single document on using Webpacker in context of rails, even with the missing sections. Thanks for shepherding this thing!

1 Like

Hey there! I’ve been frustrated with the lack of Documentation for some of Rails’ modern Javascript infrastructure, and I’m wondering if there’s anyway that I can help. Thanks for making this thread.

It’s worth emphasizing what @tvanderpol and @rossta touched on earlier in this thread. The efforts to make Webpack easier to use and more effective as a replacement for Sprockets are critical, but it’s not entirely a one-way street. As devs, we also have to address the fact that a lot of JS written back in 2012 just isn’t how things are done in the JS world in 2020, and some of it will need to be updated to work in this decade.

At work we made basically every misstep possible with our Rails app’s frontend. Written in Coffeescript, multiple ways to include dependencies, widespread use of global functions, embedded ERB, Angular 1.x :grimacing: … long story short, we’ve pretty much decided to call everything in the Sprockets pipeline “legacy”, and (re)implement it all in a modern Webpacker+Stimulus JS stack. Learning how to do that and have everything coexist has been basically fumbling in the dark at times, and Ross’s post would have been extremely helpful with that process.

All that is to say, in addition to the Webpacker guide in progress, I would be hugely in favor of a “Migrating From Sprockets to Webpacker” guide, or a “Modernizing Your Rails Frontend” guide, for those of us with mature full-stack Rails applications. Making that process easier would be a big step in consolidating from two asset pipelines to one.

6 Likes

@dzunk could you expand on using Coffeescript being a misstep?

For sure. To me it’s a microcosm of how the Javascript of yesteryear is quite different than the Javascript of today. When our Rails app was created eight years ago, Coffeescript and the Sprockets asset pipeline in general brought a lot of new, convenient, and sometimes necessary syntax and functionality. The JS ecosystem was still in its infancy - ES6 was several years away, JS package management had just become a thing (bower was released in 2012), and things we would consider essential nowadays like babel or postcss were not even conceptualized. Node.js itself was version 0.9.x.

These days, ES6+ has adopted much of the sugar that made Coffeescript so appealing in the first place, and the tooling and community around JS have grown tremendously. Many of the problems that Coffeescript was created to solve are no longer issues with a modern Javascript stack.

I think it’s fair to say the broader web development community has moved towards vanilla JS, or newer transpiled languages like TypeScript. Job applicants likely don’t know Coffeescript, and learning it in 2020 is a hard sell. There are orders of magnitude more resources, documentation, blog posts, etc. out there for Javascript vs. Coffeescript. Things like examples from package documentation or snippets from StackOverflow need to be translated to Coffeescript before they can be used or referenced.

Coffeescript is still a popular and well-maintained package, with over a million downloads a week on npm. Companies like Basecamp use it extensively, to great effect. It’s not going away anytime soon. For our app, at some point our choice to use Coffeescript went from being a time saver or force multiplier to being a maintenance burden.

I’ll try to make this post into something not completely off topic :sweat_smile: This is the only thing I could find in the existing Webpacker documentation about Coffeescript. Is it worth expanding or augmenting that at all?

2 Likes