Just for a bit of history here. When we adopted Webpack as the preferred solution to compiling modern JavaScript in Rails, it was because Webpack offered a superior solution to Sprockets when it came to JavaScript compilation. But none of those advantages were present – at least at the time! – when it came to stylesheets or images or any other static assets. The asset pipeline required less configuration, was faster, and was easier to overall integrate.
Now I appreciate that some of this perhaps comes from the fact that we don’t build SPAs at Basecamp? Maybe there are some component scenarios where people are bundling images and stylesheets together with their JavaScript work, and in those cases, it would be easier to use Webpack directly. As far as I recall, we’ve made provisions for this already. But I could totally see if it’s not very well explained.
Anyway, to the larger question: Should Rails just use Webpack for all asset compilation? The answer is: sure, when Webpack can do it better than the asset pipeline, for the non-component based approach. I haven’t looked at Webpack 5. It’s possible that this deals with CSS and digested images and all the other Sprockets concerns better. If so, I’d love to see a bake-off. Show the whole pipeline working smoother, faster, with as little or less configuration, and with as good or better error messages.
But before we even consider expanding Webpacks role in the Rails asset compilation pipeline, I think we’d do well to document, clean-up, and polish its role as the preferred JavaScript compiler and packager first.
Funny enough, when we first migrated from sprockets the reason was sprockets peformance… if we could compare it with today’s number I would bet sprockets win for a large marging (using less memory and become ready first).
Was Parcel ever explored as an asset pipeline solution? While I’ve never used it on a large project (which is a huge caveat), its commitment to zero-config opinionated compilation and its first-class treatment of non-Javascript assets has always struck me as “Railsier” than Webpack’s approach.
It’s a bit of a tangent from the main thread. But if we’re going to do an asset pipeline bakeoff, I would like to throw a Parcel-based implementation into the mix.
I’ve had some success using Eyeglass for 3 in Webpacker.
1 and 2 seem like they’d require complicated npm-rubygems reconciliation procedures in a Webpacker-only world. (Or, for that matter, in a Parcel-based solution.) So getting to a unified asset pipeline seems like it’s not a smooth path at all. Probably still worth doing…? But complicated.
I did a survey at the time of Webpacker, and I do think we looked at Parcel. But I don’t recall why we didn’t go that route. The basic premise of Webpacker, though, is indeed that it’s zero config. Or close to it. Which is where it is today for a large range of default options (I think we’ve scarcely changed the defaults much in HEY).
Yeah, last we looked at it, there really wasn’t an easy way for us to get engine-bundled JS to work like it does with sprockets. The concept of a dynamic load path wasn’t there. But if there’s a new solution that’s emerged since, that would be awesome.
I think the bottomline is that JavaScript packaging is a… complicated question. Webpacker is the current best effort at providing some great defaults that’ll work for a lot of Rails apps out of the box. But it surely won’t address all setups.
I’d be happy to see more work go into both the documentation, the default configuration, and the continuing to explore ways to make this serve even more cases. Webpacker has basically just been Gaurav, Javan, and a couple of other contributors. We could do well to see more people involved!
And, as mentioned above, I’m always game to consider a bake-off! I’d be surprised if Parcel offered such a monumental leap over Webpack that it’ll be worth the change at this point. But I love being surprised in such a way .
Looks like a great start! I left some comments on the gist. I’m interested in helping write the guide so let me know if and how you’d like to collaborate.
What happened to conceptual compression? We now make poor beginner Rails devs learn both Webpack and Sprockets, for very little gain imo. I get that Webpack is an imperfect solution but having both Webpack and Sprockets is imperfect as well! We should at least consider deprecating sprockets for upcoming Rails releases.
While I haven’t set up a fresh CSS-included Webpacker project recently (I’ll do so and report back), I’ve set up a bunch over the past few years. My experience has tended to be that, while Webpack has pretty sensible defaults, as soon as I need to override those defaults I need to to dig in to both Webpacker’s implementation details and the general difficulties of the Webpack configuration DSL.
From the perspective of “is this good enough,” yeah, I think this is good enough. It’s tricky to create a truly non-leaky abstraction here, because Webpack’s config JSON has so little unified design philosophy of its own. And so “basically reasonable defaults, and a readable config file to change frequently-project-specific things like paths” is great to have.
From the perspective of “can this be better,” I think that basing holistic asset management off of a solution like Parcel that’s committed to zero-config setup might offer better opportunities for “conceptual compression” by reducing the number of concepts that need to be compressed. This in turn might make it easier to configure beyond the defaults without the abstraction leaking. But I 100% admit that this is speculation rooted in toy projects rather than something based on using Parcel in anger.
From a practical, immediate steps perspective: I think that I ought to set up that fresh CSS-included Webpacker-but-not-Sprockets project soon, document all of the things that annoy me about the process, and see if it’s worth upstreaming the extensions I usually make.
It seems some still value Sprockets for an easier out of the box experience if you don’t need a big JS framework. I hope the maintenance situation can be improved. If a project looks kind of dead, it’s not motivating to submit PRs, so there’s a chicken and egg problem in getting new maintainers, if that’s what needed.
I think the changes to Sprockets needed to support Javascript modules would be too dramatic to go upstream. v4 was in beta for many years, and it’s quite complicated system from what I remember when I was working with it.
I also really dislike the current direction. Adding yet another javascript configuration file, es6 files needing to end in .es6 made me stay with Sprockets 3 until I replaced it with my own fork using rollup.js that keeps the old configuration style while supporting es6 and was able to remove all the directives that sprockets needed.
I still love Sprockets and hope the changes will happen.
A lot has been said already! I’ll add a few extra bits:
If webpack becomes the one and only solution at some point, people need to fully realise the incredible number of sub-dependencies this brings in (to fully grasp it, check out this visualisation). For apps with limited front-end needs, this increases the attack surface, some would say for little benefit. I wonder if Rollup (Visualization of npm dependencies) could be used more in those cases, as pointed out recently to me by @Ken_Collins1 (but if webpack is mandatory, I guess this will be complicated)
yarn is to be used, but upgrading an old Rails app to use it, I found it hard to figure out which version of yarn should be used (1 or 2). I ended up using settling on v1 for now after reading the edge guides section for Rails contributor, which specifies the v1 repository for Ubuntu (Installation | Yarn).
Both these points made upgrade from old apps a confusing/concerning experience (for a security-concerned industry).
Also, as many have said, I love the speed of Sprockets, still. Webpack integration makes reloading a page slower at this point.
Webpack integration makes reloading a page slower at this point.
I’d be interested to hear if you have any profiling info to share (here are some debugging tips and some ideas for improving webpack performance). I haven’t observed the same slowness with webpack relative to Sprockets (if anything, Sprockets has been slower for me) so I’m curious to learn about others experiences with performance.
I use to import third party themes from my project and using Sprockets everything used to work smoothly but now it’s rather difficult. Furthermore I have to change a lot in the javascript of the theme to use webpacker instead with sprockets I didn’t need to change almost anything.
I’d love if both Sprockets and Webpacker could coexist.
Thank you.
Existing guides also need to be edited. Existing guides still talk, sometimes in passing sometimes directly, as if you will be compiling your JS with Sprockets… and the things they tell or imply you to do simply won’t work with an out-of-the-box rails new rails 6 app, without changing default configuration.
A release (or even a PR) would not be considered ready for release with new functionality that lacked tests; the new func simply wouldn’t make it in if there wasn’t time to write tests; I find it frustrating that docs/guide updates to not be wrong aren’t considered equally mandatory.
Webpacker succesfully makes using modern JS toolchains in Rails way easier than before Webpacker existed.
But it is still really confusing. Some specific issues:
The Guides still don’t mention webpacker at all as far as I know. The Guides have been a really awesome reference to figuring out Rails and Rails best practices (I see the audience being someone who is already a developer and familiar with the Web, just not with Rails). But they have not been changed at all with the introduction of Webpacker. There is still an Assets guide implying/suggesting you use sprockets for JS, and I believe following it’s instructions would actually not work in a default rails new rails 6. The guides were not updated for Rails 6 asset compilation defaults changed, and they need to be, not just perfunctorily, but actually giving a good overview/introduction of what’s going on, doing what the Guides do well not just barely accurately. Since this has become an even more confusing topic, it needs more, not less, Guide attention.
The sprockets 3 to 4 migration is still REALLY CONFUSING. Rails generating a ./app/assets/config/manifest.js for a long time now, but this file is ignored until you are using sprockets 4 (which was only released recently), and then once your sprockets is using the manifest.js, the one Rails generated has quite different defaults than sprockets 3 had (and also depends on if it was generated in rails 4, 5, or 6, although it’s only used in all cases if sprockets 4). I contributed docs to try and explain what’s going on, but the whole situation is really confusing – before you wrap your head around it, you can start thinking you’re going crazy and sprockets is behaving non-deterministically, because things you weren’t expecting effect how it behaves.
It has been reported again by other users as a bug, showing how confusing it is, for instance here, with no attention
However, at this point – I’m honestly not sure what should be done. Changing behavior AGAIN to make yet another fork (if you have a manifest.js generated by Rails 6.1, then it will be default yet different again) may not be the right solution at this point. I think some real mistakes were made in the past, but they can’t be changed at this point, I’m not totally sure what should be done now, I think some real thought has to be given to what’s the right solution – yes, I am posting a problem for which I’m not sure what the right solution is, better brains than mine may have ideas! But on the topic of “WTF things that really have stymied you”, this is one of them.
.
sprockets 4, which gave me lots of pain, my motivation for trying it was better support for source maps, but I was unpleased to see that if you use SCSS (which is quite popular, right?), the source maps are still broken, it actually seems to make identification of where a line of css source code came from worse than it was in sprockets 3, unless you disable CSS source maps. sprockets 4 makes Chrome browser identification of SCSS css lines _worse_ · Issue #656 · rails/sprockets · GitHub
Thanks! note that all my comments weren’t necessarily doc-related – I’m not totally sure which ones are solved by docs and which ones are solved by code, the first step is just the pain point!
There’s also some related discussion in Sprockets abandonment, particularly around the technical constraints that are leading the projects to be maintained in parallel. Mind if I merge this thread under that one?
.
