@Betsy_Haibel Yes, you are correct. Thank you for clarifying.
This outline looks very good and promising. I would maybe think about adding section to people coming from pure webpack background explaining basic concepts of “webpacker way” in oppose to what they know. Especially the aspect of debugging webpack config webpacker provides by default. I still have no idea how to do this.
And possibly examples of setup and best practices for most common frameworks like react and vue.
While it is understandable that there are many scenarios (and wishes) in which webpacker can help and be enjoyed, it offers a compilation step to deployment that seems more obfuscated to me than what happens within sprockets (maybe that is just because sprockets wraps it all up nicely). Luckily, removing webpacker from a fresh Rails6 app is still a pretty easy thing to do and I absolutely wish that it stays so. You can build “perfect” Web applications using Rails+Sprockets and without much additional configuration.
It seems to me that this is again also a question of scale. Once you want to grow your frontend code in complexity, webpacker will add some configuration and concepts for you so that you actually have less to worry about. But if you live off 5 or 10 nice CSS files and a handful or dozen of proper Javascript snippets or microlibs, I do not see why I should have to deal with that added complexity. To me sprockets is the "just works"™ approach for smallerish (frontend-side) projects while webpacker is the “big gun, will make it work”.
5 Likes
I’ll add this, it is in the Webpacker docs.
1 Like
My biggest WTF in Webpacker was the notion that in CSS, each file needs to declare its own dependencies, either using @use or @include, and if you @include a file more than once, you will end up with a complete copy of that file’s contents each time you do so. This really breaks my mind, especially coming from the Asset Pipeline world, which operates more (in my mind) like a traditional compiler. If you have an application.scss that rolls everything up, and you @include files one-by-one inside that meta-index, each included file will have access to everything that came before it in the index:
/* application.scss */ @include ‘_definitions.scss’; @include ‘bootstrap.scss’; @include ‘custom.scss’;
both bootstrap and custom can “see” the definitions, reference and extend them internally, without any further declarations. Maybe I have my telescope around the wrong way, but if so, it’s been that way for about ten or fourteen years. (How old is the asset pipeline concept?)
It strikes me as logical that this approach (granular includes, required over and over) is more suited to the notion of breaking up the final result into multiple “packs” based on the target, but this fights against what I’ve internalized in over twenty years of HTML/CSS – put everything in one minified file, include it once, and never reload it. That’s what Turbolinks is counting on, for a fact.
Walter
1 Like
Just for a bit of history here. When we adopted Webpack as the preferred solution to compiling modern JavaScript in Rails, it was because Webpack offered a superior solution to Sprockets when it came to JavaScript compilation. But none of those advantages were present – at least at the time! – when it came to stylesheets or images or any other static assets. The asset pipeline required less configuration, was faster, and was easier to overall integrate.
Now I appreciate that some of this perhaps comes from the fact that we don’t build SPAs at Basecamp? Maybe there are some component scenarios where people are bundling images and stylesheets together with their JavaScript work, and in those cases, it would be easier to use Webpack directly. As far as I recall, we’ve made provisions for this already. But I could totally see if it’s not very well explained.
Anyway, to the larger question: Should Rails just use Webpack for all asset compilation? The answer is: sure, when Webpack can do it better than the asset pipeline, for the non-component based approach. I haven’t looked at Webpack 5. It’s possible that this deals with CSS and digested images and all the other Sprockets concerns better. If so, I’d love to see a bake-off. Show the whole pipeline working smoother, faster, with as little or less configuration, and with as good or better error messages.
But before we even consider expanding Webpacks role in the Rails asset compilation pipeline, I think we’d do well to document, clean-up, and polish its role as the preferred JavaScript compiler and packager first.
21 Likes
Webpack uses too much memory and is super slow on big codebases: Webpack dev server takes up a lot of memory (751.9MB) for GitLab CE · Issue #4363 · webpack/webpack · GitHub (2018 it was using 1GB, this week I checked again and it’s now using 2.3GB)… this adds up on some workflows… for specific reasons, I usually run 3 different instances of that project on my machine, those adds up.
Funny enough, when we first migrated from sprockets the reason was sprockets peformance… if we could compare it with today’s number I would bet sprockets win for a large marging (using less memory and become ready first).
2 Likes
@DHH, @rafaelfranca, or someone else with historical context:
Was Parcel ever explored as an asset pipeline solution? While I’ve never used it on a large project (which is a huge caveat), its commitment to zero-config opinionated compilation and its first-class treatment of non-Javascript assets has always struck me as “Railsier” than Webpack’s approach.
It’s a bit of a tangent from the main thread. But if we’re going to do an asset pipeline bakeoff, I would like to throw a Parcel-based implementation into the mix.
6 Likes
Issues I think the any “unified” asset pipeline would need to address, that Sprockets does currently address, and that AFAIK Webpacker does not:
- packaging assets with Rails Engines (ideally w/o a separate
yarn install) - packaging assets with core Rails (ditto)
- user-friendly CSS packaging & SCSS packaging (i.e. no
@import '~/node_modules/package_name/extremely/long/sub/pathnastiness)
I’ve had some success using Eyeglass for 3 in Webpacker.
1 and 2 seem like they’d require complicated npm-rubygems reconciliation procedures in a Webpacker-only world. (Or, for that matter, in a Parcel-based solution.) So getting to a unified asset pipeline seems like it’s not a smooth path at all. Probably still worth doing…? But complicated.
2 Likes
I did a survey at the time of Webpacker, and I do think we looked at Parcel. But I don’t recall why we didn’t go that route. The basic premise of Webpacker, though, is indeed that it’s zero config. Or close to it. Which is where it is today for a large range of default options (I think we’ve scarcely changed the defaults much in HEY).
Yeah, last we looked at it, there really wasn’t an easy way for us to get engine-bundled JS to work like it does with sprockets. The concept of a dynamic load path wasn’t there. But if there’s a new solution that’s emerged since, that would be awesome.
I think the bottomline is that JavaScript packaging is a… complicated question. Webpacker is the current best effort at providing some great defaults that’ll work for a lot of Rails apps out of the box. But it surely won’t address all setups.
I’d be happy to see more work go into both the documentation, the default configuration, and the continuing to explore ways to make this serve even more cases. Webpacker has basically just been Gaurav, Javan, and a couple of other contributors. We could do well to see more people involved!
And, as mentioned above, I’m always game to consider a bake-off! I’d be surprised if Parcel offered such a monumental leap over Webpack that it’ll be worth the change at this point. But I love being surprised in such a way 
.
3 Likes
Looks like a great start! I left some comments on the gist. I’m interested in helping write the guide so let me know if and how you’d like to collaborate.
What happened to conceptual compression? We now make poor beginner Rails devs learn both Webpack and Sprockets, for very little gain imo. I get that Webpack is an imperfect solution but having both Webpack and Sprockets is imperfect as well! We should at least consider deprecating sprockets for upcoming Rails releases.
I’ll probably convert the gist into an actual repo and manage it that way.
While I haven’t set up a fresh CSS-included Webpacker project recently (I’ll do so and report back), I’ve set up a bunch over the past few years. My experience has tended to be that, while Webpack has pretty sensible defaults, as soon as I need to override those defaults I need to to dig in to both Webpacker’s implementation details and the general difficulties of the Webpack configuration DSL.
From the perspective of “is this good enough,” yeah, I think this is good enough. It’s tricky to create a truly non-leaky abstraction here, because Webpack’s config JSON has so little unified design philosophy of its own. And so “basically reasonable defaults, and a readable config file to change frequently-project-specific things like paths” is great to have.
From the perspective of “can this be better,” I think that basing holistic asset management off of a solution like Parcel that’s committed to zero-config setup might offer better opportunities for “conceptual compression” by reducing the number of concepts that need to be compressed. This in turn might make it easier to configure beyond the defaults without the abstraction leaking. But I 100% admit that this is speculation rooted in toy projects rather than something based on using Parcel in anger.
From a practical, immediate steps perspective: I think that I ought to set up that fresh CSS-included Webpacker-but-not-Sprockets project soon, document all of the things that annoy me about the process, and see if it’s worth upstreaming the extensions I usually make.
5 Likes
It seems some still value Sprockets for an easier out of the box experience if you don’t need a big JS framework. I hope the maintenance situation can be improved. If a project looks kind of dead, it’s not motivating to submit PRs, so there’s a chicken and egg problem in getting new maintainers, if that’s what needed.
3 Likes
I think the changes to Sprockets needed to support Javascript modules would be too dramatic to go upstream. v4 was in beta for many years, and it’s quite complicated system from what I remember when I was working with it.
I also really dislike the current direction. Adding yet another javascript configuration file, es6 files needing to end in .es6 made me stay with Sprockets 3 until I replaced it with my own fork using rollup.js that keeps the old configuration style while supporting es6 and was able to remove all the directives that sprockets needed.
I still love Sprockets and hope the changes will happen. 
1 Like
A lot has been said already! I’ll add a few extra bits:
- If
webpackbecomes the one and only solution at some point, people need to fully realise the incredible number of sub-dependencies this brings in (to fully grasp it, check out this visualisation). For apps with limited front-end needs, this increases the attack surface, some would say for little benefit. I wonder if Rollup (Visualization of npm dependencies) could be used more in those cases, as pointed out recently to me by @Ken_Collins1 (but ifwebpackis mandatory, I guess this will be complicated) -
yarnis to be used, but upgrading an old Rails app to use it, I found it hard to figure out which version ofyarnshould be used (1 or 2). I ended up using settling on v1 for now after reading the edge guides section for Rails contributor, which specifies the v1 repository for Ubuntu (Installation | Yarn).
Both these points made upgrade from old apps a confusing/concerning experience (for a security-concerned industry).
Also, as many have said, I love the speed of Sprockets, still. Webpack integration makes reloading a page slower at this point.
9 Likes
Webpack integration makes reloading a page slower at this point.
I’d be interested to hear if you have any profiling info to share (here are some debugging tips and some ideas for improving webpack performance). I haven’t observed the same slowness with webpack relative to Sprockets (if anything, Sprockets has been slower for me) so I’m curious to learn about others experiences with performance.
1 Like
I do not have profiling info to share at this point, but if I get some, I will share it. Thanks for the links.
I use to import third party themes from my project and using Sprockets everything used to work smoothly but now it’s rather difficult. Furthermore I have to change a lot in the javascript of the theme to use webpacker instead with sprockets I didn’t need to change almost anything. I’d love if both Sprockets and Webpacker could coexist. Thank you.
1 Like
