@Betsy_Haibel Yes, you are correct. Thank you for clarifying.
This outline looks very good and promising. I would maybe think about adding section to people coming from pure webpack background explaining basic concepts of “webpacker way” in oppose to what they know. Especially the aspect of debugging webpack config webpacker provides by default. I still have no idea how to do this.
And possibly examples of setup and best practices for most common frameworks like react and vue.
While it is understandable that there are many scenarios (and wishes) in which webpacker can help and be enjoyed, it offers a compilation step to deployment that seems more obfuscated to me than what happens within sprockets (maybe that is just because sprockets wraps it all up nicely). Luckily, removing webpacker from a fresh Rails6 app is still a pretty easy thing to do and I absolutely wish that it stays so. You can build “perfect” Web applications using Rails+Sprockets and without much additional configuration.
I’ll add this, it is in the Webpacker docs.
My biggest WTF in Webpacker was the notion that in CSS, each file needs to declare its own dependencies, either using @use or @include, and if you @include a file more than once, you will end up with a complete copy of that file’s contents each time you do so. This really breaks my mind, especially coming from the Asset Pipeline world, which operates more (in my mind) like a traditional compiler. If you have an application.scss that rolls everything up, and you @include files one-by-one inside that meta-index, each included file will have access to everything that came before it in the index:
/* application.scss */ @include ‘_definitions.scss’; @include ‘bootstrap.scss’; @include ‘custom.scss’;
both bootstrap and custom can “see” the definitions, reference and extend them internally, without any further declarations. Maybe I have my telescope around the wrong way, but if so, it’s been that way for about ten or fourteen years. (How old is the asset pipeline concept?)
It strikes me as logical that this approach (granular includes, required over and over) is more suited to the notion of breaking up the final result into multiple “packs” based on the target, but this fights against what I’ve internalized in over twenty years of HTML/CSS – put everything in one minified file, include it once, and never reload it. That’s what Turbolinks is counting on, for a fact.
Anyway, to the larger question: Should Rails just use Webpack for all asset compilation? The answer is: sure, when Webpack can do it better than the asset pipeline, for the non-component based approach. I haven’t looked at Webpack 5. It’s possible that this deals with CSS and digested images and all the other Sprockets concerns better. If so, I’d love to see a bake-off. Show the whole pipeline working smoother, faster, with as little or less configuration, and with as good or better error messages.
Webpack uses too much memory and is super slow on big codebases: Webpack dev server takes up a lot of memory (751.9MB) for GitLab CE · Issue #4363 · webpack/webpack · GitHub (2018 it was using 1GB, this week I checked again and it’s now using 2.3GB)… this adds up on some workflows… for specific reasons, I usually run 3 different instances of that project on my machine, those adds up.
Funny enough, when we first migrated from sprockets the reason was sprockets peformance… if we could compare it with today’s number I would bet sprockets win for a large marging (using less memory and become ready first).
@DHH, @rafaelfranca, or someone else with historical context:
It’s a bit of a tangent from the main thread. But if we’re going to do an asset pipeline bakeoff, I would like to throw a Parcel-based implementation into the mix.
Issues I think the any “unified” asset pipeline would need to address, that Sprockets does currently address, and that AFAIK Webpacker does not:
- packaging assets with Rails Engines (ideally w/o a separate
- packaging assets with core Rails (ditto)
- user-friendly CSS packaging & SCSS packaging (i.e. no
I’ve had some success using Eyeglass for 3 in Webpacker.
1 and 2 seem like they’d require complicated npm-rubygems reconciliation procedures in a Webpacker-only world. (Or, for that matter, in a Parcel-based solution.) So getting to a unified asset pipeline seems like it’s not a smooth path at all. Probably still worth doing…? But complicated.
I did a survey at the time of Webpacker, and I do think we looked at Parcel. But I don’t recall why we didn’t go that route. The basic premise of Webpacker, though, is indeed that it’s zero config. Or close to it. Which is where it is today for a large range of default options (I think we’ve scarcely changed the defaults much in HEY).
Yeah, last we looked at it, there really wasn’t an easy way for us to get engine-bundled JS to work like it does with sprockets. The concept of a dynamic load path wasn’t there. But if there’s a new solution that’s emerged since, that would be awesome.
I’d be happy to see more work go into both the documentation, the default configuration, and the continuing to explore ways to make this serve even more cases. Webpacker has basically just been Gaurav, Javan, and a couple of other contributors. We could do well to see more people involved!
And, as mentioned above, I’m always game to consider a bake-off! I’d be surprised if Parcel offered such a monumental leap over Webpack that it’ll be worth the change at this point. But I love being surprised in such a way .
Looks like a great start! I left some comments on the gist. I’m interested in helping write the guide so let me know if and how you’d like to collaborate.
What happened to conceptual compression? We now make poor beginner Rails devs learn both Webpack and Sprockets, for very little gain imo. I get that Webpack is an imperfect solution but having both Webpack and Sprockets is imperfect as well! We should at least consider deprecating sprockets for upcoming Rails releases.
I’ll probably convert the gist into an actual repo and manage it that way.
While I haven’t set up a fresh CSS-included Webpacker project recently (I’ll do so and report back), I’ve set up a bunch over the past few years. My experience has tended to be that, while Webpack has pretty sensible defaults, as soon as I need to override those defaults I need to to dig in to both Webpacker’s implementation details and the general difficulties of the Webpack configuration DSL.
From the perspective of “is this good enough,” yeah, I think this is good enough. It’s tricky to create a truly non-leaky abstraction here, because Webpack’s config JSON has so little unified design philosophy of its own. And so “basically reasonable defaults, and a readable config file to change frequently-project-specific things like paths” is great to have.
From the perspective of “can this be better,” I think that basing holistic asset management off of a solution like Parcel that’s committed to zero-config setup might offer better opportunities for “conceptual compression” by reducing the number of concepts that need to be compressed. This in turn might make it easier to configure beyond the defaults without the abstraction leaking. But I 100% admit that this is speculation rooted in toy projects rather than something based on using Parcel in anger.
From a practical, immediate steps perspective: I think that I ought to set up that fresh CSS-included Webpacker-but-not-Sprockets project soon, document all of the things that annoy me about the process, and see if it’s worth upstreaming the extensions I usually make.
It seems some still value Sprockets for an easier out of the box experience if you don’t need a big JS framework. I hope the maintenance situation can be improved. If a project looks kind of dead, it’s not motivating to submit PRs, so there’s a chicken and egg problem in getting new maintainers, if that’s what needed.
.es6 made me stay with Sprockets 3 until I replaced it with my own fork using
rollup.js that keeps the old configuration style while supporting es6 and was able to remove all the directives that sprockets needed.
I still love Sprockets and hope the changes will happen.
A lot has been said already! I’ll add a few extra bits:
webpackbecomes the one and only solution at some point, people need to fully realise the incredible number of sub-dependencies this brings in (to fully grasp it, check out this visualisation). For apps with limited front-end needs, this increases the attack surface, some would say for little benefit. I wonder if Rollup (Visualization of npm dependencies) could be used more in those cases, as pointed out recently to me by @Ken_Collins1 (but if
webpackis mandatory, I guess this will be complicated)
yarnis to be used, but upgrading an old Rails app to use it, I found it hard to figure out which version of
yarnshould be used (1 or 2). I ended up using settling on v1 for now after reading the edge guides section for Rails contributor, which specifies the v1 repository for Ubuntu (Installation | Yarn).
Both these points made upgrade from old apps a confusing/concerning experience (for a security-concerned industry).
Also, as many have said, I love the speed of Sprockets, still. Webpack integration makes reloading a page slower at this point.
Webpack integration makes reloading a page slower at this point.
I’d be interested to hear if you have any profiling info to share (here are some debugging tips and some ideas for improving webpack performance). I haven’t observed the same slowness with webpack relative to Sprockets (if anything, Sprockets has been slower for me) so I’m curious to learn about others experiences with performance.
I do not have profiling info to share at this point, but if I get some, I will share it. Thanks for the links.