I just wanna put this out there too, as its kinda related – the story for engines + webpacker is not great!
10 Likes
As @Betsy_Haibel said, CSS doesn’t feel like a first-class citizen in Webpack, and as a Rails old-timer it’s hard to let go of how nice/easy/familiar it is to put my CSS under assets and see it just work. But I can see how a beginner could be confused about the split.
I suppose the dream, “unification” solution would be to have Webpacker do what Sprockets does for things in the assets directory out of the box. Personally I’d like to keep my stylesheets under assets, it bothers me at some core level to put CSS under a directory named “javascripts” 
2 Likes
Even though Sprockets stale and there is a lot of room for improvement to be done and I miss the simplicity of it.
I’m not the biggest fan of Webpacker, I found it unintuitive and didn’t like the extra config files it brought last time I used it.
Over the years I’ve worked on an overhaul for Sprockets, mainly adding native Javascript module support and simplifying the pipeline to reduce the time to see if a file can be transpiled to the requested content type.
Your welcome to give it a go, it’s called Condenser:
It’s been a while since I’ve dropped sprockets but from what I recall it was relatively simple to swap out.
One caveat is the javascript globals can’t work across imports. While migrating I’ve go around this but assigning the global I want the import to add to window. Might not be the prettiest buts lets you start migrating without a big rewrite.
Can’t say it’s bug free, but a couple of us have been using it over the years on various projects, and I’d love to get more input!
http://github.com/malomalo/condenser http://github.com/malomalo/condenser-rails https://github.com/malomalo/middleman-condenser
3 Likes
I definitely agree that Webpacker should handle CSS by default! That’s one of the biggest annoyances I have with default Rails apps right now.
I have all my JavaScript (well, TypeScript) and SCSS loading through Webpacker in my Rails 6 app. I also load SVG icons through Webpacker.
This is what my javascript directory looks like (simplified a bit to make the example shorter):
app
├── javascript
│ ├── icons
│ │ ├── balance-scale-left.svg
│ │ ├── chevron-down.svg
│ │ ├── clipboard.svg
│ │ ├── cog.svg
│ │ └── user.svg
│ ├── packs
│ │ └── application.ts
│ └── src
│ ├── application.scss
│ ├── bulma.ts
│ ├── components
│ │ ├── add-game-to-library.vue
│ │ ├── avatar-input.vue
│ │ ├── compare-libraries.vue
│ │ ├── fields
│ │ │ ├── date-field.vue
│ │ │ ├── file-select.vue
│ │ │ ├── multi-select-generic.vue
│ │ │ └── text-field.vue
│ │ ├── game-card-dropdown.vue
│ │ ├── game-form.vue
│ │ ├── search.vue
│ │ └── user-statistics.vue
│ ├── settings.ts
│ ├── stylesheets
│ │ ├── bulma-variables.scss
│ │ ├── home.scss
│ │ ├── padding-and-margins.scss
│ │ └── utilities.scss
│ ├── toggleable-buttons.ts
│ ├── turbolinks-adapter.ts
│ ├── utils.ts
│ ├── vue-loader.ts
│ └── vue-shims.d.ts
Or if we want to simplify it down to just the relevant SCSS parts:
app
└── javascript
├── packs
│ └── application.ts
└── src
├── application.scss
└── stylesheets
├── bulma-variables.scss
├── home.scss
├── padding-and-margins.scss
└── utilities.scss
The app is open source so you can see it here if you want.
The main thing I needed to make this work is this, in config/webpack/environment.js:
const { environment } = require('@rails/webpacker');
// resolve-url-loader must be used before sass-loader
environment.loaders.get('sass').use.splice(-1, 0, {
loader: 'resolve-url-loader'
});
module.exports = environment;
Also needs the resolve-url-loader package in package.json.
Unfortunately, in order to concatenate the files together into one CSS file I need to import each file in the application.scss, like this:
@import './stylesheets/bulma-variables.scss';
@import '~bulma/bulma.sass'; /* Load the bulma library from yarn */
@import './stylesheets/padding-and-margins.scss';
@import './stylesheets/utilities.scss';
@import './stylesheets/home.scss';
Then in my application.ts, though it should also work fine in application.js (this is necessary, otherwise Webpacker won’t be able to find application.css, I’m sure there’s a better way to solve this problem but I’m not aware of it):
import '../src/application.scss';
Then in my <head> I have this:
<%= stylesheet_pack_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>`
And then I think you need to set extract_css to true in config/webpacker.yml.
And that should be everything necessary to get SCSS working with Webpacker
I’ve been using Rails since version 2.something and javascript in the Webpacker era has been the biggest nightmare in upgrading I can recall. I’ve got a codebase that’s been around since that time and has survived every major rails migration without too much trouble but that means I’ve got jquery-rails lying around, for example, and generally have some JS-through-gems stuff that was best practise at the time.
I’m at a point with it where I think it might be easier to literally re-implement every scrap of JS on the site than it is to try and migrate what I’ve got across. I’m not a 100% front-end guy so dipping my toe into this world once every 6 months or so means it’s seemingly impossible to keep up with how stuff is meant to work.
I’ll give this thread another close read on Monday with the code in front of me and see I I can’t glean some wisdom but I just wanted to add my voice to this - the migration to Webpacker, especially for existing long-standing code bases, has been disappointingly opaque for a Rails migration. And I’ve been through most of them.
9 Likes
I recently migrated some of my personal projects to webpacker. It wasn’t too bad as I’ve been using webpack in some other form somewhere else.
The main problem for me is processing erb files through webpacker is way slower: it involves booting up separate rails app for each erb files. It’s probably not too bad when using spring but I have so little faith in it it’s the first thing I disable (also my development platform is neither macOS nor Linux so it seems to be usually less well supported). And that means each erb files adds over 5 seconds to the compilation time which is way longer than I’d ever like.
In the end I went back to sprockets for stuff requiring erb. It serves a nice bridge between webpack and rails.
I’m really excited to see the discussion in this topic. I think education is the big missing piece here. Webpacker and webpack is not just a drop-in replacement for Sprockets: it’s a totally different paradigm of working with assets.
I’ve invested a lot of time learning about Webpacker and webpack over the past few years having upgraded a number of apps from Sprockets to Webpacker for JavaScript, CSS, etc. The path was not an easy one.
For Rails developers who’ve only worked with Sprockets prior to Webpacker, there are new JavaScript and webpack-specific concepts to learn. Just some examples:
- how does import / require() work?
- what are loaders and plugins in webpack?
- why doesn’t webpack require all my images by default?
- why do I have to import CSS from my JS?
- omg what even is any of this webpack configuration?
That’s not to mention all the Webpacker concepts that are layered on top
- Why do I need a gem and an NPM package?
- What’s this webpacker.yml thing?
- What’s so special about this app/javascript/packs directory?
- How do I customize the default Webpacker config?
- Why are these devDependencies not found when I compile for production?
Then there are gotchas for folks trying to upgrade from Sprockets to Webpacker that stem from the assumption things will work similarly such as dealing with jQuery plugins that rely on a global jQuery variable or expecting the webpack will share code across bundles if you use multiple packs (it doesn’t by default).
It’s clear there are big hurdles to adopting Webpacker both for experienced Rails developers and newbies alike. This kind of thing leads to frustration and eventually giving up on the platform. I’d hate to see that happen.
I’ve tried to help by capturing some of this knowledge in blog posts like How we switched from Sprockets to webpack and a conference talk, A Webpack Survival Guide for Rails Developers. I also recommend this RailsConf 2020 Couch Edition talk by Justin Gordon Webpacker, It-Just-Works, But How?.
That said, I agree that a formal Rails guide could go a long way to help get developers started without having to comb through Webpacker readme docs or random GH issues. I’d love to help write them. I’ll bet others with hard-earned experience will have much to offer as well. Who’s interested?
7 Likes
I believe that leaving Sprockets behind in favour of Webpacker is inevitable in the end but there is one are where it still cannot replace Sprockets completely. It is development of gems and engines with assets. In Sprockets land including engine assets was dead simple. You just added them to the app manifest and off you go.
Try doing this clearly in webpacker. You just can’t. I understand most of it is caused by problems with npm/yarn and webpack files path resolution itself but without it this switch will never happen and it leaves engine authors with more and more outdated Sprockets. The only alternative is developing and delivering frontend part of engine as separate npm package. It’s doable but definitely not easy to do and work with.
I know there is a guide about engines in Webpacker repo here but:
- For me following it to migrate engine to Webpacker failed more times then it succeeded
- Idea of launching second instance of webpacker just to compile engine assets is more dirty hack than a solution.
In general if we are to leave Sprocket some day the Webpacker needs to be better, a lot better, and I almost stopped believing it will ever be as good and as reliable as Sprockets used to be.
9 Likes
Noticing a lot of crossover between this and App/javascript feels like the wrong name for that directory - #17 by Betsy_Haibel.
1 Like
I think this would be a good candidate for its own WTF topic so it doesn’t get buried in this one.
@Betsy_Haibel, the lack of any official Rails Guides chapter about Webpacker is an emerging candidate for Frequently WTFed WTFs.
Hey, I’m going to post an outline for a potential Rails Guide for Webpacker tomorrow, and we can start filling in the bits. The readme and wiki actually have a lot of the information, just not well structured.
4 Likes
Okay, here’s a draft outline based on the structure of the Asset Pipeline guide and also the docs in the Webpacker repo webpacker_guide.md · GitHub.
Does this seem reasonable? Would the core team accept this as a guide?
6 Likes
I think webpacker with rails engines should also be considered in guides
6 Likes
@the-spectator Clarifying – do you mean you’d like to see a guide on how to use Webpacker within a Rails Engine?
@Betsy_Haibel Yes, you are correct. Thank you for clarifying.
This outline looks very good and promising. I would maybe think about adding section to people coming from pure webpack background explaining basic concepts of “webpacker way” in oppose to what they know. Especially the aspect of debugging webpack config webpacker provides by default. I still have no idea how to do this.
And possibly examples of setup and best practices for most common frameworks like react and vue.
While it is understandable that there are many scenarios (and wishes) in which webpacker can help and be enjoyed, it offers a compilation step to deployment that seems more obfuscated to me than what happens within sprockets (maybe that is just because sprockets wraps it all up nicely). Luckily, removing webpacker from a fresh Rails6 app is still a pretty easy thing to do and I absolutely wish that it stays so. You can build “perfect” Web applications using Rails+Sprockets and without much additional configuration.
It seems to me that this is again also a question of scale. Once you want to grow your frontend code in complexity, webpacker will add some configuration and concepts for you so that you actually have less to worry about. But if you live off 5 or 10 nice CSS files and a handful or dozen of proper Javascript snippets or microlibs, I do not see why I should have to deal with that added complexity. To me sprockets is the "just works"™ approach for smallerish (frontend-side) projects while webpacker is the “big gun, will make it work”.
5 Likes
I’ll add this, it is in the Webpacker docs.
1 Like
My biggest WTF in Webpacker was the notion that in CSS, each file needs to declare its own dependencies, either using @use or @include, and if you @include a file more than once, you will end up with a complete copy of that file’s contents each time you do so. This really breaks my mind, especially coming from the Asset Pipeline world, which operates more (in my mind) like a traditional compiler. If you have an application.scss that rolls everything up, and you @include files one-by-one inside that meta-index, each included file will have access to everything that came before it in the index:
/* application.scss */ @include ‘_definitions.scss’; @include ‘bootstrap.scss’; @include ‘custom.scss’;
both bootstrap and custom can “see” the definitions, reference and extend them internally, without any further declarations. Maybe I have my telescope around the wrong way, but if so, it’s been that way for about ten or fourteen years. (How old is the asset pipeline concept?)
It strikes me as logical that this approach (granular includes, required over and over) is more suited to the notion of breaking up the final result into multiple “packs” based on the target, but this fights against what I’ve internalized in over twenty years of HTML/CSS – put everything in one minified file, include it once, and never reload it. That’s what Turbolinks is counting on, for a fact.
Walter
1 Like