Sprockets abandonment

Agree with @masterleep, if Webpacker is the way the go we should have a guide that reflects that.

8 Likes

I also agree that if we’re committing to Webpacker, the guides need some significant love or we need more out of the box functionality around it. Part of the magic of Rails is that it just works - and the Webpacker integration, as it is, seems to require a non-trivial amount of effort to match the functionality that we’ve had with Sprockets.

15 Likes

My team found incorporating CSS ultimately just a matter of having the right boilerplate and directory structure, and turning the applicable configuration knobs. However, building our understanding was a real drag.

The real WTF for me was (and remains) that Webpacker lacks a guide, not rating even a single mention or footnote in the Asset Pipeline or Working with JavaScript in Rails guides despite being the only clear path forwards for both. The documentation in the Webpacker repository is not introductory material, it’s all definitely Hard Mode stuff.

Everything practical I know about Webpacker is therefore from trial & error, reading source code, and puts debuggering.

12 Likes

I think part of the problem with Webpacker and CSS is that Webpack itself doesn’t really treat CSS like a first-class citizen. I know that newer versions of Webpack are trying to catch up on this, but it seems like there’s a lot of technical debt in their way.

It sounds like this is a case where Rails, ironically, isn’t nearly opinionated enough, and that a combination of better documentation and a bit more guidance from the framework could really help folks out.

@inopinatus it sounds like your team’s CSS structure works really well for y’all – would you mind sharing more details of what your team does?

8 Likes

My team has also made a lot of strides towards using Webpack exclusively for assets. We’re in the middle of migrating assets from Sprockets to Webpacker. Webpack 5 was a big step up, surprisingly.

I actually really like Webpacker as a npm package because it’s everything you need in terms of modern JS tooling. I agree the docs need some work, but I don’t think it’s an impossible task.

I think there are two great points here:

  1. Sprockets maintenance is not happening.
  2. Rails including two tools where each can do everything the other can.

The first point is entirely mea culpa. I’m being the maintainer of Sprockets for years and I inherited the project after the two original authors left it. At the time I was still doing frontend work. After I moved companies I was getting further and further aways from frontend development. This caused my motivation to work on Sprockets to drop since I was not using it anymore in our apps. All that being said, Sprockets is another example of project that could use a motivate contributor and I’m happy to support that person on this.

To the second point, I can’t speak for DHH, but my feeling as a uses is that sprockets were better for CSS and images than Webpack. That could be familiarity bias since I’m more used with Sprockets than Webpack. But, with JavaScript files, Webpack is orders of magnitude better. Our intention were to provide the best for each kind of files. But we are lacking in documentation and guidance. For example, the assets pipeline guide has no mention to Webpacker. We need to fix it. This is another place where the community can help.

22 Likes

Ah yes, the great Sprockets to Webpacker migration! A few years ago, static-site generator Middleman dropped support for Sprockets in favor of external pipelines (eg. Webpacker), because it’s the way of the future!™ …which I can pretty much agree on. But god did it wreak havoc on my codebases and meant I lost the ability to easily configure asset compilation. Nowadays, in Middleman-land, I still tend to avoid any CSS compilation.

Pretty much any time I’ve been in contact with Webpack and it’s configuration, I’ve sunk a few hours into configuring, and after a long day of frustration, went “f— this. I’m going back.” Rinse, repeat, every few months/years. – And that, even if nowadays, I code more stuff with Vue/JS on the front-end than in Rails. – Not a happy one.

That’s pretty much my experience with the Javascript tooling in general. The configuration is horrible, there is boilerplate everywhere, you need to specify everything, and as soon as you go off the very mainstream path (eg. use Coffeescript), you’re on your own with unmaintained Github projects with half a star. Convention-over-configuration – a core tenet of Rails in my opinion, and one of its treasures – is not most JS tools’ forte. These experiences have not instilled good feelings in me.

With that said, I’m always hoping this year is the right one for ease of using Webpacker and other JS stuffs. The day I can throw at it a bunch of Sass, a bunch of Coffee, minimal configuration, and be done with it, I’ll be won over. If there’s one community who can push that forward, on the Rails side and on the Webpacker side, it’s this one.

6 Likes

@Betsy_Haibel Elements of our workflow include:

Strict use of modules and imports. No globals, not even a window.$ for legacy parts. We avoid ProvidePlugin and expose-loader. If it’s webpack’d, it’s written as modular application code for import/export.
HMR for development. It’s an amazing productivity enhancer. Some browser issues to begin with, that are now ironed out by time. We start the webpack dev server from a foreman Procfile.
Components include scoped CSS. We use a fair bit of Vue, with only a smattering of global CSS to tie it together. Most styles are in Vue components.
All under app/javascript/. If it’s built by Webpacker, we keep it under there (unless it’s from packages). The name isn’t perfect, but it’s established as a single place to look.
Diff reduction. Configs should vary from “Webpacker default” in ways that ease merging of future updates. E.g. no structural changes, block insertions only.

Within app/javascript we differ from Webpacker’s out-of-the-box samples and take an approach that feels more Rails-y, grouping source files by type and trying for connaissance of name wherever possible.

packs/:packname.js. Packs generally correspond to a layout in app/views/layouts to minimise proliferation and complexity.
styles/:packname.scss. Global styles for a given pack, which activates them by import styles/packname.scss in the js.
images/:packname/:image.png. Graphics assets for a given pack. Resolved as nonrelative paths e.g. background-image: url(~images/admin/brushed4.png), which writes the correct URLs for our CDN.
components/:component-name.vue. Vue components, namespaced by classification (not responsibility) when a flat directory gets unwieldy.

We’ve noted caching, FOUT, and compatibility issues with CSS-in-JS, so in production it’s extract_css: true, precompiling all assets to CDN in the deploy pipeline, and referenced via stylesheet_pack_tag and javascript_pack_tag separately.

I reckon the only thing missing from Webpacker’s capabilities is a good story for engine assets. Webpack (and it’s ecosystem) is a powerful enabler, and I really would not willingly return to Sprockets today. However (with a nod to those of us who follow DHH’s other career), it’s like we’ve been given a LMP to drive, but the assembly instructions for a bicycle.

All this said, I would not be at all surprised if other folks had very different perspectives and practices. I can’t claim any of the above is a “best practice”, it’s all just a comfortable local maxima, discovered for ourselves.

2 Likes

I agree that an up to date Rails Guide on Webpacker would be a big help. Even if it just duplicated the Webpacker readme, it’d be easier to find in Rails Guides…

Also, at the risk of sounding self-promotional, and feel free to ignore, Modern Front End Development For Rails goes pretty in-depth on Webpacker. I haven’t caught up to 5.0 yet, though I will. I’ve even got a promo code that was meant for RailsConf, but will do fine for A May of WTFs: RailsConf_CouchEdition_2020.

7 Likes

A vote from Twitter:

1 Like

Sassc gives me a segfault when I try to use it (segfault on 2.3.0 · Issue #197 · sass/sassc-ruby · GitHub) so switching completely to webpacker has been very helpful.

1 Like

I just wanna put this out there too, as its kinda related – the story for engines + webpacker is not great!

9 Likes

As @Betsy_Haibel said, CSS doesn’t feel like a first-class citizen in Webpack, and as a Rails old-timer it’s hard to let go of how nice/easy/familiar it is to put my CSS under assets and see it just work. But I can see how a beginner could be confused about the split.

I suppose the dream, “unification” solution would be to have Webpacker do what Sprockets does for things in the assets directory out of the box. Personally I’d like to keep my stylesheets under assets, it bothers me at some core level to put CSS under a directory named “javascripts” :slight_smile:

2 Likes

Even though Sprockets stale and there is a lot of room for improvement to be done and I miss the simplicity of it.

I’m not the biggest fan of Webpacker, I found it unintuitive and didn’t like the extra config files it brought last time I used it.

Over the years I’ve worked on an overhaul for Sprockets, mainly adding native Javascript module support and simplifying the pipeline to reduce the time to see if a file can be transpiled to the requested content type.

Your welcome to give it a go, it’s called Condenser:

It’s been a while since I’ve dropped sprockets but from what I recall it was relatively simple to swap out.

One caveat is the javascript globals can’t work across imports. While migrating I’ve go around this but assigning the global I want the import to add to window. Might not be the prettiest buts lets you start migrating without a big rewrite.

Can’t say it’s bug free, but a couple of us have been using it over the years on various projects, and I’d love to get more input!

3 Likes

I definitely agree that Webpacker should handle CSS by default! That’s one of the biggest annoyances I have with default Rails apps right now.

I have all my JavaScript (well, TypeScript) and SCSS loading through Webpacker in my Rails 6 app. I also load SVG icons through Webpacker.

This is what my javascript directory looks like (simplified a bit to make the example shorter):

app
├── javascript
│   ├── icons
│   │   ├── balance-scale-left.svg
│   │   ├── chevron-down.svg
│   │   ├── clipboard.svg
│   │   ├── cog.svg
│   │   └── user.svg
│   ├── packs
│   │   └── application.ts
│   └── src
│       ├── application.scss
│       ├── bulma.ts
│       ├── components
│       │   ├── add-game-to-library.vue
│       │   ├── avatar-input.vue
│       │   ├── compare-libraries.vue
│       │   ├── fields
│       │   │   ├── date-field.vue
│       │   │   ├── file-select.vue
│       │   │   ├── multi-select-generic.vue
│       │   │   └── text-field.vue
│       │   ├── game-card-dropdown.vue
│       │   ├── game-form.vue
│       │   ├── search.vue
│       │   └── user-statistics.vue
│       ├── settings.ts
│       ├── stylesheets
│       │   ├── bulma-variables.scss
│       │   ├── home.scss
│       │   ├── padding-and-margins.scss
│       │   └── utilities.scss
│       ├── toggleable-buttons.ts
│       ├── turbolinks-adapter.ts
│       ├── utils.ts
│       ├── vue-loader.ts
│       └── vue-shims.d.ts

Or if we want to simplify it down to just the relevant SCSS parts:

app
└── javascript
    ├── packs
    │   └── application.ts
    └── src
        ├── application.scss
        └── stylesheets
            ├── bulma-variables.scss
            ├── home.scss
            ├── padding-and-margins.scss
            └── utilities.scss

The app is open source so you can see it here if you want.

The main thing I needed to make this work is this, in config/webpack/environment.js:

const { environment } = require('@rails/webpacker');

// resolve-url-loader must be used before sass-loader
environment.loaders.get('sass').use.splice(-1, 0, {
  loader: 'resolve-url-loader'
});

module.exports = environment;

Also needs the resolve-url-loader package in package.json.

Unfortunately, in order to concatenate the files together into one CSS file I need to import each file in the application.scss, like this:

@import './stylesheets/bulma-variables.scss';
@import '~bulma/bulma.sass'; /* Load the bulma library from yarn */
@import './stylesheets/padding-and-margins.scss';
@import './stylesheets/utilities.scss';
@import './stylesheets/home.scss';

Then in my application.ts, though it should also work fine in application.js (this is necessary, otherwise Webpacker won’t be able to find application.css, I’m sure there’s a better way to solve this problem but I’m not aware of it):

import '../src/application.scss';

Then in my <head> I have this:

<%= stylesheet_pack_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>`

And then I think you need to set extract_css to true in config/webpacker.yml.

And that should be everything necessary to get SCSS working with Webpacker :slight_smile:

I’ve been using Rails since version 2.something and javascript in the Webpacker era has been the biggest nightmare in upgrading I can recall. I’ve got a codebase that’s been around since that time and has survived every major rails migration without too much trouble but that means I’ve got jquery-rails lying around, for example, and generally have some JS-through-gems stuff that was best practise at the time.

I’m at a point with it where I think it might be easier to literally re-implement every scrap of JS on the site than it is to try and migrate what I’ve got across. I’m not a 100% front-end guy so dipping my toe into this world once every 6 months or so means it’s seemingly impossible to keep up with how stuff is meant to work.

I’ll give this thread another close read on Monday with the code in front of me and see I I can’t glean some wisdom but I just wanted to add my voice to this - the migration to Webpacker, especially for existing long-standing code bases, has been disappointingly opaque for a Rails migration. And I’ve been through most of them.

9 Likes

I recently migrated some of my personal projects to webpacker. It wasn’t too bad as I’ve been using webpack in some other form somewhere else.

The main problem for me is processing erb files through webpacker is way slower: it involves booting up separate rails app for each erb files. It’s probably not too bad when using spring but I have so little faith in it it’s the first thing I disable (also my development platform is neither macOS nor Linux so it seems to be usually less well supported). And that means each erb files adds over 5 seconds to the compilation time which is way longer than I’d ever like.

In the end I went back to sprockets for stuff requiring erb. It serves a nice bridge between webpack and rails.

I’m really excited to see the discussion in this topic. I think education is the big missing piece here. Webpacker and webpack is not just a drop-in replacement for Sprockets: it’s a totally different paradigm of working with assets.

I’ve invested a lot of time learning about Webpacker and webpack over the past few years having upgraded a number of apps from Sprockets to Webpacker for JavaScript, CSS, etc. The path was not an easy one.

For Rails developers who’ve only worked with Sprockets prior to Webpacker, there are new JavaScript and webpack-specific concepts to learn. Just some examples:

  • how does import / require() work?
  • what are loaders and plugins in webpack?
  • why doesn’t webpack require all my images by default?
  • why do I have to import CSS from my JS?
  • omg what even is any of this webpack configuration?

That’s not to mention all the Webpacker concepts that are layered on top

  • Why do I need a gem and an NPM package?
  • What’s this webpacker.yml thing?
  • What’s so special about this app/javascript/packs directory?
  • How do I customize the default Webpacker config?
  • Why are these devDependencies not found when I compile for production?

Then there are gotchas for folks trying to upgrade from Sprockets to Webpacker that stem from the assumption things will work similarly such as dealing with jQuery plugins that rely on a global jQuery variable or expecting the webpack will share code across bundles if you use multiple packs (it doesn’t by default).

It’s clear there are big hurdles to adopting Webpacker both for experienced Rails developers and newbies alike. This kind of thing leads to frustration and eventually giving up on the platform. I’d hate to see that happen.

I’ve tried to help by capturing some of this knowledge in blog posts like How we switched from Sprockets to webpack and a conference talk, A Webpack Survival Guide for Rails Developers. I also recommend this RailsConf 2020 Couch Edition talk by Justin Gordon Webpacker, It-Just-Works, But How?.

That said, I agree that a formal Rails guide could go a long way to help get developers started without having to comb through Webpacker readme docs or random GH issues. I’d love to help write them. I’ll bet others with hard-earned experience will have much to offer as well. Who’s interested?

9 Likes

I believe that leaving Sprockets behind in favour of Webpacker is inevitable in the end but there is one are where it still cannot replace Sprockets completely. It is development of gems and engines with assets. In Sprockets land including engine assets was dead simple. You just added them to the app manifest and off you go.

Try doing this clearly in webpacker. You just can’t. I understand most of it is caused by problems with npm/yarn and webpack files path resolution itself but without it this switch will never happen and it leaves engine authors with more and more outdated Sprockets. The only alternative is developing and delivering frontend part of engine as separate npm package. It’s doable but definitely not easy to do and work with.

I know there is a guide about engines in Webpacker repo here but:

  • For me following it to migrate engine to Webpacker failed more times then it succeeded
  • Idea of launching second instance of webpacker just to compile engine assets is more dirty hack than a solution.

In general if we are to leave Sprocket some day the Webpacker needs to be better, a lot better, and I almost stopped believing it will ever be as good and as reliable as Sprockets used to be.

10 Likes

Noticing a lot of crossover between this and App/javascript feels like the wrong name for that directory.

1 Like