Our platform is divided on several smaller gems and engines. Some of the gems and engines have js files in them. This is not something that could be packed on npm. Those are not npm modules. Just some javascript “sprinkles” as they are referred to.
My question is How could we successfully include in our host app a js from another engine?
The whole process of including them and packing them with javascript_pack_tag “application.js” is frustrating and we still can not do it. I am on it for a couple of hours and here is my progress.
I had to figure out that when you add to application.js you can only add files located in app/javascript/packs. But our js files are located in the engines.
You can install webpacker erb and change the application.js.erb to do
ERROR in ./app/javascript/packs/application.js.erb
Module not found: Error: Can't resolve '/home/user/my_gem/app/assets/javascript/fc/main/custom.js' in '/home/user//app/javascript/packs'
@ ./app/javascript/packs/application.js.erb 20:0-100
I have tried different ways to change webpacker.yml
resolved_paths: ["fc-main/app/assets"]
but it is not clear what are these paths relative to and should they have app/assets, or app/assets/javascript or somethings else.
The webpacker readme does not have much information about this. It just tells you - you put the js in app/javascripts/pack. But I can’t since they are coming from an engine that is also used in other platforms.
I don’t have the full context of your application, but, generally speaking, any JS code can be packaged into modules. What makes it not work for you?
Importing JS using ERB to look up file paths looks like that might be painful. If it were my application, I might be looking to distribute my gem as two distinct distributions; Ruby from rubygems (or a private gem repository) and the JS as a package on NPM (or private NPM repository).
Thanks @rossta
This might be a long term solution, but is not viable for any existing code base. It might take months, we must set up an entirely different production for packing on npm, supporting a private npm repo and many more.
It also does not make sense to us. In a true rails way the js that we have is more like functionality here and there. These are “sprinkles”. It is not like a js app that works separated from the rails app and depends on a json return from it. So the js is for the html that was returned.
We also have the case where the “theme” (css, layout partials and js) are packed in a gem. This allows us to separated the theme from the app and we have one gem for the main layout and one gem for the admin layout.
So the gems come with the js,css,images for the layout.
If we build it as a ruby gem it is difficult to import the js.
If we build it as an npm, it is difficult to import the css.
So we are at the situation where we must split the css from the js of the same layout into two different package managers and production builds and this brings nothing. Just more dependencies and coupling.
Yes, I certainly understand. If the cost to migrating to another solution is too high, that might mean it makes more sense to just stick with Sprockets rather than try to workaround the packaging concerns.
There are certain “Rails way” expectations that don’t align with how things work in the JavaScript ecosystem, e.g., I don’t know that Webpacker should try to retro-fit all the functionality of asset gems when we can rely on NPM. That said, there may be smarter ways of doing this that haven’t materialized yet.
I didn’t understand this comment. With webpack/Webpacker, you can import CSS, images, fonts, along with JS from an NPM package and I don’t know of any inherent difficulties that makes the content-type an issue for inclusion in an NPM package. I would expect all your shared static assets could come from NPM package(s).
I didn’t understand this comment. With webpack/Webpacker, you can import CSS, images, fonts, along with JS from an NPM package and I don’t know of any inherent difficulties that makes the content-type an issue for inclusion in an NPM package. I would expect all your shared static assets could come from NPM package(s).
When you start a rails 6 app you get “sprockets for css, images” , webpack for “js”. That’s the default. These are the rails. So you should actually go outside of your way to do webpack for css or js for sprockets. It can certainly be done, but it will lead us on a strange path. It is not possible for an es6 js to have sprockets so you have to move entirely on npm for everything.
Assuming that rails 6 is really sprockets for css and webpack for js you are in a deadlock when you have something like a theme that contains both css and js. You should split the theme in two separated places - 1 gem and 1 npm package which makes the maintenance even more difficult.
I actually don’t have a big problem of moving away from rails and going to npm if this is where rails is heading for. There are no guides for webpacker for adding js to your rails project anywhere on https://guides.rubyonrails.org/. Does this mean that rails is abandoning js and every js that you would like to use in more than one project should be packaged in an npm package. Do you know anything about this?
“Sprockets for CSS and Webpacker for JS” is the Rails default because that’s what works for Basecamp. This would be a guess, but I don’t think the Rails defaults here have the use case you’re describing in mind.
I also wouldn’t say that it’s strange to go “all in” on Webpacker for assets or just stick with Sprockets for all assets; to me, one of these two options would be the most pragmatic for your setup rather than trying to split across CSS and JS. In all my applications, I use Webpacker for all assets because there are a ton of benefits; it generally makes more sense to me to use a single asset pipeline regardless of how Basecamp chooses what’s best for them.
As for guides, I agree that’s a current shortcoming. A group of us are helping write a Rails guide for Webpacker based on other discussions in this forum and the draft is in its early stages.
I recently hit the same issue. I found it strange that there are no backwards compatability. There are a lot of gems tailored to asset pipeline. Since now “Sprockets for CSS and Webpacker for JS” is the default, all those gems need to be updated to use npm and in some cases it doesn’t make sense.
Even with webpacker as a default, shipping JS inside gem has strong advantages in some cases:
Gem functionality is locked to specific JS version
JS is small, so it’s more hassle with publish it to npm
Better integration experience
Specific example: Gem message_bus ships with a little js library. It’s not Rails specific, but authors tailor gem to work with asset pipeline for a better experience. With webpacker in place, it’s a big wtf moment how to integrate with it. The only solution that I managed to get working is to copy js file to the app.
In this case pain of dealing with webpacker is much greater then the pain of copy pasting code and outdated dependency.
Just to keep the thread up to day of what I’ve done.
For js I’ve abandoned rails engines. I move all the js from the engines into the host app and I am working on moving them to npm packages. The frustration of having a js in an engine and trying to pack it with webpacker is just to much. I kind of managed to do it, but the amount of configuration was even difficult to keep track of.
My resolution is that if I need to share js between projects I am better of keeping the js in npm, like rails-ujs, like turbolinks for example. They don’t come as engines, but as npms. Much cleaner.
Hello! Sorry this is such a late response to this thread. I was able to do something pretty similar to what you described in your opening post. I’m not sure what exactly what’s different about our approaches, but hopefully this is helpful.
My frontend asset sources are in a non-standard directory. One note, I compile my assets before publishing my gem. For example, I vendor Rails UJS so that users wouldn’t need to worry about it, and I compile my PostCSS into CSS so that users don’t need to worry about installing plugins.
As I mentioned, I pre-compile my assets. My asset build script writes the results into Sprockets’ usual app/assets/ as well as a nonstandard dist directory for Webpacker. These two directories contain the exact same built assets (the directory tree is a little different, but the file contents are exactly the same!)
I have two demos (that I can’t link to since I can only post 5 links as a new user!)
In my project repo, I have a test app under test/dummy60_webpacker4
In my GitHub profile, I have a separate project called super_demo
This has worked for me in both Webpacker 4 and 5. I tested it both in CI about 20 days ago. In Webpacker 4, I received a warning about the ERB in an import, but I haven’t seen those when using Webpacker 5.
Hope this helps! I’ll try to answer some questions, but it’s been a very long while since I set this all up.