Expanding Rails scope into the Cloud?

[Lucifer]

Why is there “no rail” to guide me how to setup Data Pipelines, Warehouses/Lakes, etc.?

Rails seems to have a convention and rule for everything, from front-end asset pipelines to Models and SQL schemas. But once I’m in the cloud I’m on my own without any structure or conventions whatsoever.

Does the Rails community plan to expand the scope of Rails into modern cloud products?

[sdubois (Sébastien Dubois)]

This discussion might be relevant:

Rails deployments

My first message here, and before anything just to thank all the people that makes Rails possible. I’m in love with it. Still learning it. My main “point of improvement” would be the deployment process. Having done a few things in PHP in which you basically copy files, and set a config in place for the database, the deployment process in rails got me really frustrated to the point to almost abandoning the project the first time. I do know there things easier that other when it comes to changes…

[Lucifer]

I don’t mean deployments, I mean for example I’m logging Events in my Rails app that will be saved in a Data Warehouse like Amazon Redshift. Although there are known patterns for that (e.g. denormalization), Rails stops guiding me at that point. Why is the scope of Rails not extended to guide me when using modern Cloud Services?

Rails also doesn’t offer any guidance for Key/Value storage solutions (not even Redis), and I feel those are commonly used things.

There are many more examples. I don’t get where Rails draws the line and why.

It feels awkward when developing Rails and suddenly I’m left on my own without any pattern/convention/guidance on how to add my Redis or e.g. Kafka implementation.

It seems to me that there ARE known patterns for pretty much everything, be it data pipelines, machine learning, data streaming, etc etc. Rails is good at taking those patterns and building conventions around it, but why do you stop at SQL databases?

Maybe some of those things can’t be written in Ruby, but Rails is not just about Ruby anyway. For example there is a strong convention and asset pipeline about JavaScript. I think it would be awesome if Rails had guidance for working with modern Apache Products (Spark/Beam/Kafka/etc.) and Cloud Solutions.

[pinzonjulian (Julián Pinzón Eslava)]

I believe DHH mentioned something this week about the redis storage problem you mentioned. Rails 7 (the next version) will hace this by default.

[ruurd (Ruurd Pels)]

Because it is extremely hard to identify where the added value of a framework and a way of thinking is not large enough anymore to warrant adhering to it. The example given (why no guidance when I’m in the cloud) is quite difficult to turn into something readily available and applicable from a high level. Too many rapidly developing cloud products that need to be taken into consideration when you would want to framework that.

Have a look at ActiveStorage for example. How much time did it take to turn cloud storage into ActiveStorage? Quite long. And it still has some rough edges IMHO. It takes time to turn information technology phenomena into a framework. And sometimes development of the phenomena is so rapid that this is even undoable.

And last but not least: maybe some of the phenomena just lie outside of the scope of Rails to begin with.