There are circumstances where Arel is the only way to write a given scope or predicate composably.
Competent developers use it, and write tests for their Arel expressions because they know it’s an API that may have breaking changes across minor releases.
Unfortunately, issue reports involving Arel in application code are met with a bureaucratic and patronising “it’s an internal/private API”. I’m always left wondering, “private/internal to who?” Disappearing the Arel documentation was a further dismaying barrier to contribution.
It’s not just a WTF. These incidents are a FU, every time.
I hear you. Arel was always a leaky abstraction of Rails. It was a public gem, indeed, but it was never part of the public API of the framework. At the time it was introduced into the framework years ago it was not meant to be used in user applications, but, because the abstraction was leaky it worked in a few methods.
Its API never had a proper API review and it was always inconsistent where passing an Arel AST as argument worked inside Active Record. Across many patch level releases it was not rare to break by mistake a method that used to work when passing an Arel AST, and knowing those methods upfront is impossible.
This is why for years we (the Rails Core team) consider Arel private API of the framework and we don’t recommend developers to use. Of course they can use it, given the abstraction is so leaky. Because we considered Arel as private API, we continued to change APIs without taking in consideration backward compatibility and that only caused more problems in applications, just increasing the frustration of the users every time those backward incompatible changes happened.
The reason we merged Arel inside Active Record was to try to fix this situation. We want to expose Arel with a consistent, reviewed, tested and well defined low level API so people can use in their applications. But so far we didn’t make much progress on it. Most of the members of the core team don’t have applications where Arel is being used directly, and we all have different priorities, so we didn’t have yet the motivation to start this work.
This is where a motivated contributor can help. If you use Arel in your applications and want to help us doing the work to define the public API of Arel, I would be happy to provide guidance and support to help you to implement it.
@inopinatus actually said it best. The issues with the Rails Core team saying “It’s a private API” have actually straight up turned me off from wanting to help.
@rafaelfranca I’m not going to lie, my frustration here has primarily been with you. I am sorry to grind my axe here publicly, but I feel like your responses have been a consistent source of frustration when it comes to Arel in particular. I know we have spoken on several Rails issues. The Arel.sql one comes to mind. I think how that whole thing was handled was not good. But I can move past it if we can work together on it.
I use Arel directly in my applications all the time. Arel provides composable queries without me having to use strings. I am a vocal advocate of its power, and I would love to help others use it too. I can provide plenty of real world examples where we use Arel in situations where ActiveRecord’s query methods simply do not work. Maybe we can just extend ActiveRecord so that it has a more powerful interface and Arel doesn’t need to be public. But either way, there is a need for better SQL writing inside Rails.
No problem with calling this out publicly, thank you for doing that, and I’m sorry I made you feel that way. My behavior were also public and while it was not my intention to cause frustration on anyone if that happened, for any reason, I’m totally fine with people calling that out.
The idea of this entire category is to find patterns and actionable items. In my opinion this is very actionable. There is already consensus in the desire to make the Arel API public, we just need someone to do the work. I’m happy to support you on that. Just let me know and I’ll invite you to a Basecamp project so we can start making this happen.
It is not. But most of the artifacts of those projects are public. Like the issues, pull requests and commits. You can use those to read the progress and it usually have information on why the changes are being made.
I wonder if the old DataMapper 1.0 APIs would serve as a useful inspiration here? One of the things that drew me to DM back in the day was that it was built on several layers of public API. Including a rather nice composable relational algebra API.
I would highly recommend taking an inspiration from Sequel here, whose SQL expression API is public and very well designed. Instead of relying on methods, it overrides ruby operators, which means you don’t have to use as many parentheses. It’s like the Squeel gem, but the instance-eval block syntax is optional.
A while ago I’ve written a blog post showing what is possible, though it would probably be better to look at Sequel docs.