Is there a way to check of an async database query has actually loaded?

rubyonrails-core
1

Given an asynchronous database query, e.g. @posts = Post.all.load_async, is there a way to check if it’s loaded before waiting on it?

I’m trying to make a feature in phlex-rails that will flush the internal output buffer to the HTTP stream whenever waiting on IO. I expected to be able to do something like this.

def await(query)
  flush unless query.loaded?
  query
end

You could then use it like this

h1 { "Posts" }

await(@posts).each do |post|
  h2 { post.title }
end

If @posts is loaded already, we continue rendering. But if we’re going to wait for it, we may as well push the current buffer (in this case <h1>Posts</h1>) to the browser over the HTTP stream.

Unfortunately, it seems the loaded? predicate always returns true for an asynchronous query, whether or not it’s actually loaded, so we can’t take advantage of the waiting time to perform the flush.

Here’s a simple reproduction:

posts = Post.all.select("pg_sleep(1)").load_async
puts posts.loaded?
sleep(2)
puts posts.loaded?

I expected this to puts false then true but puts true twice.

2

Looking at the code, I found that @loaded is set to true immediately after scheduling the future result. I also noticed scheduled? can be used to check if the result was scheduled but this doesn’t help determine if the future result is ready yet or if we’d need to wait for it.

I was able to get the correct answer with

@posts.instance_variable_get(:@future_result).pending?

I wonder if the project would be open to a PR that delegates pending? to @future_result if it exists? Perhaps something like this on ActiveRecord::Relation:

def pending?
  !!@future_result&.pending?
end

Alternatively, perhaps loaded? could be improved like this:

def loaded?
  if @future_result
    !@future_result.pending?
  else
    @loaded
  end
end
1 Like
3

load? doesn’t mean what I think you are expecting. It means “was this Relation materialized?” or in other words “was a query sent to the database to get a result?”.

in that sense, the current implementation is correct. If the future is already scheduled, the Relation was materialized, and a query was sent to the database.

pending? is an interesting new concept, but it would require us to expose the “future” to the users, that is something we want to avoid as much as possible. The current async API and its implementation should not be exposed to the users other than the load_async method call. But I can see value for libraries as yours.

I’m ok with a new method pending? in a Relation, but it will be tricky to explain in the documentation how it could return a different value than false without exposing that a future exists. If you think you can do that, let’s add the method.

1 Like
4

What I would want to know, is if the data fetched by the query was ready for use. To my mind ready? captures the concept as a synchronously loaded relation is ready when it is loaded, but an asynchronously loaded relation is ready when the result is available. Hopefully easier to explain.

def ready?
  if @future_result
    !@future_result.pending?
  else
    @loaded
  end
end
1 Like
5

That’s a nice option too. I’ll mention this on the PR.