Delete_all, destroy_all support batch deletion

Our tables of database is too huge but it is sharded. so it needs to delete in batch. I am using code to delete in batch.

e.g.

codes.in_groups_of(1000, false).each {|batch| Model.where(code: batch).delete_all}

I’d be happy with new feature e.g.

Model.where(code: codes).batch_delete_all(1000)

the example for is only for example. it may be another form under implementation.

Why I’m using batch deletion is because huge table and huge deletion rows can occupy database. that affect other processes because of starvation.

1 Like

If this feature seems to be good, I’ll start implementing. 7.0 supports batch deletion. however that does not support splitting parameter.

1 Like

@KSH-code, can’t that be achieved with .in_batches?

Model.where(code: codes).in_batches(of: 1000).delete_all
1 Like

@jorgemanrubia that works. but that’s with a problem. What if size of codes array 100,000?

Then, error occurs, because of buffer size. and query is too heavy.

Oh I understand now @KSH-code :+1:. My first impression is wondering if Rails needs first-level support for this. I’m sure a good PR showing the code and motivations will help a lot.

1 Like

Great, I’ll be on it. Thanks your opinion!

1 Like
def delete_all_in_batches(condition, batch_size: 1000, error_on_ignore: nil)
  key, value = hash.to_a.first
  value.in_groups_of(batch_size, false).each do |batch|
    self.where(key => batch).in_batches(batch_size).delete_all
  end
end

I’ve been writing a test.