Forking and CoW are killing my machine when running GC

rubyonrails-core
1

Here is the problem : the memory is duplicated when forking and calling GC in 2.2.1. The main issue here is that I am operating on huge data, going up to 1GB, and when the GC is running and I am forking a process, it is killing my machine since the 1GB becomes 2GB.

We have written a small program that reproduces the issue (see attached file).

  1. The program instantiates an object and then forks into two processes.
  2. The GC is called in the child process.
  3. The memory allocation (as given by /proc/pid/smaps) changes from shared to private, thereby indicating a doubling of memory consumption. Here is the output of the program (size is in mb):
   ruby version 2.2.1
time pid message shared private
4.011s 4723 Parent pre GC 68 0
4.013s 4737 Child pre GC 68 0
8.019s 4723 Parent post GC 5 62
8.093s 4737 Child post GC 5 66

We have tested the program on Ubuntu 14.04 with ruby 1.9.3 and 2.2.1. The tests have been performed on a freshly installed Ubuntu machine with no other software.

We have also tried to fork 10 children and see a 10 doubling of the memory consumption, the issue only occurs after running the GC.

mem.rb (1.25 KB)

2

Small update. I have played with the GC and have run several tests. I manage to solve the problem by disabling the GC with GC.disable, but then my memory gets filled with garbages. I have also disabled hugepages on linux and no effects. Finally I’ve played with some of the parameters of the GC like the heap size, and reducing its activity to 0 (GC.count) but no changes. Every test have been performed on the same machine with the same parameters.

3

Thomas, is this a Rails app or related to Rails in any way? Otherwise this should be posted in the Ruby mailing list.

4

It’s just Ruby stuff. Is it possible to move this to Ruby mailing list ?

5

As mohamed mentioned, this forum isn’t for general debugging but rather for people talking about developing the core of Rails. It’s not an appropriate place for bug reports. If you think you found a bug in the way Ruby uses fork, I would try to reproduce it with a minimal ruby script and without rails.

The behavior you’re describing is unusual, and i’ve not seen it before. If before you fork and your app uses 100mb of ram, then after you fork it should use more ram but still be less than double, something probably around 180 or 150mb of ram total. If you’re seeing double the ram, its likely that your application code is doing something to modify or create many objects. I would recommend looking into allocation_tracer gem or memory_profiler gem to see where new objects are being created after you fork, maybe derailed_benchmarks however that isn’t set up to run with a forking server.

Hope that helps, please re-post to another list, you can let us know the link after you do so we can followup if anyone is interested.