Real-Time Morse Code Translation Becomes Slow Under Concurrent Requests in My Ruby on Rails Application

[joeroot]

I run a Morse Code website that allows users to convert text into Morse code and decode Morse messages through an interactive web interface, and I recently started moving some of the backend functionality into a Ruby on Rails application. The translation itself is extremely lightweight because it mainly consists of looking up characters in a Morse mapping and returning the corresponding result. However, when several users access the application at the same time, response times occasionally increase significantly even when each individual request contains only a short piece of text. I am trying to understand why such a simple operation can become noticeably slower when concurrent requests reach the Rails application.

The frontend sends requests to the Rails backend for some translation-related operations, while other interface features such as audio playback continue to run directly in the browser. During testing with only a few users, the application responds almost instantly and there is no obvious performance problem. When I simulate a larger number of simultaneous requests, however, response times begin increasing and some requests appear to wait before being processed. The server CPU does not always appear to be saturated, which makes me suspect that the bottleneck may be related to Rails request handling, Puma configuration, database access, or connection pooling rather than the Morse Code algorithm itself.

I have already checked the translation code and confirmed that it does not perform any particularly expensive computation or external API calls for each request. The Morse mapping is kept in memory, so I would not expect the lookup operation itself to consume meaningful resources even under considerable traffic. I am also using a relatively simple controller action, but I am unsure whether there are Rails-specific request lifecycle costs that become more visible when many small requests arrive simultaneously. I would like to determine whether I should first investigate Puma worker and thread settings, Rails database connection pools, middleware, or another part of the request pipeline.

Another complication is that the website is designed to feel real time because users expect the Morse output to appear almost immediately while interacting with the translator. Introducing aggressive batching or asynchronous processing on the frontend could make the application feel less responsive, while sending every small interaction to the backend may create unnecessary request volume. I am therefore trying to find the right Rails architecture for handling lightweight, frequent requests without introducing avoidable overhead. I am particularly interested in whether a Rails API-only setup, background processing, caching, or keeping more translation logic entirely client-side would be considered the better approach.

I have also considered using caching because the same characters and common Morse combinations are translated repeatedly, but I am not sure whether caching individual translation results would provide any meaningful benefit given how inexpensive the underlying operation is. Instead, I suspect the more important optimization may involve reducing unnecessary database queries and ensuring that Rails is not creating excessive connections or waiting on limited resources during traffic spikes. I would like to benchmark the application properly rather than making changes based only on CPU usage or individual request timings. Any recommendations for profiling a Rails application under concurrent lightweight workloads would be particularly useful.

Has anyone encountered a similar performance pattern in a Rails application where individual requests are extremely cheap but response times increase significantly as concurrency rises? I would appreciate advice on how to systematically identify whether the bottleneck is Puma concurrency, database connection pooling, middleware, application code, or server configuration before attempting a major architectural change. For a Morse Code application where low latency is important but the actual computation is trivial, I would also like to know whether experienced Rails developers would keep the translation entirely client-side and reserve Rails for persistence and account-related functionality. Any guidance on benchmarking, profiling, and designing this type of workload would be greatly appreciated. Sorry for long post!

[patrickdonahue]

“The server CPU does not always appear to be saturated” is the most important sentence in your post, and it is the symptom rather than a red herring.

In CRuby, threads do not execute Ruby code in parallel. The Global VM Lock allows exactly one thread to run Ruby at a time, so a purely CPU-bound workload like a character-map lookup gains nothing from more threads. It simply queues. One core busy while the machine as a whole looks idle is precisely what that looks like from the outside.

Your own generated config says as much. Rails 8’s config/puma.rb ships with this comment and this default:

# ...due to CRuby's Global VM Lock (GVL) it has diminishing returns and will
# degrade the response time (latency) of the application.
threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)
threads threads_count, threads_count

and WEB_CONCURRENCY defaults to 1. So out of the box you are running a single process with three threads, which for CPU-bound work means an effective parallelism of about one and two requests waiting their turn. Simulate more than a few simultaneous users and the queue is the latency you are measuring.

For CPU-bound work the fix is processes, not threads:

WEB_CONCURRENCY=auto   # or an explicit count, roughly your core count

Each Puma worker is a separate process with its own GVL, so workers genuinely run in parallel. Threads only buy concurrency while a request is waiting, on the database or a network call, because the GVL is released around IO. Morse translation waits on nothing, so threads buy you nothing.

Two things worth ruling out while you are in there.

The connection pool. If any part of that request touches the database, the pool must be at least as large as the thread count, or threads block waiting to check a connection out. The default checkout_timeout is five seconds:

# activerecord/lib/active_record/database_configurations/hash_config.rb
(configuration_hash[:checkout_timeout] || 5).to_f

so a starved pool shows up as requests taking a suspiciously round five seconds before either succeeding or raising ActiveRecord::ConnectionTimeoutError. If you see 5s in your timings, stop looking at Puma. Note also that the pool is per process, so pool should track RAILS_MAX_THREADS rather than your total worker count.

Where the time actually goes. Get the numbers instead of inferring them. Rails already logs Completed 200 OK in 43ms (Views: 1.2ms | ActiveRecord: 0.4ms). If the total is large while views and Active Record are both tiny, the time is being spent queueing rather than working. Better still, have your proxy stamp an X-Request-Start header so you can measure queue time separately from service time. That single distinction is the whole diagnosis here, it takes an afternoon to instrument, and it removes all the guessing.

One last thought, offered kindly: if the translation really is a character-map lookup, the fastest version of this endpoint is the one that does not exist. That is a few lines of JavaScript, and you already run audio playback in the browser. Sending a short string to a server and back in order to perform a hash lookup is a network round trip spent on nothing. Keep Rails for the parts that need persistence, accounts or sharing, do the translation client-side, and the concurrency question disappears rather than needing to be tuned.