Combining API/DB results into one - is this a good case for sidekiq or rabbitmq?

rubyonrails-talk
1

Hello! I’m building a finance related app, and have one endpoint that:

  • accepts input
  • collects data from multiple REST endpoints with the input
  • saves the data to my db for later access
  • returns the data to the user

I’ve managed to do it by simply putting all of it in rails, but my app is getting a lot of users now and I’m getting problems on peak times.

I’ve looked into background jobs (sidekiq, rabbitmq) and was wondering if these were the right solutions for my problem? A background job that would trigger the REST endpoints and save it to my db?

I also want to, if possible, offer different prioritization queues to my endpoint (based on user tier) and it seems like these message queues support prioritization.

2

Yes absolutely.

Have your web request kick off a background job to poll the REST endpoint and do updates, and then either the data will be available on future requests for your users, or you can look into something like Action Cable to push live updates to your users.

3

I don’t know enough about Rails yet to specifically answer your question. But what you mention sounds a lot like an Event-Driven Architecture (EDA) to me and with that I do have considerable experience.

If your interested in the conceptual side, I can recommend the following resource:

It would be great if you could share back your findings. Thanks.