Hi everyone, I’m building a Rails API to collect sensor data from ESP32 devices and want to make sure my endpoints are structured efficiently for frequent, small POST requests. For context, I’ve been following this ESP32 tutorial that shows how to send sensor readings to a cloud service via HTTP POST: https://www.theengineeringprojects.com/2022/02/sending-data-to-cloud-with-esp32-and-thingspeak.html. I’ve also seen Arduino forum threads and Raspberry Pi community projects where people push sensor values using REST calls or MQTT brokers to backend services. In Rails, should I use standard JSON APIs with ActiveModelSerializers, background jobs with Sidekiq/ActiveJob, or streaming with ActionCable to maintain fast and scalable ingestion, especially when handling data from dozens of devices simultaneously?
Hard to say. If dozens means something like 50 to 120 devices, I think any one of those methods is fine. I’d go for standard just because standard means easier.
The places where it can get tricky and affect performance, as I see it, are:
- What do you do when you get the request? Is it just an insert? Are there any data consistency concerns when writing to the DB for any of the requests?
- Are there any concerns around what would be easier for the ESP32 to handle? Is JSON the best format? Does it even matter?
I’ve never worked with this so I wouldn’t know anything about 2, but the question came to mind. For number 1, if you have no consistency concerns around data insertion, making sure you don’t exhaust your connection pool should pretty much have you covered.
Another question that comes to mind is if these POSTs are so often that, effectively, at any given moment you’d be getting a number of requests roughly equivalent to the amount of devices you have. If that’s the case, I’d be wondering just how many simultaneous connections can your app server and database handle…
Hi!
If this is more of an industrial-style or long-running project, I’d strongly consider going in the MQTT direction. In one of my projects I set up a Mosquitto broker and integrated it with a Rails app, using the Rails database to authenticate devices, and it worked very reliably for sensor ingestion.
That said, for a smaller setup (dozens of ESP32s), a simple JSON POST endpoint in Rails can still be perfectly fine, especially if you keep the controller lightweight and push any heavier processing into background jobs.
A couple of important questions that might help guide the architecture:
- What do you actually want to do with the incoming data (store raw values, aggregate, trigger alerts, etc.)?
- Do you need two-way communication with the devices, or is this strictly device → backend?
The answers to those usually make it much clearer whether plain HTTP, background jobs, or something like MQTT is the best fit. 
Write a few lines about the task and I’ll help you. What database server are you using? It’s important because it’s possible that it would be easier to aggregate incoming data with stored procedures, for example.