So I am writing an app helping with reporting based on data from a third party service.
My plan is to first, ingest the data from the API, and then to run reports on that data.
The API is paged, and sluggish, with the occasional timeout. How do I ingest large amounts of data from such a beast?
Ideas? Experience? Advice?
