I'm about to start on my first Rails project, and I could use some advice. I'm going to build a "Data Blender" to replace a lot of frantic work that gets done here by hand using Microsoft Access. Using the Data Blender, you'll be able to import our variety of 30,000 to 200,000 record data files into mySQL, then slice off parts, two counts, export, etc.. Initially it will make it easy to save and modify import and export formats (a lot of the data I get have relatively similar but not the same formats). Eventually, you'll be able to build a simple data recipes -- a very stripped down version of Yahoo pipes. For storing items such as import/export formats, I'll use Active Record. But for data files I'm importing on-the-fly, is it worth it, or does it make more sense to do it all in SQL? Any advice would be greatly appreciated.
Thanks, Anders
