Best approach for importing data from a file

Hi All,

I've got a basic Rails app started. One wrinkle is that the user wants ideally to be able to browse the files system (a la Windows Explorer) to select a file and initiate the importation of data from the file.

All files selected will be of a predetermined fixed format, e.g. CSV with Name, Address, etc.

It seems to me that the browser is isolated from the file system, albeit Rails' internals know how to get to it. Is there a way to implement the above-described functionality in a Rails app? Can YAML be used for such functionality?

The only approach I see right now is a Ruby database-updater using perhaps Active Record with ar-extensions or Active Scaffold.

Any guidance would be most appreciated.

Best wishes, Richard

A Google search for

rails import csv

returns several promising results.

Regards, Craig

Hi All,

I've thought more about this and think I know how to proceed with a Rails-integrated approach.

Thanks anyway if you've looked at the original post.

Best wishes, Richard

Thanks, Craig. That search suggestion does help.

Best wishes, Richard

Take a look at attachment_fu.

Hi Rick,

I've been busy, so I haven't checked for responses to my posts for a week or so. Sorry for the delay in this response.

Thanks for your response. I just ordered Advanced Rails Recipes and looked at the attachment_fu.rb from the book. I'm too weak in Rails to see how that's useful for my project.

I think I've concocted a solution to my main concern: how to access a CVS file from the file-system from within a Rails app. I now believe I can: 1. get the filenames housed in a specific directory from within a controller using normal Ruby programming. 2. display those filenames as a list in a view and allow the user to select any one of then 3. open the selected file and read its records, one by one 4. extract the fields from each cvs-formatted record and update the database with the contents of those fields.

Does that sound sensible to you?

Best wishes, Richard

Hey Richard,

Sorry it's been so long, I seem to have dropped this thread.

1) My thought about attachment_fu is that it allows you to pull files that are local to your "rails client" machine and bring them into the rails application space. One way it's used is to enable rails users to post their own images onto a site for sharing with the community. There is a browser included that works much like the usual file browser to aid with locating / selecting file for inclusion.

I'm not sure where your csv files live right now but if they're not somewhere below RAILS_ROOT + "public" then you'll need to figure out how to get them there. Attachment_fu can help here if the files are user generated - field data and the like - that the user will upload into the system.

2) The next thing is moving the data into your Model(s). I'm not sure how comfortable you are with the quality of the data but I guarantee that, no matter what approach you take, you'll need to constantly work on data validation.

You'll probably do best if you start with one model for each file type. That way you can use the model.rb file to validate, validate, validate,... the data on it's way to the database.

3) Of course if it's your own data and you know it's always complete and correct, you can take the migration file approach. It's a little more raw than what is generally considered socially acceptable in the RoR community. The real drawback is then specific database methods leak in past the abstraction boundary. Notice the code for postgresql differs from that for mysql - and we haven't even looked at sqlite3 or db2 or ...:stuck_out_tongue:

class CreateStateAbbrevs < ActiveRecord::Migration   def self.up     create_table :state_abbrevs do |t|       t.string :name       t.string :abbrev     end     csv_file = "#{RAILS_ROOT}/db/migrate/abbr_state.csv"     fields = '(name, abbrev)'

    # PostgreSQL load from file...     execute "COPY state_abbrevs #{fields} FROM '#{csv_file}' WITH CSV"

    # MySQL load from file...     # execute "LOAD DATA INFILE '#{csv_file}' INTO TABLE state_abbrevs FIELDS " +     # "TERMINATED BY ',' OPTIONALLY ENCLOSED BY \"\"\"\" " +     # "LINES TERMINATED BY '\n' " + fields

  end

  def self.down     drop_table :state_abbrevs   end end

So many buttons and so little time, Rick