Problem in saving fields to database from csv using fastercsv

rubyonrails-talk
1

Hi,

First of all very sorry for this long post. I am trying to save my csv data to table items which is associated with Item model.

This is what my csv have:

‘name’;‘number’;‘sub_category_id’;‘category_id’;‘quantity’;‘sku’; ‘description’;‘cost_price’;‘selling_price’

‘Uploaded Item Number 1’;‘54’;‘KRT’;‘WN’;‘67’;‘WNKRT0054’;‘Some Description here!!’;‘780’;‘890’

‘Uploaded Item Number 2’;‘74’;‘KRT’;‘WN’;'98;‘WNKRT0074’;‘Some Description here!!’;‘8660’;‘9790’

First row show the fields for items table.

Here I am using fastercsv to process my csv and paperclip to upload.

I am able to process file read content and able to fill up the field too here is the processing code:

def proc_csv

@import = Import.find(params[:id])

@lines = parse_csv_file(@import.csv.path)

@lines.shift

@lines.each do |line , j|

unless line.nil?

line_split = line.split(“;”)

unless ((line_split[0].nil?) or (line_split[1].nil?) or (line_split[2].nil?) or (line_split[3].nil?) or (line_split[4].nil?) or (line_split[5].nil?))

I used puts to get to know about what’s going on.

puts "*“50+“line_split[0]: #{line_split[0]}”+"”*50

puts "*“50+“line_split[1]: #{line_split[1]}”+"”*50

puts "*“50+“line_split[2]: #{line_split[2]}”+"”*50

puts "*“50+“line_split[3]: #{line_split[3]}”+"”*50

puts "*“50+“line_split[4]: #{line_split[4]}”+"”*50

puts "*“50+“line_split[5]: #{line_split[5]}”+"”*50

puts "*“50+“line_split[6]: #{line_split[6]}”+"”*50

puts "*“50+“line_split[7]: #{line_split[7]}”+"”*50

puts "*“50+“line_split[8]: #{line_split[8]}”+"”*50

@item = [:name => line_split[0], :number => line_split[1], :sub_category_id => line_split[2],:category_id => line_split[3],:quantity => line_split[4], :sku => line_split[5], :description => line_split[6], :cost_price => line_split[7], :selling_price => line_split[8]]

puts “#”*100+“@item is: #{@item.inspect}”+“#”*100

end

end

end

redirect_to import_path(@import)

end

but the problem is that when it process it and when I check the @item in console it looks like this:

####################################################################################################@item is: [{:quantity=>“\000’\0006\0007\000’\000”, :name=>“\000’\000U\000p\000l\000o\000a\000d\000e\000d\000 \000I\000t\000e\000m\000 \000N\000u\000m\000b\000e\000r\000 \0001\000’\000”, :sku=>“\000’\000W\000N\000K\000R\000T\0000\0000\0005\0004\000’\000”, :cost_price=>“\000’\0007\0008\0000\000’\000”, :number=>“\000’\0005\0004\000’\000”, :selling_price=>“\000’\0008\0009\0000\000’\000”, :sub_category_id=>“\000’\000K\000R\000T\000’\000”, :description=>“\000’\000S\000o\000m\000e\000 \000D\000e\000s\000c\000r\000i\000p\000t\000i\000o\000n\000 \000h\000e\000r\000e\000!\000!\000’\000”, :category_id=>“\000’\000W\000N\000’\000”}]####################################################################################################

####################################################################################################@item is: [{:quantity=>“\000’\0009\0008\000”, :name=>“\000’\000U\000p\000l\000o\000a\000d\000e\000d\000 \000I\000t\000e\000m\000 \000N\000u\000m\000b\000e\000r\000 \0002\000’\000”, :sku=>“\000’\000W\000N\000K\000R\000T\0000\0000\0007\0004\000’\000”, :cost_price=>“\000’\0008\0006\0006\0000\000’\000”, :number=>“\000’\0007\0004\000’\000”, :selling_price=>“\000’\0009\0007\0009\0000\000’\000”, :sub_category_id=>“\000’\000K\000R\000T\000’\000”, :description=>“\000’\000S\000o\000m\000e\000 \000D\000e\000s\000c\000r\000i\000p\000t\000i\000o\000n\000 \000h\000e\000r\000e\000!\000!\000’\000”, :category_id=>“\000’\000W\000N\000’\000”}]####################################################################################################

Can anyone kindly tell me why am I getting this kind of string instead of simple string I entered in my csv file? And because of this it’s not being saved into the table too, I have tried all possible formats but nothing seems to be working. I want “Uploaded Item Number 1” instead of “\000’\0006\0007\000’\000”, :name=>“\000’\000U\000p\000l\000o\000a\000d\000e\000d\000 \000I\000t\000e\000m\000 \000N\000u\000m\000b\000e\000r\000 \0001\000’\000” . Any help will be appreciated. Thanks in advance :slight_smile:

2

This is not the right place for this sort of question, use the rubyonrails-talk group instead.

3

As others have pointed out, this isn't the right place for this. In any case, I suspect your issue is that Excel has spit out a UTF-16 file and you didn't tell whatever's parsing it about that...

--Matt Jones

4

Hi,

First of all very sorry for this long post. I am trying to save my csv data to table items which is associated with Item model.

It looks like the data you are importing UTF16 data. I’m guessing you were expecting UTF8 - I’d start by reading up on the tools available for converting between string encodings in ruby (1.8 and 1.9 are quite different in this respect)

Fred

5

Ok, Thanks for letting me know about this.

6

So what could be the possible solution? I wasted my whole day trying to figure out the issue. But didn’t get anything related with this :frowning:

7

So what could be the possible solution? I wasted my whole day trying to figure out the issue. But didn't get anything related with this :frowning:

convert the text to utf8 before you try and parse it.

Fred

8

Hi

Thanks for your kind response, Frederick. I have finally sort it out. It was my csv file which was in UTF-61le format, I made a new file with UITF-8 format and that worked as the way I wanted.

One more thing I wants to ask that is, I am getting the string now in this format - :name => “‘Uploaded item number 1’” instead of :name => “Uploaded item number 1”, and because of this the field that is being saved in the column name is: ‘Uploaded item number 1’ instead of Uploaded item number 1. how do I remove this quote ( ’ ) ? Any idea?

9

Hi

Thanks for your kind response, Frederick. I have finally sort it out. It was my csv file which was in UTF-61le format, I made a new file with UITF-8 format and that worked as the way I wanted.

 One more thing I wants to ask that is, I am getting the string now in

this format - :name => "'Uploaded item number 1'" instead of :name => "Uploaded item number 1", and because of this the field that is being saved in the column name is: 'Uploaded item number 1' instead of Uploaded item number 1. how do I remove this quote ( ' ) ? Any idea?

You can probably tell FasterCSV that your fields are quoted with ' rather than the more usual "

Fred

10

Thanks Frederick, I got it :slight_smile:

I did this:

FasterCSV.foreach(path_to_csv, :col_sep => ‘;’, :quote_char => “'”) do |row|

Used row here!

end

And it removed ’ from the string.