The Ferret Indexing is taking very 7 days to complete (table with10Million row - DB size is about 2 gb). I could see my memory is used only 40% and cpu is utilized only 10%. but still the indexing is slow.
I am running ferret through DRB
My acts_as_ferret index definition looks something like this.
module Ferret::Analysis class MyAnalyzer def token_stream(field,text) StemFilter.new(LowerCaseFilter.new(StandardTokenizer.new(text))) end end end
ActsAsFerret::define_index('my_index', :models => { Product => { :fields => { :field_one => {:boost => 0.0576}, :field_two => {:boost => 0.0432}, :field_three => {:boost => 0.0432}, :field_four => {:boost => 0.0432}, :field_five => {:boost => 0.0144}, :field_six => :boost => 0.0144}, :field_seven => {:boost => 0.0036}, :field_eight => {:boost => 0.0036}, :field_nine => {:boost => 0.0036}, :field_ten => {:boost => 0.0009}, :field_eleven => {:boost => 0.0006}, :field_twelve => {:boost => 0.0002}, :field_thirteen => {:boost => 0.0000001} } } }, :ferret => { :or_default => true, :analyzer => Ferret::Analysis::MyAnalyzer.new }, :remote => true, :max_buffer_memory => 536870912, #setting to 512 MB for faster indexing. default value is 16 Mb. :merge_factor => 5 #default value is 10. Lower number will speed up search but slow down indexing )
Is there anyway I can improve the performance of indexing ? I really love ferret and acts_as_ferret and and don't want to give-up to sphinx if there is a way.
appreciate your reply
regards madhu
