replace REXML with Hpricot

Hi,

is there any way to replace REXML with Hpricot for rendering XML? I am using Jruby so cant use libxml, and need to significantly increase the speed of XML rendering.

Thanks
Adam

AD wrote:

is there any way to replace REXML with Hpricot for rendering XML? I am using Jruby so cant use libxml, and need to significantly increase the speed of XML rendering.

What's wrong with Builder::XmlMarkup (iirc)?

Hpricot uses a C module to get fast - like libxml. So if you can't use one in Jruby...

I think I actually meant XML::Builder, I was under the initial impression this was actually a REXML Library.

At any rate my performance metrics show that 60-70% is done via “Rendering” b/c of some complicated XML generation so I am looking for some optimization mechanisms here. HPricot is faster than Builder and REXML and is supported in JRuby (running it now).

So basically I am looking to replace XML::Builder with something faster , that is not libXML (like Hpricot)

Adam

I think I actually meant XML::Builder, I was under the initial impression
this was actually a REXML Library.
At any rate my performance metrics show that 60-70% is done via "Rendering"
b/c of some complicated XML generation so I am looking for some optimization
mechanisms here. HPricot is faster than Builder and REXML and is supported
in JRuby (running it now).

So basically I am looking to replace XML::Builder with something faster ,
that is not libXML (like Hpricot)

There is a jruby version of hpricot. Builder and Rexml are unrelated.
The only bit I've ever found to be slow about builder is that it tries
to be a bit too clever with its escaping and tries to work out all
sorts of stuff that I didn't need. Overriding the _escape method to
just replace &, <, > sped things up for me

Fred

AD wrote:

So basically I am looking to replace XML::Builder with something faster , that is not libXML (like Hpricot)

Have you time-tested and profiled your code & tests to see what's fast? it's usually a surprise...

well I am just noticing when the actions get rendered, its always like (Rendering (78%)) and some of the XML rendering just seems slow in relation to the time for the DB query.

Do you have any profile recommendations besides ruby-prof ?

here is some real world output… Any feedback is welcome, notice the very high render time:

Rendering channels/index
Completed in 3.38310 (0 reqs/sec) | Rendering: 1.38117 (40%) | DB: 0.48588 (14%) | 200 OK [http://dev1.domain.com/users/1/2008/05/28/index.xml]

[http://dev1.domain.com/users/1/2008/05/28/index.xml]

Thread ID: 3067917240
Total: 2.920000

%self total self wait child calls name

5.82 0.42 0.17 0.00 0.25 18281 Fixnum#xchr
4.45 0.14 0.13 0.00 0.01 79 Array#select
4.11 0.56 0.12 0.00 0.44 2288 Array#map
3.08 0.10 0.09 0.00 0.01 29 ActiveRecord::Associations::ClassMethods#collection_accessor_methods

2.74 0.11 0.08 0.00 0.03 54266 Kernel#===
1.71 0.16 0.05 0.00 0.11 248 Hash#each
1.71 0.08 0.05 0.00 0.03 54327 Hash#[]
1.37 0.06 0.04 0.00 0.02 288 Module#define_method

1.37 0.04 0.04 0.00 0.00 15 Mysql::Result#each_hash
1.37 0.04 0.04 0.00 0.00 108 Array#-
1.03 0.03 0.03 0.00 0.00 39128 Hash#default
1.03 0.03 0.03 0.00 0.00 324 Module#constants

1.03 0.03 0.03 0.00 0.00 56670 Fixnum#==

Thread ID: 3067917860
Total: 0.770000

here is some real world output.. Any feedback is welcome, notice the
very high render time:

Fixnum.xchr is builder's escaping stuff. See my previous tip for
possible ways out.

Fred

I believe you are referencing this method

def _escape(text)
text.to_xs
end

How do I go about overriding this ?

I believe you are referencing this method

def _escape(text)
      text.to_xs
end

How do I go about overriding this ?

same way as you override any other method

class Builder::XmlMarkup
   def _escape(text)
     result = text.dup
     result.gsub!("&", "&amp;")
     result.gsub!("<", "&lt;")
     result.gsub!(">", "&gt;")
   end
end

in my case what i did was

xml = Builder::XmlMarkup.new(:target => $stdout, :indent => 2)
xml.instruct!

def xml.output(string)
   result = string.dup
   result.gsub!("&", "&amp;")
   result.gsub!("<", "&lt;")
   result.gsub!(">", "&gt;")
   self << result
end

and then call output rather than text whenever i wanted to output stuff.
There was a thread a while back on one of the ruby list about the
fastest way to do this - do you do it in one pass fixing up each of
the 3 possible substitutions in one go, or in 3 passes like i did, and
concluded that the above version was the fastest. It might be
different if you had really long strings.

See also http://bogomips.org/fast_xs/ which is a c implementation of
the escaping thing (haven't tried this, but is probably the fastest of
all).

Fred

to expand on that, just gem install fast_xs. It's faster than what I had previously

Fred

yea fast_xs looks better, but right now its native C only so no Jruby support.

Is it possible to generated XML without using builder ? If the XML structure is not too complicated, and I am OK sacrificing some of the validation/substitution, could I just @results.each { |r| } and print some XML ?

Thx
Adam

yea fast_xs looks better, but right now its native C only so no
Jruby support.

Is it possible to generated XML without using builder ? If the XML
structure is not too complicated, and I am OK sacrificing some of
the validation/substitution, could I just @results.each { |r| } and
print some XML ?

Course you can, it's just text after all! I got a significant speed
boost overwriting _escape so that might be enough for you.

Fred

Did you just drop this in application.rb ?

class Builder::XmlMarkup
def _escape(text)
result = text.dup
result.gsub!("&", “&”)
result.gsub!("<", “<”)
result.gsub!(">", “>”)
end
end

Did you just drop this in application.rb ?

class Builder::XmlMarkup
  def _escape(text)
    result = text.dup
    result.gsub!("&", "&amp;")
    result.gsub!("<", "&lt;")
    result.gsub!(">", "&gt;")
  end
end

just in a script i was writing. (but it should be ok in
application.rb), with the exception of one line which i left out :slight_smile:

it should be
   def _escape(text)
     result = text.dup
     result.gsub!("&", "&amp;")
     result.gsub!("<", "&lt;")
     result.gsub!(">", "&gt;")
     result
   end

jruby is causing you to miss out on the fast_xs fun, but you might be
able to rewrite that method in java, with similar effects.

Fred

Frederick,

Thanks for the help but i am getting an error. I dropped this outside the ApplicationController but is throwing an error

class Builder::XmlMarkup
def _escape(text)
result = text.dup
result.gsub!("&", “&”)
result.gsub!("<", “<”)
result.gsub!(">", “>”)
result
end
end

ActionView::TemplateError (You have a nil object when you didn’t expect it!
The error occurred while evaluating nil.gsub) on line #1 of channels/system.xml.builder:
1: xml.instruct!
2: xml.response {
3: xml.result {
4: xml.status “Success”

Frederick,

Thanks for the help but i am getting an error. I dropped this
outside the ApplicationController but is throwing an error

class Builder::XmlMarkup
  def _escape(text)
    result = text.dup
    result.gsub!("&", "&amp;")
    result.gsub!("<", "&lt;")
    result.gsub!(">", "&gt;")
    result
  end
end

Weird, works well enough for me. the error you got looks like what you
would get with the version without the result on the last line.

Fred

do you have it in application.rb ?

I also noticed that the _escape function is in xmlbase.rb not xmlmarkup.rb. Not sure if that is what is breaking here…

I also noticed that the _escape function is in xmlbase.rb not
xmlmarkup.rb. Not sure if that is what is breaking here...

XmlMarkup inherits from XmlBase so it shouldn't make any difference. I
didn't have it in application.rb but I doubt that would make a
difference.

Fred