backgroundrb and REXML issue

Hi,

I have an application that receives XML files, and then hands each one
off to a backgroundrb worker to parse and store the info. Everything
was working, until I needed to support a larger character set. So I
added the following to the top of my XML files: <?xml version="1.0"
encoding="ISO-8859-1"?>

Adding this worked fine when I tested it without using backgroundrb.
That is, when parsing within the main application, I get the expected
behaviour. However, when I try to hand it off to a worker, I get the
following error in backgroundrb.log:

20070423-13:36:41 (8456) Connection reset by peer -
(DRb::DRbConnError)
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in
`send_message'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in
`with_friend'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in
`respond_to?'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
205:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
203:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
125:in `match'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in
`parse'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:376:in `do_work'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in
`work_thread'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:396:in `initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:391:in `initialize'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in
`new_worker'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in
`new_worker'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:187:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/daemonize.rb:192:in `call_as_daemon'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:191:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:227:in `start'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/controller.rb:69:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:182:in `run_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/cmdline.rb:105:in `catch_exceptions'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:181:in `run_proc'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run'
20070423-13:36:41 (8456) script/backgroundrb:29

I tried removing the line that gives the XML version from the top of
the same test file, and then it crashes where I expect it to, when it
encounters invalid characters. Any ideas as to what is happening?

Thanks,

Simon

I have an application that receives XML files, and then hands each one
off to a backgroundrb worker to parse and store the info. Everything
was working, until I needed to support a larger character set. So I
added the following to the top of my XML files: <?xml version="1.0"
encoding="ISO-8859-1"?>

Adding this worked fine when I tested it without using backgroundrb.
That is, when parsing within the main application, I get the expected
behaviour. However, when I try to hand it off to a worker, I get the
following error in backgroundrb.log:

20070423-13:36:41 (8456) Connection reset by peer -
(DRb::DRbConnError)
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in
`send_message'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in
`with_friend'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in
`respond_to?'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
205:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
203:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
125:in `match'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in
`parse'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:376:in `do_work'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in
`work_thread'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:396:in `initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:391:in `initialize'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in
`new_worker'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in
`new_worker'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:187:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/daemonize.rb:192:in `call_as_daemon'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:191:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:227:in `start'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/controller.rb:69:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:182:in `run_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/cmdline.rb:105:in `catch_exceptions'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:181:in `run_proc'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run'
20070423-13:36:41 (8456) script/backgroundrb:29

I tried removing the line that gives the XML version from the top of
the same test file, and then it crashes where I expect it to, when it
encounters invalid characters. Any ideas as to what is happening?

Simon,

What I recommend doing is implementing the worker functionality in a
helper class and then simply delegate from the backgroundrb worker to
the helper. This way you can unit test the helper and don't need to
worry about backgroundrb.

# lib/app_xml_helper.rb
class AppXmlHelper

  def self.do_stuff(options={})
    ...
  end

end

# lib/workers/app_xml_worker.rb
class AppXmlWorker < BackgrounDRb::Worker::RailsBase

  def do_work(args)
    AppXmlHelper.do_stuff(args) # delegate
  end

end
AppXmlWorker.register

Now it's easy to test and you can isolate your biz logic from backgroundrb.

Hope this helps,

Hi,

Yeah, I'll try that to try and help figure out what the issue is.
Just one quick question, can I just create that helper class file by
itself, or does it need to be created through the use of any "script/
something ...."? And do I need to register it with any of my Models
or anything like that, so that the helper class has access to them?

Thanks again,

Simon

Yeah, I'll try that to try and help figure out what the issue is.
Just one quick question, can I just create that helper class file by
itself, or does it need to be created through the use of any "script/
something ...."? And do I need to register it with any of my Models
or anything like that, so that the helper class has access to them?

Simon,

Yes, just drop the helper class file in lib/ and you should be good to
go. You will have access to your models.

Zack,

I tried that, and I get the following error:

20070425-10:51:11 (22978) undefined method `store_xml' for
XmlHelper:Class - (NoMethodError)
20070425-10:51:11 (22978) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:329:in `do_work'

store_xml is the function in my XmlHelper class, which is located at
lib/xml_helper.rb. I've restarted both servers after adding the file
and making the changes. Any idea on what I'm still missing?

Thanks again,

Simon

I tried that, and I get the following error:

20070425-10:51:11 (22978) undefined method `store_xml' for
XmlHelper:Class - (NoMethodError)
20070425-10:51:11 (22978) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:329:in `do_work'

store_xml is the function in my XmlHelper class, which is located at
lib/xml_helper.rb. I've restarted both servers after adding the file
and making the changes. Any idea on what I'm still missing?

Check that store_xml is defined as a class method:

class XmlHelper
  def self.store_xml(args)
  end
end

That was it. Thanks again.

Simon