backgroundrb and REXML issue

Hi,

I have an application that receives XML files, and then hands each one off to a backgroundrb worker to parse and store the info. Everything was working, until I needed to support a larger character set. So I added the following to the top of my XML files: <?xml version="1.0" encoding="ISO-8859-1"?>

Adding this worked fine when I tested it without using backgroundrb. That is, when parsing within the main application, I get the expected behaviour. However, when I try to hand it off to a worker, I get the following error in backgroundrb.log:

20070423-13:36:41 (8456) Connection reset by peer - (DRb::DRbConnError) 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in `respond_to?' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 205:in `expr' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 203:in `expr' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 125:in `match' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in `parse' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/ parser_worker.rb:376:in `do_work' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in `work_thread' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in `work_thread' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in `work_thread' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:396:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:391:in `initialize' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in `new_worker' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in `new_worker' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:187:in `start_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/daemonize.rb:192:in `call_as_daemon' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:191:in `start_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:227:in `start' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/controller.rb:69:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons.rb:182:in `run_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/cmdline.rb:105:in `catch_exceptions' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons.rb:181:in `run_proc' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' 20070423-13:36:41 (8456) script/backgroundrb:29

I tried removing the line that gives the XML version from the top of the same test file, and then it crashes where I expect it to, when it encounters invalid characters. Any ideas as to what is happening?

Thanks,

Simon

I have an application that receives XML files, and then hands each one off to a backgroundrb worker to parse and store the info. Everything was working, until I needed to support a larger character set. So I added the following to the top of my XML files: <?xml version="1.0" encoding="ISO-8859-1"?>

Adding this worked fine when I tested it without using backgroundrb. That is, when parsing within the main application, I get the expected behaviour. However, when I try to hand it off to a worker, I get the following error in backgroundrb.log:

20070423-13:36:41 (8456) Connection reset by peer - (DRb::DRbConnError) 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in `respond_to?' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 205:in `expr' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 203:in `expr' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 125:in `match' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in `parse' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/ parser_worker.rb:376:in `do_work' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in `work_thread' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in `work_thread' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in `work_thread' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:396:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:391:in `initialize' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in `new_worker' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in `new_worker' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:187:in `start_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/daemonize.rb:192:in `call_as_daemon' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:191:in `start_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:227:in `start' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/controller.rb:69:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons.rb:182:in `run_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/cmdline.rb:105:in `catch_exceptions' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons.rb:181:in `run_proc' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' 20070423-13:36:41 (8456) script/backgroundrb:29

I tried removing the line that gives the XML version from the top of the same test file, and then it crashes where I expect it to, when it encounters invalid characters. Any ideas as to what is happening?

Simon,

What I recommend doing is implementing the worker functionality in a helper class and then simply delegate from the backgroundrb worker to the helper. This way you can unit test the helper and don't need to worry about backgroundrb.

# lib/app_xml_helper.rb class AppXmlHelper

  def self.do_stuff(options={})     ...   end

end

# lib/workers/app_xml_worker.rb class AppXmlWorker < BackgrounDRb::Worker::RailsBase

  def do_work(args)     AppXmlHelper.do_stuff(args) # delegate   end

end AppXmlWorker.register

Now it's easy to test and you can isolate your biz logic from backgroundrb.

Hope this helps,

Hi,

Yeah, I'll try that to try and help figure out what the issue is. Just one quick question, can I just create that helper class file by itself, or does it need to be created through the use of any "script/ something ...."? And do I need to register it with any of my Models or anything like that, so that the helper class has access to them?

Thanks again,

Simon

Yeah, I'll try that to try and help figure out what the issue is. Just one quick question, can I just create that helper class file by itself, or does it need to be created through the use of any "script/ something ...."? And do I need to register it with any of my Models or anything like that, so that the helper class has access to them?

Simon,

Yes, just drop the helper class file in lib/ and you should be good to go. You will have access to your models.

Zack,

I tried that, and I get the following error:

20070425-10:51:11 (22978) undefined method `store_xml' for XmlHelper:Class - (NoMethodError) 20070425-10:51:11 (22978) /usr/local/src/rails/434Wireless/lib/workers/ parser_worker.rb:329:in `do_work'

store_xml is the function in my XmlHelper class, which is located at lib/xml_helper.rb. I've restarted both servers after adding the file and making the changes. Any idea on what I'm still missing?

Thanks again,

Simon

I tried that, and I get the following error:

20070425-10:51:11 (22978) undefined method `store_xml' for XmlHelper:Class - (NoMethodError) 20070425-10:51:11 (22978) /usr/local/src/rails/434Wireless/lib/workers/ parser_worker.rb:329:in `do_work'

store_xml is the function in my XmlHelper class, which is located at lib/xml_helper.rb. I've restarted both servers after adding the file and making the changes. Any idea on what I'm still missing?

Check that store_xml is defined as a class method:

class XmlHelper   def self.store_xml(args)   end end

That was it. Thanks again.

Simon