Simon6
(Simon)
April 23, 2007, 5:40pm
1
Hi,
I have an application that receives XML files, and then hands each one
off to a backgroundrb worker to parse and store the info. Everything
was working, until I needed to support a larger character set. So I
added the following to the top of my XML files: <?xml version="1.0"
encoding="ISO-8859-1"?>
Adding this worked fine when I tested it without using backgroundrb.
That is, when parsing within the main application, I get the expected
behaviour. However, when I try to hand it off to a worker, I get the
following error in backgroundrb.log:
20070423-13:36:41 (8456) Connection reset by peer -
(DRb::DRbConnError)
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in
`send_message'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in
`with_friend'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in
`respond_to?'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
205:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
203:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
125:in `match'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in
`parse'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:376:in `do_work'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in
`work_thread'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:396:in `initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:391:in `initialize'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in
`new_worker'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in
`new_worker'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:187:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/daemonize.rb:192:in `call_as_daemon'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:191:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:227:in `start'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/controller.rb:69:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:182:in `run_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/cmdline.rb:105:in `catch_exceptions'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:181:in `run_proc'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run'
20070423-13:36:41 (8456) script/backgroundrb:29
I tried removing the line that gives the XML version from the top of
the same test file, and then it crashes where I expect it to, when it
encounters invalid characters. Any ideas as to what is happening?
Thanks,
Simon
I have an application that receives XML files, and then hands each one
off to a backgroundrb worker to parse and store the info. Everything
was working, until I needed to support a larger character set. So I
added the following to the top of my XML files: <?xml version="1.0"
encoding="ISO-8859-1"?>
Adding this worked fine when I tested it without using backgroundrb.
That is, when parsing within the main application, I get the expected
behaviour. However, when I try to hand it off to a worker, I get the
following error in backgroundrb.log:
20070423-13:36:41 (8456) Connection reset by peer -
(DRb::DRbConnError)
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in
`recv_reply'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in
`send_message'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in
`with_friend'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in
`method_missing'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in
`respond_to?'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
205:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
203:in `expr'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:
125:in `match'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in
`parse'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:376:in `do_work'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in
`work_thread'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in
`work_thread'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:396:in `initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/
slave.rb:391:in `initialize'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in
`new_worker'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in
`dispatch'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in
`new_worker'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in
`perform_without_block'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in
`perform'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in
`main_loop'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in
`initialize'
20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in
`start_service'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:187:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/daemonize.rb:192:in `call_as_daemon'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:191:in `start_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/application.rb:227:in `start'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/controller.rb:69:in `run'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:182:in `run_proc'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons/cmdline.rb:105:in `catch_exceptions'
20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/
daemons.rb:181:in `run_proc'
20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/
plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run'
20070423-13:36:41 (8456) script/backgroundrb:29
I tried removing the line that gives the XML version from the top of
the same test file, and then it crashes where I expect it to, when it
encounters invalid characters. Any ideas as to what is happening?
Simon,
What I recommend doing is implementing the worker functionality in a
helper class and then simply delegate from the backgroundrb worker to
the helper. This way you can unit test the helper and don't need to
worry about backgroundrb.
# lib/app_xml_helper.rb
class AppXmlHelper
def self.do_stuff(options={})
...
end
end
# lib/workers/app_xml_worker.rb
class AppXmlWorker < BackgrounDRb::Worker::RailsBase
def do_work(args)
AppXmlHelper.do_stuff(args) # delegate
end
end
AppXmlWorker.register
Now it's easy to test and you can isolate your biz logic from backgroundrb.
Hope this helps,
Simon6
(Simon)
April 24, 2007, 6:06pm
3
Hi,
Yeah, I'll try that to try and help figure out what the issue is.
Just one quick question, can I just create that helper class file by
itself, or does it need to be created through the use of any "script/
something ...."? And do I need to register it with any of my Models
or anything like that, so that the helper class has access to them?
Thanks again,
Simon
Yeah, I'll try that to try and help figure out what the issue is.
Just one quick question, can I just create that helper class file by
itself, or does it need to be created through the use of any "script/
something ...."? And do I need to register it with any of my Models
or anything like that, so that the helper class has access to them?
Simon,
Yes, just drop the helper class file in lib/ and you should be good to
go. You will have access to your models.
Simon6
(Simon)
April 25, 2007, 2:53pm
5
Zack,
I tried that, and I get the following error:
20070425-10:51:11 (22978) undefined method `store_xml' for
XmlHelper:Class - (NoMethodError)
20070425-10:51:11 (22978) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:329:in `do_work'
store_xml is the function in my XmlHelper class, which is located at
lib/xml_helper.rb. I've restarted both servers after adding the file
and making the changes. Any idea on what I'm still missing?
Thanks again,
Simon
I tried that, and I get the following error:
20070425-10:51:11 (22978) undefined method `store_xml' for
XmlHelper:Class - (NoMethodError)
20070425-10:51:11 (22978) /usr/local/src/rails/434Wireless/lib/workers/
parser_worker.rb:329:in `do_work'
store_xml is the function in my XmlHelper class, which is located at
lib/xml_helper.rb. I've restarted both servers after adding the file
and making the changes. Any idea on what I'm still missing?
Check that store_xml is defined as a class method:
class XmlHelper
def self.store_xml(args)
end
end
Simon6
(Simon)
April 25, 2007, 6:04pm
7
That was it. Thanks again.
Simon