Permission Denied error when trying to connect to Solr running on a different server

I have an SSL enabled ruby on rails application that used solr and the sunspot rails gem
for search. For development I just run solr locally and that works as expected. But I’m trying to set up a stand-alone solr server for production. So
I installed it on its own server and created the core for my site. I also copied over the sunspot schema.xml and solrconfig.xml files from my project to the data/conf directory on the solr server.

I updated
my application config file to point to the solr server and was able to
populate the index for my site from the command line using rails sunspot:solr:reindex (so the application server can communicate with the solr server). I
was able to verify this by accessing the Solr Dashboard web page for the solr server; I can see that my data is there (the numDocs matches with what I’d expect). I was also able to run search from my rails console

But when I try to access search through my web application I’m getting Errno::EACCES in SearchController#index Permission denied - connect(2) for “” port 8983. ( is the IP address of my solr server).

Normally this is some sort of file permission issue but I’m not really what file(s) could be causing the problem. I can’t find anything written in the solr logs on the solr server.

I’m using the default setup for solr using java 1.8 and solr 5.4.1; jetty is the service (java -server owned by solr). The
install directory is /opt/solr (owned by root) and the data directory is in /var/solr (owned by solr). I tried changing the owner for /opt/solr but get the same results.

Anybody face the same issue and if so what did you do to fix it? I really would appreciate any help I could get.

So, just to clarify:

From the *app server* command line, you can successfully run a
rake task on the *solr server* - true/false?

From the *app server* rails console, you can access the solr server
manually - true/false?

Is the rails app running as the same *nix user that initiated the rails
sunspot:* cmd?

true - I can manually reindex from the command line on my application server and the index is updated on the solr server

true - I can search from the rails console on my application server and data is returned from the solr server

not sure about the last one; when I use ps -ef all of the processes (http, rails console, sunspot:solr:reindex) are all started by me (except for the root http process which gets started by root).

FYI: We didn’t add any users to solr; its the default setup; also the web application uses SSL.

It was an apache config issue. The SELinux on the machine was not allowing apache to talk to port 8983. I verified this by temporarily turning off the enforcement (setenforce 0). Once I did this I was able to run search as expected. I then turned the enforcement back on (setenforce 1) and added a rule for port 8983:

semanage port -a -t http_port_t -p tcp 8983

Ah, does that mean you're running Passenger on Apache httpd?

Didn't think about SELinux either, but thanks for letting us know what
fixed it!

Exactly . . .