Permission Denied error when trying to connect to Solr running on a different server

I have an SSL enabled ruby on rails application that used solr and the sunspot rails gem
for search. For development I just run solr locally and that works as expected. But I’m trying to set up a stand-alone solr server for production. So
I installed it on its own server and created the core for my site. I also copied over the sunspot schema.xml and solrconfig.xml files from my project to the data/conf directory on the solr server.

I updated
my application config file to point to the solr server and was able to
populate the index for my site from the command line using rails sunspot:solr:reindex (so the application server can communicate with the solr server). I
was able to verify this by accessing the Solr Dashboard web page for the solr server; I can see that my data is there (the numDocs matches with what I’d expect). I was also able to run search from my rails console

But when I try to access search through my web application I’m getting Errno::EACCES in SearchController#index Permission denied - connect(2) for “10.76.42.43” port 8983. (10.76.42.43 is the IP address of my solr server).

Normally this is some sort of file permission issue but I’m not really what file(s) could be causing the problem. I can’t find anything written in the solr logs on the solr server.

I’m using the default setup for solr using java 1.8 and solr 5.4.1; jetty is the service (java -server owned by solr). The
install directory is /opt/solr (owned by root) and the data directory is in /var/solr (owned by solr). I tried changing the owner for /opt/solr but get the same results.

Anybody face the same issue and if so what did you do to fix it? I really would appreciate any help I could get.

So, just to clarify:

From the *app server* command line, you can successfully run a
rake task on the *solr server* - true/false?

From the *app server* rails console, you can access the solr server
manually - true/false?

Is the rails app running as the same *nix user that initiated the rails
sunspot:* cmd?

true - I can manually reindex from the command line on my application server and the index is updated on the solr server

true - I can search from the rails console on my application server and data is returned from the solr server

I’m
not sure about the last one; when I use ps -ef all of the processes (http, rails console, sunspot:solr:reindex) are all started by me (except for the root http process which gets started by root).

FYI: We didn’t add any users to solr; its the default setup; also the web application uses SSL.

It was an apache config issue. The SELinux on the machine was not allowing apache to talk to port 8983. I verified this by temporarily turning off the enforcement (setenforce 0). Once I did this I was able to run search as expected. I then turned the enforcement back on (setenforce 1) and added a rule for port 8983:

semanage port -a -t http_port_t -p tcp 8983

Ah, does that mean you're running Passenger on Apache httpd?

Didn't think about SELinux either, but thanks for letting us know what
fixed it!

Exactly . . .