text visible in browser but not in source

rubyonrails-talk
21

1) choose HTTP client 2) send request 3) parse response

22

How do I download the js and parse it directly for the data I require

23

I tried the code below but when I get to JSON.parse JSON::ParserError: 765: unexpected token at 'jQuery18307882644047005491_1545855559753({“success”:true,“code”:0,“results”:[{“productId”:32815555905, …

require ‘net/http’

require ‘json’

url = " "

uri = URI(url)

response = Net::HTTP.get(uri)

data = JSON.parse(response)

24

Net::HTTP.get() returns a string

data = JSON.parse(response)

JSON::ParserError: 765: unexpected token at 'jQuery18300000644047005491_1545822229753({“success”:true,“code”:0,“results”:[{“productId”

25

Forget about how to parse it for the moment. Is the data you want in the response?

Colin

26

yes

27

Did you ask for JSON data via the Accept request header ? What is the Content-Type response header you get back?

28

So I need to pass in accept request header json as the first parameter to NET::HTTP ?

29

Is that what the docs say?

30

‘application/json’

31

What docs? You asked if I passed in an accept request header so I asked if I need to You’re obscuring this thread

32

The JSON returned is a string How do I parse it into an object?

33

What docs are you talking about?

34

How do I scrape it

35

Yes the sources tab showed the script and it’s output, a string, but how do I parse it into an object?

36

If it is a JSON string then JSON.parse should do it. If it is not then perhaps you need to strip something off to get to be valid JSON. Not sure what any of this has got to do with Rails.

Colin

37

It does of course have to do with rails That’s why I asked here The reason it has to do with rails is because I’m working within the rails console of my app which I’m trying to scrape data for JSON.parse returns invalid token

38

> ... > Yes the sources tab showed the script and it's output, a string, but how do I parse it into an object?

If it is a JSON string then JSON.parse should do it. If it is not then perhaps you need to strip something off to get to be valid JSON. Not sure what any of this has got to do with Rails.

Colin

It does of course have to do with rails That's why I asked here The reason it has to do with rails is because I'm working within the rails console of my app which I'm trying to scrape data for JSON.parse returns invalid token

Dos the string look like valid json to you?

Colin

39

I don’t know what a valid json response is supposed to look like Also I don’t know if it makes a difference they’re using Struts framework

40

Once again I have googled for a link for you

Colin