{"success":true,"code":0,"results":[{"productId":32815555905, ... Looks like JSON to me (embedded in js admittedly). You said that the data you want is in that string. If that is correct then all you have to do is to extract it and parse it as JSON.
Colin
It’s not json it’s javascript
What is it about the string {"success":true,"code":0,"results":[{"productId":32815555905, ... that makes it not JSON?
Colin
You mean the Net::HTTP docs? I don't know I haven't thought to read them
I found this page Turning JSONP callbacks into a Ruby API with Johnson - po-ru.com but don't understand it
That's an interesting approach though `johnson` is abandoned; you could try using `therubyracer` instead.
I don’t know I’m repeating what I read somewhere else about jsonp actually
The escaped quotes further down the string, you see i left it ending in … there’s more to it i didn’t include it’s a long string when the parser reaches the first backslash it returns an error
Now I only get connection reset by peer when I try to make the request
> > > >> >> > ... >> > It's not json it's javascript so I don't have to run JSON.parse >> >> {"success":true,"code":0,"results":[{"productId":32815555905, ... >> Looks like JSON to me (embedded in js admittedly). You said that the >> data you want is in that string. If that is correct then all you have >> to do is to extract it and parse it as JSON. >> >> Colin > > > It's not json it's javascript
What is it about the string {"success":true,"code":0,"results":[{"productId":32815555905, ... that makes it not JSON?
Colin
The escaped quotes further down the string, you see i left it ending in ... there's more to it i didn't include it's a long string when the parser reaches the first backslash it returns an error
That probably means you have dropped off the end of the json onto something else, examine that area to see. Is all the data you need in the bit before that?
Colin
Can't help you there, presumably either the website has changed or you have changed the way you are fetching it. Try the url in a browser.
Colin
Yes the requests still work in a browser Maybe I just need to kill Net::HTTP process Let me try to reboot Please stand by
If you get “Connection reset by peer” while scraping a website, it is very likely that your scraping attempts were detected and automatically blocked for a while. They might have also been alerted by all of these malformed requests.
Consider pacing out your requests so that it doesn’t look like you are scraping.
Can you give me an idea how to extract it
Are you seriously asking, after the years that you have been using Ruby, that you don't know how to extract a particular section from a string? Apart from anything else, since I haven't got access to the full string (nor do I want it), I have no idea what the details surrounding the JSON section are so can't tell you how to do it.
Colin
Hi,
Can anyone help how to work this projects
It looks like this:
/**/jQuery18308525223902976695_1548819829968({“success”:true,“code”:0,“results”:[{“productId”:32755997022,“sellerId”:201591356,“oriMinPrice”:"US $275.00 … });
JSON.parse and cxt.eval both return errors even if I remove everything up to the first { and then chomp the trailing )
…
Can you give me an idea how to extract it
Are you seriously asking, after the years that you have been using
Ruby, that you don’t know how to extract a particular section from a
string? Apart from anything else, since I haven’t got access to the
full string (nor do I want it), I have no idea what the details
surrounding the JSON section are so can’t tell you how to do it.
Colin
It looks like this:
/**/jQuery18308525223902976695_1548819829968({“success”:true,“code”:0,“results”:[{“productId”:32755997022,“sellerId”:201591356,“oriMinPrice”:"US $275.00 … });
JSON.parse and cxt.eval both return errors even if I remove everything up to the first { and then chomp the trailing )
Have you checked that the trailing } you show there is the one that matches the { you show? Perhaps there is more than one JSON string there.
If you have checked that then show us where the error occurs. Presumably you have already looked at that though and have not found what I suggest.
Colin
parsed_obj=JSON.parse( res.body.gsub(/^.+(/,“”).chomp(“);”) )
puts parsed_obj
that gets me => true
I can print the outer values of parsed_obj with an iteration loop
parsed_obj.each |key, val| do
can you tell me how do i select individual keys within the nested hash ‘results’
Does that mean you have parsed it ok?
Isn't it obvious? You have to use each|key,val| on the outer level results to get their contents. With different names for key and val obviously.
Colin
I found this extended discussion on different ways of accessing hash values here hash - Accessing elements of nested hashes in ruby - Stack Overflow
How would I get results->productId