text visible in browser but not in source

You asked the question above ^^ and without looking at the docs I wouldn't know the order of arguments ("parameters") to that library.

And I didn't say you *needed* to do anything, I just asked if you were specifying the content-type you wanted in the request. There's still no guarantee that's what you'll get, but there's a better chance :grinning:

Thanks The response I’m trying to parse is JSONP which is javascript (not json) and has some prepending like /**/jQuery18307882644047003491_1545786199753 so I have to strip it out of the response or make my call to the script without the callback maybe?

Thanks The response I’m trying to parse is JSONP which is javascript (not json) and has some prepending like /**/jQuery18307882644047003491_1545786199753 so I have to strip it out of the response or make my call to the script without the callback maybe?

I think I just need to strip the leading //jQuery18307882633047005491_1545805999753 from the result //jQuery18307882633047005491_1545805999753({“success”:true,“code”:0,“results”:[{“productId”:32817749905,

Anyone can help with the regex?

If you can't work out the regex just find the position of the first { and strip to there.

Colin

I gsub’d for the first ({

json_data = res.body.gsub(/^.+({/, “({”)

json = JSON.parse(json_data)

JSON::ParserError: 765: unexpected token at '({“success”:true,“code”:0,“results”:[{“productId”:32817

I think I just need to strip the leading //jQuery18307882633047005491_1545805999753 from the result //jQuery18307882633047005491_1545805999753({“success”:true,“code”:0,“results”:[{“productId”:32817749905,

Anyone can help with the regex?

If you can’t work out the regex just find the position of the first {

and strip to there.

Colin

I gsub’d for the first ({

json_data = res.body.gsub(/^.+({/, “({”)

json = JSON.parse(json_data)

JSON::ParserError: 765: unexpected token at '({“success”:true,“code”:0,“results”:[{“productId”:32817

Look at the JSON spec again. What should the first char be?

Colin

The first character should be a curly bracket

json_data = res.body.gsub(/^.+{/, “{”).chop

json = JSON.parse(json_data)

JSON::ParserError: 765: unexpected token at '{"pvid":"805ba4ee-6446-4148-ab0a-ff5e51c0ab24","

You have not only taken out the bracket, but you have also changed all the " to \". Did you not notice that yourself?

Colin

No I didn’t notice, thanks

json_data = res.body.gsub(/^.+{/, “{”).chomp(“);”)

json = JSON.parse(json_data)

JSON::ParserError: 765: unexpected token at '{"pvid":"ff43f2dc-3f86-42a3-bfa8-49c117e9da6a","scm-cnt":"1007.13482.95643.0", …

Why have you posted this? You still have \" instead of " As I said, look again at the spec of what a JSON string should look like. Until you have that there is no point feeding into parse()

Colin

Well it seems my response shouldn’t contain the " characters if it’s json so It’s not json it’s js? because i already read that the string response is jsonp which actually contains js, not json

It’s not clear to me what the response is Can the problem be that my request is the full url with callback and other query string data Should I just be calling the script without invoking the Struts framework callback but still provide the item number as query string data

... It's not clear to me what the response is Can the problem be that my request is the full url with callback and other query string data Should I just be calling the script without invoking the Struts framework callback but still provide the item number as query string data

Earlier on you had something that looked like JSON to me, except for the stuff on the front, and you posted

I think I just need to strip the leading /**/jQuery18307882633047005491_1545805999753 from the result /**/jQuery18307882633047005491_1545805999753({"success":true,"code":0,"results":[{"productId":32817749905,

Since then you have done something that is producing \" instead of ". I suggest you go back to whatever you had then.

As an alternative you could try Hassan's advice. Whatever you do make sure you keep sufficient notes that you can always get back to the where you were when you find yourself going backwards.

Colin

That was from the beginning of the response A little further on in the response some quotes are escaped I don’t know why

I would recommend that you put the text you get back from the URL in a file (just now, for testing purposes), so you can experiment with various transforms on it. When you have the raw data in a text file, you can take advantage of your editor's color-coding to see what is what in the raw string.

You're going to have to read through the data and figure out how it works, so you can understand what parts are data and what parts are the callback functions. Read about JSONP and what it is and how and why you would use it. Until you truly understand what the developers have been doing in order to construct their page, you won't know how to pare back the page-building parts and get just the raw data out of the payload.

Once you have done this, though, the very last step (parsing the data with the standard library JSON function) will be a trivial maraschino cherry on top of the sundae.

Until you actually understand the data, you cannot scrape it or parse it. So eat your vegetables first!

Walter

All my requests are drawing ‘Errno::ECONNRESET: Connection reset by peer’ now I don’t know why

Oh ok sorry You mean the Net::HTTP docs? I don’t know I haven’t thought to read them I found this page Turning JSONP callbacks into a Ruby API with Johnson - po-ru.com but don’t understand it

It’s not json it’s javascript so I don’t have to run JSON.parse