From: Jonah on 11 Oct 2009 18:08 On Oct 9, 4:25 am, Chris Osborn <chris...(a)gmail.com> wrote: > On Oct 7, 6:08 am, jroge <jr...(a)mac.com> wrote: > > > I am looking for some advice for directly importing data from Google > > Trends? I am able to download the CSV files from Google and then load > > into Mathematica easily enough, but I would rather skip the > > intermediate steps and directly pull the data from Google into my > > Mathematica notebook. I think where I am getting stuck is that Google > > requires authentication and programming that into Mathematica is a bit > > beyond my skill sets. Any thoughts? > > > Jon > > Have you tried "Import?" It allows you to specify a URL; perhaps it > will suffice. > > Chris Hi Jon, I've been working on a way of importing data using mathematica's java functionality (using apache's Httpclient). It worked until a couple weeks ago. It still gives a successful http code response when requesting data but that data is not read correctly. It seems not to be recognizing the data format. I'm not sure what changed... maybe google updated something on its webpage. I've been distracted from working on it lately but I'd like to figure out what's going on. If anybody has any suggestions on how to fix it I'd definitely appreciate the help. Don't forget to change the PASSWORD and EMAIL to correlate to your account. << Jlink` passwordattribute = "PASSWORD"; (*google account password*) emailattribute = "EMAIL(a)gmail.com";(*google email*) (*Initiate Http client*) client = JavaNew["org.apache.commons.httpclient.HttpClient"]; client(a)getHostConfiguration[]@setHost["www.google.com", 443, "https"] (*prepare Request Parameters*) gat3t = JavaNew["org.apache.commons.httpclient.NameValuePair", "GA3T", "ouVrvynsses"]; galx = JavaNew["org.apache.commons.httpclient.NameValuePair", "GALX", "ouVrvynQwdf"]; continue = JavaNew["org.apache.commons.httpclient.NameValuePair", "continue", "http://www.google.com/insights/search/#"]; nui = JavaNew["org.apache.commons.httpclient.NameValuePair", "nui", "1"]; hl = JavaNew["org.apache.commons.httpclient.NameValuePair", "hl", "en-US"]; rmshown = JavaNew["org.apache.commons.httpclient.NameValuePair", "rmShown", "1"]; persistent = JavaNew["org.apache.commons.httpclient.NameValuePair", "PersistentCookie", "yes"]; accountId = JavaNew["org.apache.commons.httpclient.NameValuePair", "Email", emailattribute]; pass = JavaNew["org.apache.commons.httpclient.NameValuePair", "Passwd", passwordattribute]; service = JavaNew["org.apache.commons.httpclient.NameValuePair", "service", "trends"]; nameValuePairs = {gat3t, galx, continue, nui, hl, rmshown, persistent, accountId, pass, service }; postMethod = JavaNew["org.apache.commons.httpclient.methods.PostMethod", "https://www.google.com/accounts/ServiceLoginBoxAuth"]; postMethod(a)setRequestBody[nameValuePairs] client(a)executeMethod[postMethod] postMethod(a)releaseConnection[] (*make request for data*) insights[query_String] := ( redirect = JavaNew["org.apache.commons.httpclient.methods.GetMethod", "http://www.google.com/insights/search/overviewReport?q=" <> query <> "&cmpt=q&content=1&export=1"]; client(a)executeMethod[redirect]; (*get data and drop first 5 lines "header"*) (* data = Drop[ImportString[redirect(a)getResponseBodyAsStream[]], 5];*) data=Drop[ImportString[redirect(a)getResponseBodyAsString[]],5]; insightDateAndData = {DateList[ StringSplit[#[[1]], " - "][[2]]], #[[2]]} & /@ (Take[ data, (Position[data, {""}][[1]])[[1]] - 1]); redirect(a)releaseConnection[]; Return[insightDateAndData] )
First
|
Prev
|
Pages: 1 2 Prev: Image processing with ImageCompose Next: undocumented feature: TableView |