Map vs. Table
in [Mathematica]
|
Prev: label graph vertices
Next: Mathematica 6.0 - Legend
From: Bill Rowe on 28 Apr 2007 06:07 On 4/27/07 at 5:25 AM, replicatorzed(a)gmail.com (zac) wrote: >consider a set of data (y values): >data = Table[i*j, {i, 5}, {j, 10}] The same data can be generated more quickly with Outer. That is: In[15]:= Outer[Times,Range@5,Range(a)10]==Table[i*j,{i,5},{j,10}] Out[15]= True and In[16]:= Timing[Do[Table[i*j,{i,5},{j,10}],{10000}];] Out[16]= {0.947914 Second,Null} while In[17]:= Timing[Do[Outer[Times,Range@5,Range(a)10],{10000}];] Out[17]= {0.451616 Second,Null} >I want to label each Integer with an x value: >label := Table[{i, #} & /@ data[[i]], {i, Length[data]}] =46or this, MapIndexed is works nicely, i.e., In[18]:= Join@@MapIndexed[Outer[List,#2,#1]&,Outer[Times,Range@5,Range@ 10]]==Table[{i,#}&/@data[[i]],{i,Length[data]}] Out[18]= True and In[19]:= Timing[Do[Join@@MapIndexed[Outer[List,#2,#1]&,data],{10000}];] Out[19]= {0.557445 Second,Null} -- To reply via email subtract one hundred and four
From: Jens-Peer Kuska on 28 Apr 2007 06:12 Hi, try it out data = Table[i, {i, 1, 1000000}]; Timing[Table[{i, #} & /@ data[[i]], {i, Length[data]}]][[1]] gives 2.074 Seconds and Timing[ MapIndexed[{#2[[1]], #} &, data] ][[1]] gives 1.732 Seconds Regards Jens zac wrote: > Dear Group, > > consider a set of data (y values): > > data = Table[i*j, {i, 5}, {j, 10}] > > I want to label each Integer with an x value: > > label := Table[{i, #} & /@ data[[i]], {i, Length[data]}] > > I've understand that list creation is much faster with Map than with > Table. > Is there an effective way to convert the Table to Map in the label > function (either with nested Map or to incorporate the Table function > into the Map)? > Would it be really faster for very large datasets than with Table? > > thanks > > Istvan Zachar > >
From: Szabolcs on 28 Apr 2007 06:17 zac wrote: > Dear Group, > > consider a set of data (y values): > > data = Table[i*j, {i, 5}, {j, 10}] > > I want to label each Integer with an x value: > > label := Table[{i, #} & /@ data[[i]], {i, Length[data]}] > It is better to use Set instead of SetDelayed here, otherwise the Table will be re-evaluated every time you use label. (Use = instead of :=) > I've understand that list creation is much faster with Map than with > Table. It is generally a good idea to avoid explicitly indexing lists when not necessary. But here you *do* need the index i, as a label. > Is there an effective way to convert the Table to Map in the label > function (either with nested Map or to incorporate the Table function > into the Map)? > > Would it be really faster for very large datasets than with Table? > > thanks > > Istvan Zachar > There are many ways to do this calculation. If you want to avoid indexing data, then you either need to use MapIndexed or pre-generate the labels. I tried these: In[46]:= data=Table[i j, {i, 2000}, {j, 2000}]; In[47]:= Timing[label=Table[{i,#}& /@ data[[i]], {i, Length[data]}];] Out[47]= {1.031 Second,Null} In[48]:= Timing[label=MapIndexed[First(a)Outer[List, #2, #1]&, data];] Out[48]= {1.407 Second,Null} In[49]:= Timing[label=MapIndexed[Thread[Append[#2, #1]]&, data];] Out[49]= {1.609 Second,Null} In[50]:= Timing[label=Thread/@Transpose[{Range(a)Length@data, data}];] Out[50]= {1.422 Second,Null} You must not take these timings too seriously (the times are too short, and inconsistent with every run), though they suggest that using Table is faster then the rest. The problem is that with larger data matrices only Table works, all the others run out of memory! I guess that this is because data is stored as a packed array and the Table approach is the only one that doesn't break this internal representation. In[51]:= <<Developer` PackedArrayQ[data] Out[52]= True PackedArrayQ[label] is only true when label is generated by Table. So I tried to come up with an approach that is similarly simple to Table, and did this: label = Transpose(a)Inner[List, Range(a)Length@data, data, List]; But to my greatest disappointment it turned out to be the slowest of all (and also breaks the packed array representation). So probably the Table solution is the best to use (because it needs the least memory). It is also the most straightforward and most readable way of doing this. It seems that a strictly functional style is not always the best way do things in Mathematica. Can anyone come up with a good way of solving this problem without indexing data? Szabolcs
From: Szabolcs on 29 Apr 2007 03:18 Bill Rowe wrote: > On 4/27/07 at 5:25 AM, replicatorzed(a)gmail.com (zac) wrote: > >> consider a set of data (y values): > >> data = Table[i*j, {i, 5}, {j, 10}] > > The same data can be generated more quickly with Outer. That is: > > In[15]:= > Outer[Times,Range@5,Range(a)10]==Table[i*j,{i,5},{j,10}] > Just a note about this: I would also have chosen Outer to generate data, but for some reason this doesn't give a packed array. In[5]:= data=Outer[Times,Range(a)1000,Range(a)1000]; In[6]:= PackedArrayQ(a)data Out[6]= False > > and > > In[16]:= > Timing[Do[Table[i*j,{i,5},{j,10}],{10000}];] > > Out[16]= > {0.947914 Second,Null} > > while > > In[17]:= > Timing[Do[Outer[Times,Range@5,Range(a)10],{10000}];] > > Out[17]= > {0.451616 Second,Null} > Yes, but it also depends on the size of the matrix. In[10]:= Timing[Do[Table[i j,{i,1000},{j,1000}],{20}];] Out[10]= {2.125 Second,Null} In[11]:= Timing[Do[Outer[Times,Range(a)1000,Range(a)1000],{20}];] Out[11]= {8.391 Second,Null} I hope that these deficiencies of Outer and Inner (i.e that they don't work with packed arrays) will be corrected in a future Mathematica. As Jean-Marc's reply also demonstrates, it is not always easy to predict the efficiency of different approaches. Are there any published guidelines available on how to write efficient code in Mathematica? Szabolcs
From: János on 1 May 2007 03:35
On Apr 28, 2007, at 6:04 AM, Szabolcs wrote: > zac wrote: >> Dear Group, >> >> consider a set of data (y values): >> >> data = Table[i*j, {i, 5}, {j, 10}] >> >> I want to label each Integer with an x value: >> >> label := Table[{i, #} & /@ data[[i]], {i, Length[data]}] >> > > It is better to use Set instead of SetDelayed here, otherwise the > Table > will be re-evaluated every time you use label. (Use = instead of :=)= > >> I've understand that list creation is much faster with Map than with >> Table. > > It is generally a good idea to avoid explicitly indexing lists when = > not > necessary. But here you *do* need the index i, as a label. > >> Is there an effective way to convert the Table to Map in the label >> function (either with nested Map or to incorporate the Table function >> into the Map)? >> >> Would it be really faster for very large datasets than with Table? >> >> thanks >> >> Istvan Zachar >> > > There are many ways to do this calculation. If you want to avoid > indexing data, then you either need to use MapIndexed or pre-generate > the labels. > > I tried these: > > In[46]:= > data=Table[i j, {i, 2000}, {j, 2000}]; > > In[47]:= > Timing[label=Table[{i,#}& /@ data[[i]], {i, Length[data]}];] > > Out[47]= > {1.031 Second,Null} > > In[48]:= > Timing[label=MapIndexed[First(a)Outer[List, #2, #1]&, data];] > > Out[48]= > {1.407 Second,Null} > > In[49]:= > Timing[label=MapIndexed[Thread[Append[#2, #1]]&, data];] > > Out[49]= > {1.609 Second,Null} > > In[50]:= > Timing[label=Thread/@Transpose[{Range(a)Length@data, data}];] > > Out[50]= > {1.422 Second,Null} > > You must not take these timings too seriously (the times are too > short, > and inconsistent with every run), though they suggest that using Table > is faster then the rest. > > The problem is that with larger data matrices only Table works, all = > the > others run out of memory! I guess that this is because data is > stored as > a packed array and the Table approach is the only one that doesn't > break > this internal representation. > > In[51]:= > <<Developer` > PackedArrayQ[data] > > Out[52]= > True > > PackedArrayQ[label] is only true when label is generated by Table. > > So I tried to come up with an approach that is similarly simple to > Table, and did this: > > label = Transpose(a)Inner[List, Range(a)Length@data, data, List]; > > But to my greatest disappointment it turned out to be the slowest > of all > (and also breaks the packed array representation). > > So probably the Table solution is the best to use (because it needs = > the > least memory). It is also the most straightforward and most > readable way > of doing this. It seems that a strictly functional style is not always > the best way do things in Mathematica. > > Can anyone come up with a good way of solving this problem without > indexing data? > > Szabolcs I do not have a concrete example here but my newbie instinct says that using Dispatch might be of use here to catch the speed daemon :) J=E1nos= |