From: Arne Vajhøj on 11 Jan 2010 18:11 On 11-01-2010 17:09, Maarten Bodewes wrote: > Arne Vajh�j wrote: >> BTW, even long would be too small for indexes if Java >> will be used after 2074, but somehow I doubt that would >> be the case. And besides we do not have the verylong datatype >> yet. >> > You are expecting memory sizes of 9,223,372,036,854,775,807 bytes???? > > That's 9,223 PETA bytes. Hmm, weird, may happen. But it is certainly > rather large. In 2074 ? Yes ! Arne
From: Roedy Green on 12 Jan 2010 02:36 On Fri, 1 Jan 2010 18:42:36 -0800 (PST), Albretch Mueller <lbrtchx(a)gmail.com> wrote, quoted or indirectly quoted someone who said : >~ > java only uses int indexes which is giving me a hard time since I >need to index values over Integer.MAX_VALUE. Even with a BitSet, that is one heck of a lot of RAM. If the list is sufficiently sparse it may be more efficient to use a HashSet. The docs say you can index with any positive number. Have you got a SSCCE to demonstrate that does not work? See http://mindprod.com/jgloss/sscce.html If it can't handle it, you can create a FatBitSet that uses an array of BitSets to simulate a big BitSet indexed by long. -- Roedy Green Canadian Mind Products http://mindprod.com There is no end to what can be accomplished if you don�t care who gets the credit. ~ Art Rennison
From: Wojtek on 12 Jan 2010 11:16 Arne Vajh�j wrote : > On 11-01-2010 17:09, Maarten Bodewes wrote: >> Arne Vajh�j wrote: >>> BTW, even long would be too small for indexes if Java >>> will be used after 2074, but somehow I doubt that would >>> be the case. And besides we do not have the verylong datatype >>> yet. >>> >> You are expecting memory sizes of 9,223,372,036,854,775,807 bytes???? >> >> That's 9,223 PETA bytes. Hmm, weird, may happen. But it is certainly >> rather large. > > In 2074 ? Yes ! I am curious. Just what WOULD need such a large index? Every sand grain on earth? Every star in every galaxy in the universe? -- Wojtek :-)
From: Lew on 12 Jan 2010 14:08 On Jan 12, 11:16 am, Wojtek <nowh...(a)a.com> wrote: > Arne Vajhøj wrote : > > > On 11-01-2010 17:09, Maarten Bodewes wrote: > >> Arne Vajhøj wrote: > >>> BTW, even long would be too small for indexes if Java > >>> will be used after 2074, but somehow I doubt that would > >>> be the case. And besides we do not have the verylong datatype > >>> yet. > > >> You are expecting memory sizes of 9,223,372,036,854,775,807 bytes???? > > >> That's 9,223 PETA bytes. Hmm, weird, may happen. But it is certainly > >> rather large. > > > In 2074 ? Yes ! > > I am curious. Just what WOULD need such a large index? Every sand grain > on earth? Every star in every galaxy in the universe? > Where will you store the array? Either you have a crapload of RAM (one bit per atom storage density?) or the largest-capacity storage device ever invented (one bit per atom storage density?). What's the average retrieval latency? Even at one bit per atom storage density, it must take even a light beam noticeable time to reach the further reaches of the storage device; anything slower like a semiconductor must take a really long time. A silicon crystal lattice has a lattice spacing of just over half a nanometer, or 5.4 x 10^-10 m. A three-dimensional storage medium for a 9 x 10^18-element array would hold juar over 2 x 10^6 elements to the side. An average access would be halfway in each dimension, or 10^6 elements, which in a silicon lattice is about 5.4 x 10-4 m, times three for a total traversal distance of about 1.6 x 10^-3 m. Each way. For a round trip slightly over 3 x 10^-3 m. A light beam travels that in 0.1 microseconds (10^-7 s). That's about 200 clock cycles of latency on a modern processor, far more on the future processors of 2074. Either we'll find a sparse representation for such arrays, we'll invent much denser storage media and better ways to access them, we'll find some way to keep the processor busy during that latency, or we'll use super-luminal access speeds, perhaps through quantum superposition. -- Lew
From: Lew on 12 Jan 2010 14:12
Lew wrote: > Where will you store the array? Either you have a crapload of RAM > (one bit per atom storage density?) or the largest-capacity storage > device ever invented (one bit per atom storage density?). > > What's the average retrieval latency? Even at one bit per atom > storage density, it must take even a light beam noticeable time to > reach the further reaches of the storage device; anything slower like > a semiconductor must take a really long time. > > A silicon crystal lattice has a lattice spacing of just over half a > nanometer, or 5.4 x 10^-10 m. A three-dimensional storage medium for > a 9 x 10^18-element array would hold juar over 2 x 10^6 elements to > the side. An average access would be halfway in each dimension, or > 10^6 elements, which in a silicon lattice is about 5.4 x 10-4 m, times > three for a total traversal distance of about 1.6 x 10^-3 m. Each > way. For a round trip slightly over 3 x 10^-3 m. A light beam > travels that in 0.1 microseconds (10^-7 s). > Drat! Mixed up my CGS and MKS. That's 10^-5 s, or 10 microseconds. That's about 20,000 > clock cycles of latency on a modern processor, > far more on the future processors of 2074. > > Either we'll find a sparse representation for such arrays, we'll > invent much denser storage media and better ways to access them, we'll > find some way to keep the processor busy during that latency, or we'll > use super-luminal access speeds, perhaps through quantum > superposition. > -- Lew |