Prev: t_self as system column
Next: 64-bit pgbench V2
From: Greg Smith on 5 Jul 2010 19:48
Attached is an updated second rev of the patch I sent a few months ago,
to expand pgbench to support database scales larger than around
4,294--where the 32-bit integer for the account number overflows in the
current version. The current limit makes for about a 60GB database.
Last week I ran this on a system with 72GB of RAM, which are already
quite common, and wasn't able to get a test that didn't fit in RAM.
Without a bug fix here I am concerned that pgbench will ship in 9.0
already obsolete for the generation of hardware is it going to be
deployed on.

The main tricky part was figuring how to convert the \setshell
implementation. That uses strtol to parse the number that should have
been returned by the shell call. It turns out there are a stack of ways
to do something similar but return 64 bits instead:

* strtoll is defined by ISO C99
* strtoq was used on some earlier BSD systems
* MSVC has _strtoi64 for signed and _strtoui64 for unsigned 64bit integers

According to the glib docs at
http://www.gnu.org/software/gnulib/manual/html_node/strtoll.html ,
stroll is missing on HP-UX 11, OSF/1 5.1, Interix 3.5, so one of the
HP-UX boxes might be a useful testbed for what works on a trickier platform.

For prototype purposes, I wrote the patch to include some minimal logic
to map the facility available to strtoint64, falling back to the 32-bit
strtol if that's the best available. There are three ways I could
forsee this going:

1) Keep this ugly bit of code isolated to pgbench
2) Move it to src/include/c.h where the other 64-bit int abstraction is done
3) Push the problem toward autoconf

I don't have a clear argument for or against those individual options,
they all seem reasonable from some perspectives.

The only open issue I'm not sure about is whether the situation where
the code falls back to 32-bits should be documented, or even a warning
produced if you create something at a scale without some strtoll
available. Given that it only impacts the \setrandom case, it's not
really a disaster that it might not work, so long as there's
documentation explaining the potential limitations. I'll write those if
necessary, but I think that some testing on known tricky platforms that
I don't have setup here is the best next step, so I'm looking for
feedback on that.

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg(a)2ndQuadrant.com www.2ndQuadrant.us

 | 
Pages: 1
Prev: t_self as system column
Next: 64-bit pgbench V2