Testing with concurrent sessions
in [PgSql]
|
From: Markus Wanner on 19 Jan 2010 16:34 Hi, Jan Urbański wrote: > sorry to butt in to the conversation, but I have spent some time > wrapping/refining the concepts in dtester, and the results are here: > > http://git.wulczer.org/?p=twisted-psql.git;a=summary That seems to cover the concurrent psql part of dtester. But I don't see how that's wrapping or refining dtester. > I borrowed the idea of wrapping a psql in a Twisted protocol and added a > Deferred interface around it, which made it possible to run tests with > trial: the Twisted unit testing framework. dtester works pretty much the same way, except that it doesn't use trial. But the Deferred interface is about the same. > As a developer of a failry large Python system based on Twisted, that > sports hundreds of trial-based tests, I very strongly recommend trial > for asynchronous unit testing. It handles lots of boring details, is > well maintained and Twisted itself is simply designed to do asynchronous > programming. As an added bonus, the runnning and reporting > infrastructure is already there, you just write the tests. I'm using trial for two other projects as well and I've started with it for Postgres. IMO trial is very good for unit tests, but fails horribly for anything that involves complex setups and watch-guarding of multiple processes. That's where dtester shines. > My code is very rough and lacks good error reporting, for instance > failed tests will probably result in a "test hung" and the need to > Ctrl+C, but that can be easily improved. Don't underestimate that. There are lots of sources of errors. Not having a sentinel over the postmaster itself, unit tests from trial simply fail to notice errors there. (And coding for Postgres, these are the ones you are interested in). Now combine all of that and error handling and (useful) reporting is *the* major challenge in automating testing. > A thing that would help > tremendously would be a real Twisted protocol that talks to PG on the > protocol level, not by parsing psql output (which is very clumsy and > error prone IMHO). I agree. (Well, now I do). > I found one such project: > http://www.jamwt.com/pgasync/ > but it had some issues with committing Yeah, I didn't like that one, either. Regards Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: "Kevin Grittner" on 12 Feb 2010 14:58 [off-list to avoid distracting from the 9.0 wrap-up effort] Markus Wanner <markus(a)bluegap.ch> wrote: > Quoting "Kevin Grittner" <Kevin.Grittner(a)wicourts.gov>: >> I strongly encourage you to set that up on git.postgresql.org. > > I'm about to provide git repositories for Postgres-R anyway, so > I've setup two projects on git.postgres-r.org: > > dtester: that's the driver/harness code > postgres-dtests: a Postgres clone with the dtester patch applied - > this is based on the Postgres git repository, so you can easily > switch between Postgres branches. I just got to the point of having what appears to be a working but poorly optimized version of serializable transactions, so it is critical that I create a good set of tests to confirm correct behavior and monitor for regressions as I optimize. I see that you've been working on dtester recently -- should I grab what you've got here, stick with 0.1, or do you want to package something? If I should pull from your git, any hints on the best git statements to merge that in are welcome, I'm still rather new to git, and I tend not to get things right on a first try without some hints. :-/ Thanks again for this tool! -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: "Kevin Grittner" on 12 Feb 2010 15:00 I wrote: > [off-list to avoid distracting from the 9.0 wrap-up effort] Arg. I really didn't mean that to go to the list. :-( -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Markus Wanner on 13 Feb 2010 11:39
Hi, Kevin Grittner wrote: > I just got to the point of having what appears to be a working but > poorly optimized version of serializable transactions, Cool. > so it is > critical that I create a good set of tests to confirm correct > behavior and monitor for regressions as I optimize. I see that > you've been working on dtester recently -- should I grab what you've > got here, stick with 0.1, or do you want to package something? I'm preparing for a 0.1 release, yes. However, I'm trying to keep up with the postgres-dtests branch, so that should continue to work with the latest dtester code. > If I > should pull from your git, any hints on the best git statements to > merge that in are welcome, I'm still rather new to git, and I tend > not to get things right on a first try without some hints. :-/ I'd recommend merging postgres-dtests into your branch, so you'll get all of the adjustments from there, if I change dtester itself. Once 0.1 is out, I'm *trying* to remain backwards compatible in future releases. However, that holds me from releasing 0.1 in the first place :-) Independent of versions and releases: if you build your tests on top of dtester, I'm more than willing to support you with that. If you keep your code in a git repository, I could even provide patches, in case I need (or want) to change the dtester interface. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |