lightweight database
in [Lisp]
|
From: Alex Mizrahi on 16 Mar 2010 15:46 ??>> cl-prevalence doesn't support queries (last time I've checked), you ??>> have to do this stuff yourself. RC> The whole idea of cl-prevalence is that the database and the in-memory RC> lisp objects are one and the same. In other words, cl-prevalence provides almost no benefits to no database at all. That's the idea. RC> You do "queries" using ordinary common lisp introspection - e.g., RC> reading an object slot, doing a (gethash 'some-key *my-hash*), or RC> executing a function that searches your in-memory data according to RC> some predicate, etc. This is what I meant by "lightweight." Tamas wrote: " and I want to be able to do simple queries using CL (eg get all the records between two timestamps). " Hash-table cannot do this. You need a data structure like a binary tree to do this. There are no such data structures in CL standard, so you'll have to use some library for it. And last time I've checked all CL libraries for this totally sucked... RC> Could this be done by writing stuff to text files? Yes, but you'd have RC> to write and test the infrastructure to write and read clos objects, RC> hash tables, etc. to text files. Yeah, one might want to use cl-prevalence for its serialization, but Tamas wrote:"The database consists of timestamp/count pairs, a timestamp is a date&time, while the count is a single integer". This stuff can be serialized by CL natively, so I don't see how cl-prevalence is relevant. RC> Quite convenient if you want fast, lightweight persistence and the RC> ability to have lisp run the show, not a database query language. Maybe I'm missing something, but this: (execute system (make-transaction 'tx-add-number candidate)) doesn't look particularly convenient to me. Neither does this: (defmethod (setf get-firstname) (value (person person)) (execute-transaction (tx-change-object-slots *test-system* 'person (get-id person) (list (list 'firstname value))))) (This is from cl-prevalence test/test-managed-prevalence.lisp). If we compare this to Elephant, with Elephant you can just forget about persistence most of the time -- Elephant handles slot reads and writes via metaclasses, so you can work with objects as if they were normal CLOS objects. While in cl-prevalence it is all over the place. So I'd say that Elephant is less intrusive. RC> Is it large scale, industrial strength? "Large scale", "industrial strength" is very vague. Real question is what cl-prevalence can give you and when it makes sense to use it. Without a doubt, there are situations where it can be very useful. But it is very special kind of beast and one should carefully consider whether it makes sense to use it or not. RC> Maybe not, but its author, Sven Van Caekenberghe, used it to run a RC> ticketing website for a theater, so RC> it is known to be usable for a small scale commercial system... I'm pretty sure text files can be usable for small scale commercial systems too. :)
From: Tamas K Papp on 16 Mar 2010 16:13 On Tue, 16 Mar 2010 21:46:19 +0200, Alex Mizrahi wrote: > ??>> cl-prevalence doesn't support queries (last time I've checked), you > ??>> have to do this stuff yourself. > > RC> The whole idea of cl-prevalence is that the database and the > in-memory RC> lisp objects are one and the same. > > In other words, cl-prevalence provides almost no benefits to no database > at all. That's the idea. > > RC> You do "queries" using ordinary common lisp introspection - e.g., > RC> reading an object slot, doing a (gethash 'some-key *my-hash*), or > RC> executing a function that searches your in-memory data according to > RC> some predicate, etc. This is what I meant by "lightweight." > > Tamas wrote: " and I want to be able to do simple queries using CL (eg > get all the records between two timestamps). " > > Hash-table cannot do this. You need a data structure like a binary tree > to do this. > There are no such data structures in CL standard, so you'll have to use > some library for it. > And last time I've checked all CL libraries for this totally sucked... Dear Alex and others, Thanks for taking the time to understand my requirements. It seems that SQLite is the best solution for me, especially because the application generating the data may not be written in CL at all, so having a lightweight database could provide a common interface. I just need to learn a bit of SQL, but I have always wanted to do that. Thanks, Tamas
From: vanekl on 16 Mar 2010 18:15 Alex Mizrahi wrote: > Tamas wrote: " and I want to be able to do simple queries using CL > (eg get all the records between two timestamps). " > > Hash-table cannot do this. You need a data structure like a binary > tree to do this. > There are no such data structures in CL standard, so you'll have to > use some library for it. Because hash-map/reduce doesn't exist. Oh, wait... (defun {} (&rest pairs) "Syntax sugar to make hash tables." (let ((h (make-hash-table :test 'equal))) (loop for (key value) on pairs by #'cddr do (setf (gethash key h) value)) h)) (defun between (i lower-bound upper-bound) "Is number 'i' between the two bounds (inclusive)?" (and (<= lower-bound i) (<= i upper-bound))) (let* ((db ({} 1 "one" 2 "two" 3 "three" 4 "four" 5 "five")) search-results) (time (dotimes (idx 10000) (setq search-results (iter (for (key val) in-hashtable db) (when (between key 2 3) (collect key)))))) (iter (for answer in search-results) (print answer))) Evaluation took: 0.010 seconds of real time 0.010000 seconds of total run time (0.010000 user, 0.000000 system) 100.00% CPU 11,524,904 processor cycles 159,744 bytes consed 2 3 NIL
From: Alain Picard on 16 Mar 2010 22:21 Tamas K Papp <tkpapp(a)gmail.com> writes: > Thanks for taking the time to understand my requirements. It seems > that SQLite is the best solution for me, especially because the > application generating the data may not be written in CL at all, so > having a lightweight database could provide a common interface. I > just need to learn a bit of SQL, but I have always wanted to do that. Be careful - if you need simultaneous inserts from your other process into the same database that CL is reading via SQLite, you'll need to be prepared with some inserts failing returning SQLITE_BUSY. Check that whatever interface you're using handles that appropriately. Another possibility for you is to use Elephant with the Berkeley DB engine, or just Berkeley DB with one of the simple, direct interfaces, if you can live with the Berkeley DB license. There, you'd just work with strings, and not need to learn any SQL, etc. Good luck!
From: Dmitry Kalyanov on 16 Mar 2010 22:56
On Mar 16, 5:22 pm, Tamas K Papp <tkp...(a)gmail.com> wrote: > Hi, > > I need to use a simple database from CL. The database consists of > timestamp/count pairs, a timestamp is a date&time, while the count is a > single integer. The idea is that a server which runs the database > collects the data continuously (not necessarily using CL, but that can be > achieved), and I want to be able to do simple queries using CL (eg get > all the records between two timestamps). I am looking for something > that is > > 1. robust, > 2. simple to setup (would prefer to avoid SQL) > 3. has reasonably mature CL bindings, or makes it easy to implement them. > > I have no experience with these things, so any help would be appreciated. > > Thanks, > > Tamas SQLite is a lightweight robust database with good CL bindings (cl- sqlite). |