Prev: hunchentoot start
Next: (expt 2 #c(2d0 0))?
From: Alan Malloy on 9 May 2010 23:33
I'm trying to represent a hierarchical data structure, one with a lot of
sub-trees which are each identical but with slightly different
modifiers. This seems like a perfect example to design a domain-specific
language for concisely specifying data files (for now, the user can type
these by hand), which I'd like to expand (presumably with macros) into a
tree of fully-specified nodes so that it's easier to determine what
properties a given node has. My plan is to have the user define
"template" sub-trees (which may contain other templates) and then set
their modifiers. A simple example using less obscure object-types:

(template building
(...some specification for buildings))
(template suburb
(neighborhood uptown
(speed-limit 25)
(building house)
(building park))
(neighborhood downtown
(speed-limit 35)
(building strip-mall)))
(template urban-area
(...same kind of stuff as suburb))
(city
(name "New York")
(rent-per-month 10000)
(urban-area))


speed-limit, name, and rent-per-month are properties that should apply
to each sibling and all of their descendant nodes, so that if, after
expansion, I have a reference to a strip-mall building I don't have to
trace up the tree until I find a speed-limit definition. There are a
fixed number of such persistent properties, so they can be hard-coded,
but the definition of how to construct a city is up to the user, and
will involve deeply-nested templates. The final output is just a tree of
buildings (they nest within each other, which is where my "building"
example breaks down) with varying properties.

So I know what I want my data format to look like, but I don't really
know how to get started on defining the macros that will be needed to
transform this into a more-usable in-memory representation. Some sample
output might be a list like:
'(...
(:type park :name "New York Park" :speed-limit 35
:rent-per-month 10000 :neighborhood downtown)...)
#don't have to specify "building", because everything is a building

Can someone give me a push in the right direction?

--
Cheers,
Alan (San Jose, California, USA)
From: Tim X on 10 May 2010 00:40
Alan Malloy <alan.NO.SPAM(a)malloys.org> writes:

> I'm trying to represent a hierarchical data structure, one with a lot of
> sub-trees which are each identical but with slightly different modifiers. This
> seems like a perfect example to design a domain-specific language for
> concisely specifying data files (for now, the user can type these by hand),
> which I'd like to expand (presumably with macros) into a tree of
> fully-specified nodes so that it's easier to determine what properties a given
> node has. My plan is to have the user define "template" sub-trees (which may
> contain other templates) and then set their modifiers. A simple example using
> less obscure object-types:
>
> (template building
> (...some specification for buildings))
> (template suburb
> (neighborhood uptown
> (speed-limit 25)
> (building house)
> (building park))
> (neighborhood downtown
> (speed-limit 35)
> (building strip-mall)))
> (template urban-area
> (...same kind of stuff as suburb))
> (city
> (name "New York")
> (rent-per-month 10000)
> (urban-area))
>
>
> speed-limit, name, and rent-per-month are properties that should apply to each
> sibling and all of their descendant nodes, so that if, after expansion, I have
> a reference to a strip-mall building I don't have to trace up the tree until I
> find a speed-limit definition. There are a fixed number of such persistent
> properties, so they can be hard-coded, but the definition of how to construct
> a city is up to the user, and will involve deeply-nested templates. The final
> output is just a tree of buildings (they nest within each other, which is
> where my "building" example breaks down) with varying properties.
>
> So I know what I want my data format to look like, but I don't really know how
> to get started on defining the macros that will be needed to transform this
> into a more-usable in-memory representation. Some sample output might be a
> list like:
> '(...
> (:type park :name "New York Park" :speed-limit 35
> :rent-per-month 10000 :neighborhood downtown)...)
> #don't have to specify "building", because everything is a building
>
> Can someone give me a push in the right direction?

My advice would be to forget about macros initially. I think trying to
code macros now will be counter productive because I'm not sure you
really have a clear idea or enough understanding of what is needed yet.

From your description, I expect CLOS might be closer to the type of
abstraction you are looking for, but lets leave that aside for now. YOu
can look at that later as, like macros, you will probably appreciate it
better once you have grasped the problem better and gained some
understanding of more basic abstractions. It is possible that later, a
different type of abstraction that is better suited to the problem you
are addressing will become evident and CLOS may not buy you much after all.

I would start by just defining some functions that abstract the
creation/update/printing of your basic data structures. Once you have
these functions defined, you will be in a better position to judge
whether your initial data bastraction is the right one and you will have
a better feel for it. (depending on the application, you may also want
to include a couple of the most common processing functions - its no
good having an abstraction you can create and modify very easily if at
the same time, you have to jump through a million hoops to perform the
basic actions of the application). Once you have done this, you will
have a better feel for things and udnerstand where/how macros could make
things better. I'm assuming your thinking of some kind of DSL to manage
these data elements. Depending on the number/size of the data elements
you plan to work with, you may find structure you have defined above is
inefficient or difficult to manage. For example, even though all nodes
in your structure don't need the same elements or they can have elements
that are easily derived from parent nodes, it may be better to repeat
data elements or have elements default to nil if that makes your code
simpler, clearer and easier to maintain. You may find that rather than
keywords, set positions in the list is more efficient or that a list of
structures is better or .....

The point is that if you define your functions well, it will hide the
underlying structure. You will then be able to change/modify that
structure without breaking your interface. This approach is IMO a more
lispy way to work. At least it is for me. I start with a bit of an idea,
fire up the repl and start coding/experimenting. I don't sit down and
define my data structures in deatil from the start. I tend to start
simple and become more complex as I find a need. I do tend to have a
general idea of what I think is a good starting abstraction and often,
my initial choice turns out to be correct or reasonably correct.
However, I'm always ready to rethink the problem and if necessary, try a
new approach. If I find the code appears to be getting overly complex
when doing what should be fairly simple and straight-forward things, it
generally indicates there is a problem with my abstraction and I need to
step back, think for a few minutes and start some new experiments.

HTH

Tim

--
tcross (at) rapttech dot com dot au
From: Paul Donnelly on 10 May 2010 04:37
Alan Malloy <alan.NO.SPAM(a)malloys.org> writes:

> So I know what I want my data format to look like, but I don't really
> know how to get started on defining the macros that will be needed to
> transform this into a more-usable in-memory representation. Some
> sample output might be a list like:
> '(...
> (:type park :name "New York Park" :speed-limit 35
> :rent-per-month 10000 :neighborhood downtown)...)
> #don't have to specify "building", because everything is a building
>
> Can someone give me a push in the right direction?

If it's a data file you're working with, you don't need macros. Just
read the data then construct appropriate structures... or perhaps use
structures and read them directly.

I'm not really sure what you're trying to do, but could you just keep a
list of templates defined, then substitute them into your data
structures as you read them? Something like this no doubt buggy code.

(defun process (data)
(let (templates)
(reverse (let (o)
(dolist (d data o)
(case (car d)
(template (push (cdr d) templates))
(t (push (expand d) o))))))))

(defun expand (data)
(loop for d in data
collect (if (atom d)
d
(or (cdr (assoc (car d) *templates*))
d))))

(process '((template foo c) (hi) (city (foo))))
=> ((HI) (CITY (C)))
From: Pascal J. Bourguignon on 10 May 2010 04:42
Tim X <timx(a)nospam.dev.null> writes:

> Alan Malloy <alan.NO.SPAM(a)malloys.org> writes:
>
>> I'm trying to represent a hierarchical data structure, one with a lot of
>> sub-trees which are each identical but with slightly different modifiers. This
>> seems like a perfect example to design a domain-specific language for
>> concisely specifying data files (for now, the user can type these by hand),
>> which I'd like to expand (presumably with macros) into a tree of
>> fully-specified nodes so that it's easier to determine what properties a given
>> node has. My plan is to have the user define "template" sub-trees (which may
>> contain other templates) and then set their modifiers. A simple example using
>> less obscure object-types:
>>
>> (template building
>> (...some specification for buildings))
>> (template suburb
>> (neighborhood uptown
>> (speed-limit 25)
>> (building house)
>> (building park))
>> (neighborhood downtown
>> (speed-limit 35)
>> (building strip-mall)))
>> (template urban-area
>> (...same kind of stuff as suburb))
>> (city
>> (name "New York")
>> (rent-per-month 10000)
>> (urban-area))
>>
>>
>> speed-limit, name, and rent-per-month are properties that should apply to each
>> sibling and all of their descendant nodes, so that if, after expansion, I have
>> a reference to a strip-mall building I don't have to trace up the tree until I
>> find a speed-limit definition. There are a fixed number of such persistent
>> properties, so they can be hard-coded, but the definition of how to construct
>> a city is up to the user, and will involve deeply-nested templates. The final
>> output is just a tree of buildings (they nest within each other, which is
>> where my "building" example breaks down) with varying properties.
>>
>> So I know what I want my data format to look like, but I don't really know how
>> to get started on defining the macros that will be needed to transform this
>> into a more-usable in-memory representation. Some sample output might be a
>> list like:
>> '(...
>> (:type park :name "New York Park" :speed-limit 35
>> :rent-per-month 10000 :neighborhood downtown)...)
>> #don't have to specify "building", because everything is a building
>>
>> Can someone give me a push in the right direction?
>
> I would start by just defining some functions that abstract the
> creation/update/printing of your basic data structures. Once you have
> these functions defined, you will be in a better position to judge
> whether your initial data bastraction is the right one and you will have
> a better feel for it.

Indeed. Macros would be useful only if you wanted to integrate these
data definition (the templates) and data manipulation (the "insert" data
declarations) languages into normal lisp code.

If you keep these data in separate files, or separate input streams
(ie. user input), then you can merely define functions to read and
process them:

(defun load-area-data (file &key verbose print
(if-does-not-exist :error)
(external-format :default))
...
(loop
...
collect (read-area-data stream)
...))


(defun read-area-data (&optional (input-stream *standard-input*)
(eof-error-p t) eof-value)
(let* ((my-eof-value (load-time-value (gensym)))
(form (read stream eof-error-p my-eof-value)))
(if (eq form my-eof-value)
eof-value
(eval-area-data form))))

(defvar *templates* '() "The collection of templates defined so far.")

(defun eval-area-data (form)
(if (atom form)
(error "~S cannot be an Area data expression.~%" form)
(case (first form)
((template) (destructuring-bind (op ...) form
...
(register-template ...)))
(otherwise
;; parse the form to find called up templates
;; and build the data objects.
...))))




--
__Pascal Bourguignon__
From: Alan Malloy on 10 May 2010 05:49
Pascal J. Bourguignon wrote:
> Tim X <timx(a)nospam.dev.null> writes:
>
>> Alan Malloy <alan.NO.SPAM(a)malloys.org> writes:
>>
>>> I'm trying to represent a hierarchical data structure, one with a lot of
>>> sub-trees which are each identical but with slightly different modifiers. This
>>> seems like a perfect example to design a domain-specific language for
>>> concisely specifying data files (for now, the user can type these by hand),
>>> which I'd like to expand (presumably with macros) into a tree of
>>> fully-specified nodes so that it's easier to determine what properties a given
>>> node has. My plan is to have the user define "template" sub-trees (which may
>>> contain other templates) and then set their modifiers. A simple example using
>>> less obscure object-types:
>>>
>>> (template building
>>> (...some specification for buildings))
>>> (template suburb
>>> (neighborhood uptown
>>> (speed-limit 25)
>>> (building house)
>>> (building park))
>>> (neighborhood downtown
>>> (speed-limit 35)
>>> (building strip-mall)))
>>> (template urban-area
>>> (...same kind of stuff as suburb))
>>> (city
>>> (name "New York")
>>> (rent-per-month 10000)
>>> (urban-area))
>>>
>>>
>>> speed-limit, name, and rent-per-month are properties that should apply to each
>>> sibling and all of their descendant nodes, so that if, after expansion, I have
>>> a reference to a strip-mall building I don't have to trace up the tree until I
>>> find a speed-limit definition. There are a fixed number of such persistent
>>> properties, so they can be hard-coded, but the definition of how to construct
>>> a city is up to the user, and will involve deeply-nested templates. The final
>>> output is just a tree of buildings (they nest within each other, which is
>>> where my "building" example breaks down) with varying properties.
>>>
>>> So I know what I want my data format to look like, but I don't really know how
>>> to get started on defining the macros that will be needed to transform this
>>> into a more-usable in-memory representation. Some sample output might be a
>>> list like:
>>> '(...
>>> (:type park :name "New York Park" :speed-limit 35
>>> :rent-per-month 10000 :neighborhood downtown)...)
>>> #don't have to specify "building", because everything is a building
>>>
>>> Can someone give me a push in the right direction?
>> I would start by just defining some functions that abstract the
>> creation/update/printing of your basic data structures. Once you have
>> these functions defined, you will be in a better position to judge
>> whether your initial data bastraction is the right one and you will have
>> a better feel for it.
>
> Indeed. Macros would be useful only if you wanted to integrate these
> data definition (the templates) and data manipulation (the "insert" data
> declarations) languages into normal lisp code.
>
> If you keep these data in separate files, or separate input streams
> (ie. user input), then you can merely define functions to read and
> process them:
>
> (defun load-area-data (file &key verbose print
> (if-does-not-exist :error)
> (external-format :default))
> ...
> (loop
> ...
> collect (read-area-data stream)
> ...))
>
>
> (defun read-area-data (&optional (input-stream *standard-input*)
> (eof-error-p t) eof-value)
> (let* ((my-eof-value (load-time-value (gensym)))
> (form (read stream eof-error-p my-eof-value)))
> (if (eq form my-eof-value)
> eof-value
> (eval-area-data form))))
>
> (defvar *templates* '() "The collection of templates defined so far.")
>
> (defun eval-area-data (form)
> (if (atom form)
> (error "~S cannot be an Area data expression.~%" form)
> (case (first form)
> ((template) (destructuring-bind (op ...) form
> ...
> (register-template ...)))
> (otherwise
> ;; parse the form to find called up templates
> ;; and build the data objects.
> ...))))
>
>
>
>

Hmmm, thanks. I had somehow gotten the idea that the idiomatic way to
use sexps to store data in lisp is to have appropriate macros defined so
that when you read/eval the "data", it is transformed into code that
creates the in-memory data structure you want. I've written a few simple
I/O functions for reading basic text data, and if the "right" way to do
this is to read my tree (or other abstraction, as Paul points out)
through a set of actual functions it should be a fair bit easier for me
to grasp anyway.

--
Cheers,
Alan (San Jose, California, USA)
 |  Next  |  Last
Pages: 1 2 3
Prev: hunchentoot start
Next: (expt 2 #c(2d0 0))?