Prev: Is LISP divine?
Next: keywords & macros
From: Tim X on 16 Apr 2010 00:25
Jochen Schmidt <js(a)crispylogics.com> writes:

> On 2010-04-14 00:29:34 +0200, Juanjo said:
>
>> That said, I have read XCVB's manual and I feel it introduces an
>> unneeded level of complexity in it all. Parallel compilation is nice,
>> but forcing the use of makefiles, permitting the developers to scatter
>> the dependencies of their software over different files, and relying
>> on external drivers, plus the size of the software itself -- ASDF
>> could and should be so lean! --, does not really convince me.
>
> That was also my feelings when I investigated it. I quickly came to the
> conclusion "XCVB" must stand for "Xtra Complicated Variant of a Buildsystem".
> Actually I think the idea behind XCVB is good, but I fear that the majority of
> lisp projects do not face the complexities which triggered it. More, I think a
> successor to what ASDF provides today should really be as simple as possible.
> A separation of concerns would certainly a step in the right direction.
>
> A system in its minimal core doesn't need any inherent facilities to build it
> or install it or package it - it should only be a straightforward description
> which allows others to implement stuff like building, installing and so on ON
> TOP of it. It should be readable by lisp but NOT mandate a complete
> implementation of CL to interpret it. It shouldn't even be mandatory to write
> system descriptions by hand -
> there could easily be generators and IDE support to do this.
>
> To me the most simple form of a system consists of a set of CL components,
> their dependencies and the external dependencies to other systems.
>

Yes, I like this approach. Part of where I feel things have fallen down
as been due to trying to do too much all at once. I like the idea of
building up from a good base and that the most fundamental part of that
base is some way of expressing/representing dependencies. Once we crack
that nut, I think we can build more complex functionality. The hard part
is identifying/defining how to best do this and doing so in such a way
that the simple case is still simple and the complex case is no more
complex than it needs to be.

Tim
From: Faré on 16 Apr 2010 02:39
>>>: Juanjo
>>: Jochen Schmidt
>: Tim X

>>> That said, I have read XCVB's manual and I feel it introduces an
>>> unneeded level of complexity in it all.

OK. How would you simplify it?

One simplification I had in mind would have been that at a deep level,
XCVB, like Make and other build systems, should manage files,
and not abstract entities called Lisp modules.

>>> Parallel compilation is nice,
>>> but forcing the use of makefiles, permitting the developers to scatter
>>> the dependencies of their software over different files, and relying
>>> on external drivers, plus the size of the software itself -- ASDF
>>> could and should be so lean! --, does not really convince me.

Wait! Size of the software?

No, you should compare ASDF to XCVB-master.
XCVB-master is what would replace ASDF.
It is ten times smaller than ASDF
(though I plead guilty to making ASDF three times larger).
That's the only thing that lives in the Lisp image and allows
for interactive compilation and loading of stuff.

The rest of XCVB (not counting dependency libraries)
is only twice as big as ASDF, but does much much more.
It includes tool to migrate from ASDF to XCVB and vice versa,
enforcing and non-enforcing Makefile backends,
the beginning of a standalone parallel backend,
an extensive command-line interface,
support for cross-compilation,
plenty of documentation and error-checking, etc.
You should compare to a hypothetical set of ASDF extensions
that would do as much.
And if you don't use all these features --
they don't impact your Lisp image, unlike they do with ASDF.

XCVB does not force you to use Makefiles. It can use them for you.
In the future (when I don't have to maintain ASDF and can focus on
XCVB),
it will do without them altogether.

There are many ways in which XCVB does suck
but I don't think the points you raise are relevant.
XCVB suckage include:
* an architecture that is too specific to ASDF-like defsystems,
* difficulty to extend it into a general-purpose tool.
* CLOS is too rigid, notably; Sheeple might be better (or not).
* no standalone backend yet - it is waiting on IOlib support for
processes,
support for CFFI grovel and Ironclad readtable extensions to ASDF,
etc.


>> That was also my feelings when I investigated it. I quickly came to the
>> conclusion "XCVB" must stand for "Xtra Complicated Variant of a Buildsystem".
Backward compatibility makes XCVB complex.
Please explain to me
1- how you would do a build system if you could forget backward
compatibility.
2- how you would provide a backward compatibility bridge to the above.

>> Actually I think the idea behind XCVB is good, but I fear that the majority of
>> lisp projects do not face the complexities which triggered it. More, I think a
>> successor to what ASDF provides today should really be as simple as possible.
>> A separation of concerns would certainly a step in the right direction.
>
You want simple? Look at XCVB-Master. As simple as can be.


>> A system in its minimal  core doesn't need any inherent facilities to build it
>> or install it or package it - it should only be a straightforward description
>> which allows others to implement stuff like building, installing and so on ON
>> TOP of it. It should be readable by lisp but NOT mandate a complete
>> implementation of CL to interpret it. It shouldn't even be mandatory to write
>> system descriptions by hand -
>> there could easily be generators and IDE support to do this.
>>
Please be more specific. Are you envisioning something like
Smalltalk's
Monticello or something?

For all the Lisp rant about code is data, Common Lisp code is
surprisingly
like files, and very little like data. Moreover, files with complex,
brittle dependencies with each other, rather than declarative pieces
that can be processed as independent units.

Building Common Lisp software these days has all the disadvantages of
building C++ software plus all the disadvantages of building Smalltalk
software, with none of the advantages.

>> To me the most simple form of a system consists of a set of CL components,
>> their dependencies and the external dependencies to other systems.
>
XCVB solves that simple case to perfection.
ASDF does it mostly well (except for how it mismanages system
dependencies).
What are you complaining about?

> Yes, I like this approach. Part of where I feel things have fallen down
> as been due to trying to do too much all at once. I like the idea of
> building up from a good base and that the most fundamental part of that
> base is some way of expressing/representing dependencies. Once we crack
> that nut, I think we can build more complex functionality. The hard part
> is identifying/defining how to best do this and doing so in such a way
> that the simple case is still simple and the complex case is no more
> complex than it needs to be.
>
I'm most open to suggestions on how I could rewrite XCVB in a better
way.
Please explain.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org
]
To authorize all commercial relations between consenting adults.
From: Juanjo on 16 Apr 2010 02:43
On Apr 16, 12:45 am, "o.jas...(a)gmail.com" <o.jas...(a)gmail.com> wrote:
> Note: in the linked lisp file, there are two mistakes in (def-a-
> system :t..) (that is why it doesn't load)
>
> You can do anything with dependencies with the given libs, including
> calling different actions back and forth. Compiling/loading would be
> something like this:http://www.ojasper.nl/src/lisp/dep/compiler.lisp

This is what ASDF is doing right now, but what ASDF does (and I think
it gets it right as for the goal, but probably not for the means) not
to execute things immediately, but to produce a "plan", a
topologically sorted lists of actions.

> I don't mean to imply any sort of dismissiveness of the data, Juanjo,
> this really is something that needs attention, and it is good to have
> a list of projects that misuse asd files. Not sure that i'd consider
> two Defsystems in an asd file an error.(Well, seems like asdf doesn't
> read in both, but hardly much of an issue in-principle if it would.)

Please understand that my study does not qualify them as "errors" from
the ASDF point of view, but as making the system unusable for
automated inspection, for it is actually hiding the dependencies: we
do not know all the systems that exist unless we read in all the
specifications. The study is about "practices" and knowing how people
use ASDF and how this would interfere with other ways of doing things.
From: Juanjo on 16 Apr 2010 02:50
On Apr 16, 5:26 am, RPG <rpgold...(a)gmail.com> wrote:
> On Apr 15, 3:06 pm, Nathan Froyd <froy...(a)gmail.com> wrote:
>
> > On Apr 13, 6:29 pm, Juanjo <juanjose.garciarip...(a)googlemail.com>
> > wrote:
>
> > > * Restrict the set of operations that ASDF supports to a few: compile,
> > > load & test
>
> > Please don't do this.[...]
>
> I would like to second this.  More and more I find that I write
> systems that combine Lisp with other tools.  It is common for me to write new
> operations in my ASDF files.
>
> It might be reasonable to think harder about what the BUILDING BLOCKS
> for such new operations and components should be --- I think ASDF got a
> number of things wrong here --- but not take this ability away.

Please understand that I do not want to remove extensibility, but I
would like it to be more "rational" in a sense that polluting the
space of methods with actions for which you do not declare any
relation between them (which ones do produce files, how is the result
of their action affecting the system, which actions do result in the
lisp loading things,...) prevent any kind of analysis of the resulting
dependencies.

In other words, by using operations or "classes" that are separate
from the ASDF specification you are preventing any kind of automated
analysis of what you are going to do.

Note that I am the most interested one in getting interoperation with
the outer world (C, C++, Fortran) right! What is ECL but a C library
in the end?

The problem is noticing that there are two different things to do
here: 1) providing means to make lisp systems depend and actually
process external software and data, and 2) making them integrate well
with other build systems (makefiles, other projects, etc).

Role 1) is what is making people right now write new kinds of
components, methods, and other tools that live in the *.asd file
itself and from which it is difficult to extract information such as
the actual set of lisp sources that make up a program. Furthermore
these tools actually have complicated dependencies on the software
they build, making for all sort of ugly workarounds such as creating
packages in the asd file itself, double loading, etc.

Role 2) is more related to exposing the information that ASDF has in a
programmable way, so that one can create makefiles, or allowing you
the developer to gather information about files that have to be
installed, where they live, etc.
From: o.jasper on 16 Apr 2010 10:02
I somehow got the impression that you were saying that load/compile
weren't possible that way.
> it[asdf] gets it right as for the goal, but probably not for the means) not
> to execute things immediately, but to produce a "plan", a
> topologically sorted lists of actions.

It does seem silly to first gather a list of things to do and then do
them when you can just do it off/hand like this. Although one has to
think about wanting to use use threading later, however this should be
doable by calling the action-onning the lower systems in threads first
and the process of action-on will wait for those to finish as it
depends on it, and then after that, do it's own thing. Pretty much as
it is one now. (Infact ithink that if mapcar had threadding in it, it
would already be doing it like that) (Of course making such an plan of
action list is perfectly possible with an action.)

> Please understand that my study does not qualify them as "errors"
True, they're not exactly errors from the users pov, but they pretty
much seem like some sort of mistake to me if they hamper future
development and creation of other tools to asdf,

I just had a thought, by overloading the load/compile classes and
making appropriate methods with :around, one should be able to make it
download too, or even hook it into stuff like aptitude repos. However,
it needs to be clear where the downloaded code/binaries goes, and how
that is somehow equivalent to a way of downloading manually.(Perhaps
asking for a directory, with a default in user space, at least for
development-mode) Further _any_ sort of repo entry must be dutifully
updated, otherwise it is nothing more than a fancy download tool.

Btw, presuming people will update their asdf version, we could do
backwards compatibility by 1) modifying asdf to be able to load
systems of the new asdf system 2) making the new system load asdf
systems when needed, it can just look in the new first and then the
old: (or (when (get-system name) ..new way..) (when (find-system
name) ..call old way..), it would extensions would also only have to
implement for both old and new systems, or just focus on one of the
two. (One might have to use a special variable to transmit information
through while asdf is doing stuff.)

Extending might be slightly complicated, but i really don't think any
project that isn't specifically about extending has to extend it, and
i think one could even provide another little package which helps that
a little too, or a *system-hook* and a *file-hook* called upon the
systems and files in appropriate order.

> Furthermore these tools actually have complicated dependencies on the software they build, making for all sort of ugly workarounds such as creating packages in the asd file itself, double loading, etc.

I don't really get what could be complicated about the dependencies. I
mean, they're dependencies, it is gonna be a tree. Something like lack
of knowledge of the dependencies, or limitations is likely the
problem.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5 6 7 8 9 10 11 12
Prev: Is LISP divine?
Next: keywords & macros