Abstractions
in [Lisp]
|
From: RG on 31 Jul 2010 11:30 In article <8bie5qF849U1(a)mid.individual.net>, Pascal Costanza <pc(a)p-cos.net> wrote: > I don't share your sense of humor. Apparently. FWIW, there was a serious point implied in that comment, namely, that your position on abstraction seems contradictory to your position on dynamic typing. See below. > >>> Do you believe that latent typing has value? > >> > >> Yes. > >> > >>> What exactly is that value > >>> if not the ability to defer the choice of data type? > >> > >> That's one among many advantages of "latent" typing. There are others. > > > > Really? Like what? > > Static type systems have to reject programs that can be nevertheless > correct, which can get in the way. No, they don't *have* to reject them. They just *do* as a matter of common practice. There is nothing to prevent a compiler for a statically typed language from compiling the program in such a way that it produces errors at run time rather than compile time. > Dynamically typed languages can support higher degrees of reflection at > runtime. Also not true. Statically typed languages *can* have just as much run-time reflection as dynamically typed languages. But as a matter of practice they don't because it is *possible* (but not necessary) to compile these languages in such a way that there is nothing left at run time to reflect upon. > >>> What is the point > >>> of deferring the choice of a data type if you cannot also defer the > >>> choice of algorithm? (What does it even *mean* to defer the choice of > >>> data type without deferring the choice of algorithm?) > >> > >> If you have to change the data type, you probably also have to change > >> the algorithm manually yourself, at least that seems to be the case > >> based on my experience. > >> > >> A good example is 'elt: It's a good accessor for vectors, but a very bad > >> one for lists. > > > > Really? What would you recommend to use instead? NTH? > > I would recommend not to use random access into lists, but rather using > mapcar and loops over list elements, and the likes. You're making a tacit assumption that I'm iterating over the elements of the list in sequence. What if that's not what I'm doing? What if I just want to (repeatedly) extract a single element located at a position that isn't known until run time? > >> There are better examples, I know. > > > > Indeed. A much better example is GETF/ASSOC/GETHASH. There is no > > equivalent in CL of ELT for these accessors, so the design of CL > > *requires* you to make a premature optimization if you want an > > associative map. (Worse, CL doesn't provide reader syntax for hash > > tables, which implicitly disparages them in favor of ALists and PLists, > > both of which are almost never the right choice for anything except the > > simplest examples.) > > Alists have the easiest way of determining whether an association was > actually in a list or not. That's not an inherent property of ALists, that's a consequence of the way ASSOC is written, so that you have to make an additional call to separate the key and value. You could do exactly the same thing with PLists with an accessor whose return convention was analogous to MEMBER. But this is beside the point. > Hash tables are only more efficient if there are more than 50, > or so, mappings in a table (unless the hash table does automatic > adaptation of its internal representation, but I'm not aware of a good > implementation of such an adaptive hash table). To me, this is another > good example of data structures that provide slightly interesting > differences that I nevertheless find worthwhile to be able to take > advantage of. You find the difference between O(1) access and O(n) access "slightly interesting"?!? My, we really do have a difference in perspective here. This is exactly my point: if you believe you will have <50 elements that will lead you to make a particular choice. If you later learn that you in fact have more than 50 elements and you need to change the representation in order to achieve adequate performance and you have not used an abstraction layer then you will have a lot more work to do to fix the problem than if you had used an abstraction layer. Ergo one ought to use abstraction layers except in cases where there is no doubt that all of the factors that go into such decisions are reliably known and will not change. I submit that in real-world situations, such cases are rare. Furthermore, the fact that they are rare is the main reason that latent typing is useful. > I would not like these differences to be removed because > of an attempt to make everything look the same. Neither would I. Just because you use an abstraction doesn't mean you must relinquish all control of the underlying implementation. > Fortunately, there is no contradiction there. You could use a collection > library, while I could continue to use the low-level substrates... If you were just writing code for your own use I could accept that. But you are an academic. You pass your ideas on to your students and people who read your papers. You are also a well known and highly respected member of this community and people listen to what you have to say. So the damage you do by advancing bad advice is not limited to your own code. > >> But they don't really convince me > >> that much either. Maybe your experiences are just different from mine, > >> who knows. > > > > Have you never had to change large sections of code because you found > > out after you wrote it that it ran too slowly because you used a list > > where you should have used a vector? Or an AList where you should have > > used a hash table? Or a hash table where you should have used a > > red-black tree? > > Sure. It's a trade off. I still prefer being able to make the > fine-grained distinctions. So do I. Maybe this is the source of your confusion so I'll reiterate: using an abstraction layer does NOT require you to relinquish control over the underlying implementation. > >>> Or do you think > >>> that latent typing has no benefits beyond "simple examples" and that > >>> complex systems ought to be written in statically typed languages? > >> > >> This is not an implication of what I said. > > > > It wasn't intended to be. It was intended to be a possible explanation > > of why you said what you said. > > I don't know what to make of this. I'm pretty sure you are aware of my > position against static typing. Of course. > So it seems to me that you're asking > this rhetorical question to provoke a certain reaction in me. Why do you > do that? In order to get at the root of our disagreement. We agree about the desirability of dynamic typing but not about abstraction. But this is odd because IMO the reason dynamic typing is desirable is *because* it *is* an abstraction. There is no essential difference in my mind between forcing me to choose at compile time whether a variable is an int or a float, and forcing me to choose whether it is an AList or a PList. So by asking you to explain your position one of two things will happen. Either you will come to this realization, or you will be able to explain to me why AList/PList *is* a distinction that one ought to be forced to make when one writes code even though int/float isn't, and I will learn something. > >> I'm not trying to argue against building a good abstract collection API, > >> if you believe that's going to buy you something. I'm just trying to > >> express my skepticism about their usefulness, which seems to be > >> considered higher than it actually is, IMHO. YMMV. > > > > But that is exactly what puzzles me. You see the value in latent > > typing, but not in abstraction. And yet the value in those two things > > is exactly the same, namely, the ability to write code in a way that > > lets you defer decisions. So I don't understand how you can value one > > and not the other. > > I see value in abstraction, but not as much as seems to be the common > case. It's not an either-or thing, there are different degrees possible. > For example, static typing allows for stronger abstractions than dynamic > typing, yet I still prefer dynamic typing, with a reasonable amount of > abstractions here and there. (What is a "stronger" abstraction?) But this is just what I'm trying to get at. *Why* do you see more value in the particular abstraction of dynamic typing than you do in abstraction in general? rg
From: Pascal Costanza on 31 Jul 2010 16:53 On 31/07/2010 17:30, RG wrote: >>>>> Do you believe that latent typing has value? >>>> >>>> Yes. >>>> >>>>> What exactly is that value >>>>> if not the ability to defer the choice of data type? >>>> >>>> That's one among many advantages of "latent" typing. There are others. >>> >>> Really? Like what? >> >> Static type systems have to reject programs that can be nevertheless >> correct, which can get in the way. > > No, they don't *have* to reject them. They just *do* as a matter of > common practice. There is nothing to prevent a compiler for a > statically typed language from compiling the program in such a way that > it produces errors at run time rather than compile time. Ok, I rephrase: static type systems have to produce errors for certain programs that may otherwise run correctly. >> Dynamically typed languages can support higher degrees of reflection at >> runtime. > > Also not true. Statically typed languages *can* have just as much > run-time reflection as dynamically typed languages. This is demonstrably false. Dynamically typed languages allow for degrees of reflection that violate assumptions that static type systems usually make - for example that slots or functions are not removed at run time. This can cause run-time errors where a static type system would predict the absence of such errors. Supporters of static type systems wouldn't be very happy with a type system that "proves" the absence of certain errors, but allows the run-time system to reintroduce these errors. >>>>> What is the point >>>>> of deferring the choice of a data type if you cannot also defer the >>>>> choice of algorithm? (What does it even *mean* to defer the choice of >>>>> data type without deferring the choice of algorithm?) >>>> >>>> If you have to change the data type, you probably also have to change >>>> the algorithm manually yourself, at least that seems to be the case >>>> based on my experience. >>>> >>>> A good example is 'elt: It's a good accessor for vectors, but a very bad >>>> one for lists. >>> >>> Really? What would you recommend to use instead? NTH? >> >> I would recommend not to use random access into lists, but rather using >> mapcar and loops over list elements, and the likes. > > You're making a tacit assumption that I'm iterating over the elements of > the list in sequence. What if that's not what I'm doing? What if I > just want to (repeatedly) extract a single element located at a position > that isn't known until run time? Then a list is not a good choice of data structure in such a case. >>>> There are better examples, I know. >>> >>> Indeed. A much better example is GETF/ASSOC/GETHASH. There is no >>> equivalent in CL of ELT for these accessors, so the design of CL >>> *requires* you to make a premature optimization if you want an >>> associative map. (Worse, CL doesn't provide reader syntax for hash >>> tables, which implicitly disparages them in favor of ALists and PLists, >>> both of which are almost never the right choice for anything except the >>> simplest examples.) >> >> Alists have the easiest way of determining whether an association was >> actually in a list or not. > > That's not an inherent property of ALists, that's a consequence of the > way ASSOC is written, so that you have to make an additional call to > separate the key and value. You could do exactly the same thing with > PLists with an accessor whose return convention was analogous to MEMBER. Maybe. It just happens not to be the case. > But this is beside the point. Indeed. >> Hash tables are only more efficient if there are more than 50, >> or so, mappings in a table (unless the hash table does automatic >> adaptation of its internal representation, but I'm not aware of a good >> implementation of such an adaptive hash table). To me, this is another >> good example of data structures that provide slightly interesting >> differences that I nevertheless find worthwhile to be able to take >> advantage of. > > You find the difference between O(1) access and O(n) access "slightly > interesting"?!? My, we really do have a difference in perspective here. You're discussion style is degrading again. A difference between O(1) and O(n) is obviously not a slight difference, so instead of assuming that I'm stupid, you could also assume that I meant something else. > This is exactly my point: if you believe you will have<50 elements that > will lead you to make a particular choice. If you later learn that you > in fact have more than 50 elements and you need to change the > representation in order to achieve adequate performance and you have not > used an abstraction layer then you will have a lot more work to do to > fix the problem than if you had used an abstraction layer. Ergo one > ought to use abstraction layers except in cases where there is no doubt > that all of the factors that go into such decisions are reliably known > and will not change. I submit that in real-world situations, such cases > are rare. Well, that's your opinion. It's good that we can create a situation where we are both happy. > Furthermore, the fact that they are rare is the main reason > that latent typing is useful. The existence of collection frameworks in statically typed languages proves that this issue is independent of whether you have latent or static typing. >> I would not like these differences to be removed because >> of an attempt to make everything look the same. > > Neither would I. Just because you use an abstraction doesn't mean you > must relinquish all control of the underlying implementation. > >> Fortunately, there is no contradiction there. You could use a collection >> library, while I could continue to use the low-level substrates... > > If you were just writing code for your own use I could accept that. But > you are an academic. You pass your ideas on to your students and people > who read your papers. You are also a well known and highly respected > member of this community and people listen to what you have to say. So > the damage you do by advancing bad advice is not limited to your own > code. I hope that people don't just blindly follow other people, but instead form their own opinion. What we discuss here is a matter of style, not a matter of right or wrong, or bad or good. I encourage my students to use their own programming styles, and this works quite well. I disagree with your assessment that my style creates 'damage'. >>>> But they don't really convince me >>>> that much either. Maybe your experiences are just different from mine, >>>> who knows. >>> >>> Have you never had to change large sections of code because you found >>> out after you wrote it that it ran too slowly because you used a list >>> where you should have used a vector? Or an AList where you should have >>> used a hash table? Or a hash table where you should have used a >>> red-black tree? >> >> Sure. It's a trade off. I still prefer being able to make the >> fine-grained distinctions. > > So do I. Maybe this is the source of your confusion so I'll reiterate: > using an abstraction layer does NOT require you to relinquish control > over the underlying implementation. I'm not stopping anybody from creating a good collection framework. If the result convinces me, I'm happy to use it. What I have seen so far doesn't convince me, but who am I to predict there can't be anything better? >>>>> Or do you think >>>>> that latent typing has no benefits beyond "simple examples" and that >>>>> complex systems ought to be written in statically typed languages? >>>> >>>> This is not an implication of what I said. >>> >>> It wasn't intended to be. It was intended to be a possible explanation >>> of why you said what you said. >> >> I don't know what to make of this. I'm pretty sure you are aware of my >> position against static typing. > > Of course. > >> So it seems to me that you're asking >> this rhetorical question to provoke a certain reaction in me. Why do you >> do that? > > In order to get at the root of our disagreement. We agree about the > desirability of dynamic typing but not about abstraction. But this is > odd because IMO the reason dynamic typing is desirable is *because* it > *is* an abstraction. There is no essential difference in my mind > between forcing me to choose at compile time whether a variable is an > int or a float, and forcing me to choose whether it is an AList or a > PList. So by asking you to explain your position one of two things will > happen. Either you will come to this realization, or you will be able > to explain to me why AList/PList *is* a distinction that one ought to be > forced to make when one writes code even though int/float isn't, and I > will learn something. I think here lies your confusion: I don't want to force anybody to do anything. I just wanted to express my skepticism about collection frameworks, that's all. Maybe you're right, maybe the decision between alist, plists, etc., that we have to make in Common Lisp is suboptimal. But I think the kind of abstraction provided by collection frameworks is also overrated. "Abstraction" is not a goal in its own right, it's only a goal to the extent that it serves a purpose and solves an actual problem. I don't see what problem a collection framework solves. (Yes, I understand your examples of wanting to change between alists, plists and hash tables, but I don't think they are as harsh as some people, including you, believe them to be). >>>> I'm not trying to argue against building a good abstract collection API, >>>> if you believe that's going to buy you something. I'm just trying to >>>> express my skepticism about their usefulness, which seems to be >>>> considered higher than it actually is, IMHO. YMMV. >>> >>> But that is exactly what puzzles me. You see the value in latent >>> typing, but not in abstraction. And yet the value in those two things >>> is exactly the same, namely, the ability to write code in a way that >>> lets you defer decisions. So I don't understand how you can value one >>> and not the other. >> >> I see value in abstraction, but not as much as seems to be the common >> case. It's not an either-or thing, there are different degrees possible. >> For example, static typing allows for stronger abstractions than dynamic >> typing, yet I still prefer dynamic typing, with a reasonable amount of >> abstractions here and there. > > (What is a "stronger" abstraction?) As an example, a static type like Collection<T> is such a stronger abstraction, because it constrains the collection to have elements of all the same type, without specifying what that particular type is. > But this is just what I'm trying to get at. *Why* do you see more value > in the particular abstraction of dynamic typing than you do in > abstraction in general? I'm only interested in dynamic typing to the extent that it allows me to do things that I cannot do in statically typed languages. The connection between "abstraction" and "dynamic typing" exists in your mind, not in mine. Pascal -- My website: http://p-cos.net Common Lisp Document Repository: http://cdr.eurolisp.org Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: RG on 1 Aug 2010 02:14 In article <8bjgqeFk85U1(a)mid.individual.net>, Pascal Costanza <pc(a)p-cos.net> wrote: > >> Hash tables are only more efficient if there are more than 50, > >> or so, mappings in a table (unless the hash table does automatic > >> adaptation of its internal representation, but I'm not aware of a good > >> implementation of such an adaptive hash table). To me, this is another > >> good example of data structures that provide slightly interesting > >> differences that I nevertheless find worthwhile to be able to take > >> advantage of. > > > > You find the difference between O(1) access and O(n) access "slightly > > interesting"?!? My, we really do have a difference in perspective here. > > You're discussion style is degrading again. A difference between O(1) > and O(n) is obviously not a slight difference, so instead of assuming > that I'm stupid, you could also assume that I meant something else. I'm not assuming that you're stupid, I am hypothesizing that we are employing very different quality metrics. > > Furthermore, the fact that they are rare is the main reason > > that latent typing is useful. > > The existence of collection frameworks in statically typed languages > proves that this issue is independent of whether you have latent or > static typing. We've gotten too hung up on latent typing. Let me try this another way: the whole point of dynamicism in Lisp, whether it's latent typing, or being able to redefine functions and methods at run time, or being able to do things like change-class, is to be able to easily make changes at run time. It is puzzling to me that someone who is so enthusiastic about Lisp in particular and dynamicism in general is so unenthusiastic about collection abstractions, which also serve to make it easy to make changes at run time. > > If you were just writing code for your own use I could accept that. But > > you are an academic. You pass your ideas on to your students and people > > who read your papers. You are also a well known and highly respected > > member of this community and people listen to what you have to say. So > > the damage you do by advancing bad advice is not limited to your own > > code. > > I hope that people don't just blindly follow other people, but instead > form their own opinion. What we discuss here is a matter of style, not a > matter of right or wrong, or bad or good. I encourage my students to use > their own programming styles, and this works quite well. > > I disagree with your assessment that my style creates 'damage'. It's not your style that's the problem. It's the substance of what you say. > >>>> But they don't really convince me > >>>> that much either. Maybe your experiences are just different from mine, > >>>> who knows. > >>> > >>> Have you never had to change large sections of code because you found > >>> out after you wrote it that it ran too slowly because you used a list > >>> where you should have used a vector? Or an AList where you should have > >>> used a hash table? Or a hash table where you should have used a > >>> red-black tree? > >> > >> Sure. It's a trade off. I still prefer being able to make the > >> fine-grained distinctions. > > > > So do I. Maybe this is the source of your confusion so I'll reiterate: > > using an abstraction layer does NOT require you to relinquish control > > over the underlying implementation. > > I'm not stopping anybody from creating a good collection framework. If > the result convinces me, I'm happy to use it. What I have seen so far > doesn't convince me, but who am I to predict there can't be anything better? What have you looked at? Where do the things you've looked at fall short? > >>>>> Or do you think > >>>>> that latent typing has no benefits beyond "simple examples" and that > >>>>> complex systems ought to be written in statically typed languages? > >>>> > >>>> This is not an implication of what I said. > >>> > >>> It wasn't intended to be. It was intended to be a possible explanation > >>> of why you said what you said. > >> > >> I don't know what to make of this. I'm pretty sure you are aware of my > >> position against static typing. > > > > Of course. > > > >> So it seems to me that you're asking > >> this rhetorical question to provoke a certain reaction in me. Why do you > >> do that? > > > > In order to get at the root of our disagreement. We agree about the > > desirability of dynamic typing but not about abstraction. But this is > > odd because IMO the reason dynamic typing is desirable is *because* it > > *is* an abstraction. There is no essential difference in my mind > > between forcing me to choose at compile time whether a variable is an > > int or a float, and forcing me to choose whether it is an AList or a > > PList. So by asking you to explain your position one of two things will > > happen. Either you will come to this realization, or you will be able > > to explain to me why AList/PList *is* a distinction that one ought to be > > forced to make when one writes code even though int/float isn't, and I > > will learn something. > > I think here lies your confusion: I don't want to force anybody to do > anything. I just wanted to express my skepticism about collection > frameworks, that's all. No, I get that. What I don't get is the source of your skepticism. It doesn't jibe with your lack of skepticism about Lisp in general. > Maybe you're right, maybe the decision between alist, plists, etc., that > we have to make in Common Lisp is suboptimal. But I think the kind of > abstraction provided by collection frameworks is also overrated. Overrated according to whom? I don't hear anyone saying that collection frameworks are going to save the world. I'm certainly not saying that. All I'm saying is that all else being equal, hiding different implementations of the same interface behind a common abstraction is a win. Maybe not a huge win, but a win nonetheless. > "Abstraction" is not a goal in its own right, it's only a goal to the > extent that it serves a purpose and solves an actual problem. I don't > see what problem a collection framework solves. (Yes, I understand your > examples of wanting to change between alists, plists and hash tables, > but I don't think they are as harsh as some people, including you, > believe them to be). OK, I guess we'll just have to agree to disagree about that. Maybe you never experienced the pain of having to make a change in a large system that was not properly abstracted. rg
From: Pascal Costanza on 2 Aug 2010 06:27 On 01/08/2010 08:14, RG wrote: > In article<8bjgqeFk85U1(a)mid.individual.net>, > Pascal Costanza<pc(a)p-cos.net> wrote: > >>>> Hash tables are only more efficient if there are more than 50, >>>> or so, mappings in a table (unless the hash table does automatic >>>> adaptation of its internal representation, but I'm not aware of a good >>>> implementation of such an adaptive hash table). To me, this is another >>>> good example of data structures that provide slightly interesting >>>> differences that I nevertheless find worthwhile to be able to take >>>> advantage of. >>> >>> You find the difference between O(1) access and O(n) access "slightly >>> interesting"?!? My, we really do have a difference in perspective here. >> >> You're discussion style is degrading again. A difference between O(1) >> and O(n) is obviously not a slight difference, so instead of assuming >> that I'm stupid, you could also assume that I meant something else. > > I'm not assuming that you're stupid, I am hypothesizing that we are > employing very different quality metrics. Well, your hypothesis is wrong, I meant something else. >>> If you were just writing code for your own use I could accept that. But >>> you are an academic. You pass your ideas on to your students and people >>> who read your papers. You are also a well known and highly respected >>> member of this community and people listen to what you have to say. So >>> the damage you do by advancing bad advice is not limited to your own >>> code. >> >> I hope that people don't just blindly follow other people, but instead >> form their own opinion. What we discuss here is a matter of style, not a >> matter of right or wrong, or bad or good. I encourage my students to use >> their own programming styles, and this works quite well. >> >> I disagree with your assessment that my style creates 'damage'. > > It's not your style that's the problem. It's the substance of what you > say. I also disagree that the substance of why I say creates 'damage'. >> I'm not stopping anybody from creating a good collection framework. If >> the result convinces me, I'm happy to use it. What I have seen so far >> doesn't convince me, but who am I to predict there can't be anything better? > > What have you looked at? Where do the things you've looked at fall > short? Ah, picked up from another part of this thread: I'm always forgetting FSet in such discussions. FSet looks really promising. I should really take a closer look at it, maybe that one will convince me... >> Maybe you're right, maybe the decision between alist, plists, etc., that >> we have to make in Common Lisp is suboptimal. But I think the kind of >> abstraction provided by collection frameworks is also overrated. > > Overrated according to whom? I don't hear anyone saying that collection > frameworks are going to save the world. I'm certainly not saying that. > All I'm saying is that all else being equal, hiding different > implementations of the same interface behind a common abstraction is a > win. Maybe not a huge win, but a win nonetheless. Well, what I'm trying to get across is that not all else is equal, IMHO. >> "Abstraction" is not a goal in its own right, it's only a goal to the >> extent that it serves a purpose and solves an actual problem. I don't >> see what problem a collection framework solves. (Yes, I understand your >> examples of wanting to change between alists, plists and hash tables, >> but I don't think they are as harsh as some people, including you, >> believe them to be). > > OK, I guess we'll just have to agree to disagree about that. Maybe you > never experienced the pain of having to make a change in a large system > that was not properly abstracted. Yes, I did, but not because of collections. Pascal -- My website: http://p-cos.net Common Lisp Document Repository: http://cdr.eurolisp.org Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: x on 2 Aug 2010 13:41
RG <rNOSPAMon(a)flownet.com> wrote in news:rNOSPAMon-2C2847.23140231072010(a)news.albasani.net: > OK, I guess we'll just have to agree to disagree about that. Maybe > you never experienced the pain of having to make a change in a large > system that was not properly abstracted. "Properly abstracted" is the issue. Good software uses a tree of abstractions. If you have to make changes throughout your code because of a change in usage of a collection abstraction, it's because the higher level abstractions weren't done properly, not because the collection abstraction wasn't done properly. If you change a list to a vector, why should you have to change access to it in dozens of different places? Why not only in the abstraction where the list or vector is used? That's the fallacy of the value of collection frameworks. They're inherently low level. Programmers use them incorrectly as substitutes for proper abstraction. |