Prev: Dave's difficulties with manual activities
Next: Lisp job available in Sydney, Australia
From: Francogrex on 8 Sep 2009 12:27
Why the below to read the test.csv file doesn't work well in sbcl?
---------------------------------------------------------------------------------------------
test.csv:

one,two
three,four,
five,six
---------------------------------------------------------------------------------------------
(defun get-token (delimiter string start)
(let ((pos (position delimiter string :start start)))
(cond ((null pos)
(cond ((= start (length string)) (values "" start))
(t (values (string-trim '(#\Space #\Tab #\Newline)
(subseq string start)) (length string)))))
((= start pos) (values "" pos))
(t (values (string-trim '(#\Space #\Tab #\Newline) (subseq
string start pos)) pos)))))

(defun read-csv-line (&optional input-stream eof-error-p eof-value
recursive-p)
(let ((line (read-line input-stream eof-error-p eof-value recursive-
p)))
(if (eq line eof-value) (return-from read-csv-line eof-value))
(loop with len = (length line) and pos = 0 and token
do (multiple-value-setq (token pos) (get-token #\, line pos))
(incf pos)
collect token
while (< pos len))))

(defun read-csv (input-stream &optional eof-error-p eof-value)
(when (not (eq eof-value (peek-char t input-stream eof-error-p eof-
value)))
(loop for lst = (read-csv-line input-stream eof-error-p eof-value)
while (not (eq lst eof-value))
collect lst)))

(with-open-file (str "c:/test.txt" :direction :input)
(defvar *mycsv* (read-csv str)))

;;In ECL
> *MYCSV*
> (("one" "two") ("three" "four") ("five" "six"))

;;In SBCL it doesn't work well!
*MYCSV*
") ("five" "six"))
From: Michael Weber on 8 Sep 2009 13:25
On Sep 8, 6:27 pm, Francogrex <fra...(a)grex.org> wrote:
> ;;In ECL
>
> > *MYCSV*
> > (("one" "two") ("three" "four") ("five" "six"))
>
> ;;In SBCL it doesn't work well!
> *MYCSV*
> ") ("five" "six"))

I can hardly believe that this is what sbcl printed.

Anyway, perhaps you wanted DEFPARAMETER, because DEFVAR does not set
the variable's value if it is already bound. Also, I'd normally put
the DEFVAR/DEFPARAMETER on toplevel, like so:

(defvar *mycsv*
(with-open-file ...))

Finally, with <http://foldr.org/~michaelw/lisp/tokenizer.lisp>:

(defparameter *foo*
(with-open-file (s "test.csv")
(loop for line = (read-line s nil nil) while line
collect (tokenize line :delimiters #\, :whitespace t))))

*foo*
==> (("one" "two") ("three" "four") ("five" "six"))
From: Francogrex on 8 Sep 2009 15:22
On Sep 8, 7:25 pm, Michael Weber <mw+goo...(a)foldr.org> wrote:
> On Sep 8, 6:27 pm, Francogrex <fra...(a)grex.org> wrote:
>
> > ;;In ECL
>
> > > *MYCSV*
> > > (("one" "two") ("three" "four") ("five" "six"))
>
> > ;;In SBCL it doesn't work well!
> > *MYCSV*
> > ") ("five" "six"))
>
> I can hardly believe that this is what sbcl printed.

I couldn't believe it either. I had tried version 1.0.23 and now
1.0.29, those are on windows (XP), and the above is what is really
printed. First I had thought that version 23 was defective but now
that I see this aberration in a newer version as well makes me think
it's rather a serious bug that should be reported.
From: Zach Beane on 8 Sep 2009 15:34
Francogrex <franco(a)grex.org> writes:

> On Sep 8, 7:25 pm, Michael Weber <mw+goo...(a)foldr.org> wrote:
>> On Sep 8, 6:27 pm, Francogrex <fra...(a)grex.org> wrote:
>>
>> > ;;In ECL
>>
>> > > *MYCSV*
>> > > (("one" "two") ("three" "four") ("five" "six"))
>>
>> > ;;In SBCL it doesn't work well!
>> > *MYCSV*
>> > ") ("five" "six"))
>>
>> I can hardly believe that this is what sbcl printed.
>
> I couldn't believe it either. I had tried version 1.0.23 and now
> 1.0.29, those are on windows (XP), and the above is what is really
> printed. First I had thought that version 23 was defective but now
> that I see this aberration in a newer version as well makes me think
> it's rather a serious bug that should be reported.

Or maybe it's an issue of SBCL using Unix line-ending conventions, even
in Windows?

Zach
From: Francogrex on 8 Sep 2009 17:28
On Sep 8, 9:34 pm, Zach Beane <x...(a)xach.com> wrote:
> Or maybe it's an issue of SBCL using Unix line-ending conventions, even
> in Windows?

Indeed, that's exactly it! I used in msys $ tr -d '\r' <c:/test.csv>
c:/unix.csv
and now it works:
(("one" "two") ("three" "four") ("five" "six"))
[BTW, I like also very much Michael Weber's tokenize he posted above
I'm using it now.]
Can I have this dos to unix conversion already set into sbcl so i
don't have to convert manually every file before reading it in?

 |  Next  |  Last
Pages: 1 2
Prev: Dave's difficulties with manual activities
Next: Lisp job available in Sydney, Australia