From: J-P. Rosen on
Niklas Holsti a �crit :
>> And of course, you are welcome to have a look at my scanner in
>> AdaControl ;-)
>
> I did, and I even compiled and tried it, but since your scanner ignores
> and skips all line terminators (there is no "end of line" token kind),
> its operation does not illuminate the question of the "wart".
>
Huh? There is an At_EoL boolean, even if it is not a full fledged token...

--
---------------------------------------------------------
J-P. Rosen (rosen(a)adalog.fr)
Visit Adalog's web site at http://www.adalog.fr
From: Niklas Holsti on
J-P. Rosen wrote:
> Niklas Holsti a �crit :
>>> And of course, you are welcome to have a look at my scanner in
>>> AdaControl ;-)
>> I did, and I even compiled and tried it, but since your scanner ignores
>> and skips all line terminators (there is no "end of line" token kind),
>> its operation does not illuminate the question of the "wart".
>>
> Huh? There is an At_EoL boolean, even if it is not a full fledged token...

Ah yes, there is an At_Eol variable in the package body, and a
like-named component in the private type Scanner_State, but those are
not accessible to clients of the scanner.

I modified your package Framework.Language.Scanner, to provide a
function that queries At_Eol, and made a test program that calls
Start_Scan and then calls Next_Token repeatedly, until it returns the
Eof (end of file) token. After each call of Next_Token the program
prints out the line number of Standard_Input, Current_Token.Kind, and
the value of At_Eol.

In my experiments, the output is exactly the same for a file that
consists of just one line of text, as for a file that consists of this
one line of text followed by a null line.

A client of your scanner thus cannot detect that the input file has a
final empty line, which is the "wart" that has been discussed in this
thread.

I have found a work-around for the wart, so it is possible to use
Text_IO to detect final empty lines. The following program counts the
number of lines in the input file, and works even for empty files
(/dev/null) and for files with final null lines. The trick is to use an
extra call of Skip_Line to see if there is a final line terminator (for
a null line) before the file terminator:

with Ada.Text_IO; use Ada.Text_IO;
procedure Count_Lines
is
Lines : Natural := 0;
begin

while not End_Of_File loop
-- A non-null line at this point, or a null line
-- that is not immediately followed by end of file.
-- Process the line in some way; omitted here.
Lines := Lines + 1;
Skip_Line;
end loop;

-- Here we are at End_Of_File, but there may
-- still be a line terminator for a null line
-- immediately before the true end of the file.

begin
Skip_Line;
-- Skip_Line raises End_Error if there is no line
-- terminator before the end of the file.
Lines := Lines + 1;
exception
when End_Error => null;
end;

Put_Line ("Lines:" & Natural'Image (Lines));

end Count_Lines;


--
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
. @ .
From: Niklas Holsti on
I am replying to my own post to show an even simpler solution to the
"final empty line" problem. I guess this solution is what J-P. Rosen and
others have suggested, earlier in this thread, but I have been too dense
to understand.

The "problem" is that the Text_IO functions End_Of_Line and End_Of_File
give the same result when the input file is positioned at true end of
file as when the file is positioned at a line terminator preceding the
true end of file. This means that a reading loop of the form "while not
End_Of_File loop <read a line> end loop" will ignore a final empty line.

The simple solution is to use a loop that uses Get_Line (or Skip_Line)
and terminates on the End_Error exception. This program reads and counts
the lines in the input file and works for /dev/null (zero lines) as well
as for files that end with empty lines:

with Ada.Text_IO; use Ada.Text_IO;
procedure Count_Lines is
Lines : Natural := 0;
begin
loop
declare
The_Line : constant String := Get_Line;
begin
Lines := Lines + 1;
end;
end loop;
exception
when End_Error =>
Put_Line ("Lines:" & Natural'Image (Lines));
end Count_Lines;

So, with apologies for any confusion about "warts" that I may have
generated, I will now shut up.

--
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
. @ .