Write a Lexer for the source language that you have chosen. The output of the lexer must be a “summary” of the tokens in the program.
Example
For a program in the C language:
Input:
main ( )
{
i n t a = 9 5 9 ;
f o o ( a ) ;
r e t u r n 0 ;
}
Expected Output:
Token
Occurrances
Lexemes
OP_ASGN
1
=
’(’
2
(
’)’
2
)
IDENTIFIER
3
a
foo
main
BLOCK_BEGIN
1
{
BLOCK_END
1
}
TYPE
1
int
KEYWORD_RET
1
return
INT_CONST
2
959
1
STMT_TERMINATOR
3
;
Details
• You are free to select your own token names.
• Your implementation should read the source filename as its first command-line parameter; it should produce its output on STDOUT.
• You must use a lexer generator like Lex, Flex etc.
• The tool should be robust; any failure in tokenizing due to errors in the input program must be reported properly.
• You have to submit a zipped folder (name the folder “asgn1”) with:
– the source of the implementation (in a folder called “src” within “asgn1”;
– a Makefile to build the implementation (it should generate an executable called “lexer” in the folder
“asgn1/bin”;
– a set of at least 5 test cases that you have used to check your implementation (in a folder “asgn1/test”);
– a README file with a brief description for building and running it (within “asgn1”).
Binaries should NOT be part of the submission. Clean the folder of all object and executable files before submission.
• We will apply the following set of commands to build and run your implementation; make sure that your implementation works correctly with these sequence of commands:
– cd asgn1
– make
– bin/lexer test/test1.c (to execute the first test-case file test1.c)
