Difference between revisions of "Lexical analysis"
From Wiki @ Karl Jones dot com
Karl Jones (Talk | contribs) (Etc) |
Karl Jones (Talk | contribs) (etc) |
||
| Line 3: | Line 3: | ||
A program that performs lexical analysis may be called a lexer, tokenizer, or scanner (though "scanner" is also used to refer to the first stage of a lexer). | A program that performs lexical analysis may be called a lexer, tokenizer, or scanner (though "scanner" is also used to refer to the first stage of a lexer). | ||
| − | Such a lexer is generally combined with a parser, which together analyze the [[Syntax (computer | + | Such a lexer is generally combined with a parser, which together analyze the [[Syntax (computer programming)|syntax]] of [[Programming language|programming languages]], [[Web page|web pages]], and so forth. |
== See also == | == See also == | ||
Revision as of 19:05, 26 August 2015
In computer science, lexical analysis is the process of converting a sequence of characters (such as a computer program or web page) into a sequence of tokens (strings with an identified "meaning").
A program that performs lexical analysis may be called a lexer, tokenizer, or scanner (though "scanner" is also used to refer to the first stage of a lexer).
Such a lexer is generally combined with a parser, which together analyze the syntax of programming languages, web pages, and so forth.
See also
External links
- Lexical analysis @ Wikipedia
