Posted in: Python

What are Tokens – Python

In this article, we are going to learn what are tokens in Python Programming Language.

Tokens in Python
Tokens – Python

Definition of Token: A program in Python contains a sequence of instructions. Python breaks each statement into a sequence of lexical components known as tokens.

Each token corresponds to a substring of a statement.

Type of Tokens

Python have different types of tokens. The following list shows the tokens supported by Python :

  • Keywords
  • Identifiers/Variables
  • Operators
  • Delimiters
  • Literals