C/C++ Tokens

A token is the smallest element of a program that is meaningful to the compiler. Tokens can be classified as follows:

1. Keywords
2. Identifiers
3. Constants
4. Strings
5. Special Symbols
6. Operators

1. Keyword: Keywords are pre-defined or reserved words in a programming language. Each keyword is meant to perform a specific function in a program. Since keywords are referred names for a compiler, they can’t be used as variable names because by doing so, we are trying to assign a new meaning to the keyword which is not allowed. You cannot redefine keywords. However, you can specify text to be substituted for keywords before compilation by using C/C++ preprocessor directives.C language supports 32 keywords which are given below:

   
   auto         double      int        struct
   break        else        long       switch
   case         enum        register   typedef
   char         extern      return     union
   const        float       short      unsigned
   continue     for         signed     void
   default      goto        sizeof     volatile
   do           if          static     while
    

   While in C++ there are 31 additional keywords other than C Keywords they are:

   
   asm          bool        catch          class
   const_cast   delete      dynamic_cast   explicit 
   export       false       friend         inline 
   mutable      namespace   new            operator 
   private      protected   public         reinterpret_cast
   static_cast  template    this           throw
   true         try         typeid         typename 
   using        virtual     wchar_t 
   

2. Identifiers: Identifiers are used as the general terminology for naming of variables, functions and arrays. These are user defined names consisting of arbitrarily long sequence of letters and digits with either a letter or the underscore(_) as a first character. Identifier names must differ in spelling and case from any keywords. You cannot use keywords as identifiers; they are reserved for special use. Once declared, you can use the identifier in later program statements to refer to the associated value. A special kind of identifier, called a statement label, can be used in goto statements.
   
   There are certain rules that should be followed while naming c identifiers:
   
   • They must begin with a letter or underscore(_).
   • They must consist of only letters, digits, or underscore. No other special character is allowed.
   • It should not be a keyword.
   • It must not contain white space.
   • It should be up to 31 characters long as only first 31 characters are significant.

   Some examples of c identifiers:

   name	Remark
   _A9	Valid
   Temp.var	Invalid as it contains special character other than the underscore
   void	Invalid as it is a keyword

   C program:

   
   
   

    void main()
   {
     int a = 10;
   }
   

   In the above program there are 2 identifiers:

   1. main: method name.
   2. a: variable name.
3. Constants: Constants are also like normal variables. But, only difference is, their values can not be modified by the program once they are defined. Constants refer to fixed values. They are also called as literals.
   Constants may belong to any of the data type.Syntax:

   const data_type variable_name; (or) const data_type *variable_name;
   Types of Constants:

   1. Integer constants – Example: 0, 1, 1218, 12482
   2. Real or Floating point constants – Example: 0.0, 1203.03, 30486.184
   3. Octal & Hexadecimal constants – Example: octal: (013 )₈ = (11)_10, Hexadecimal: (013)₁₆ = (19)₁₀
   4. Character constants -Example: ‘a’, ‘A’, ‘z’
   5. String constants -Example: “GeeksforGeeks”
4. Strings: Strings are nothing but an array of characters ended with a null character (‘’).This null character indicates the end of the string. Strings are always enclosed in double quotes. Whereas, a character is enclosed in single quotes in C and C++.Declarations for String:
   • char string[20] = {‘g’, ’e’, ‘e’, ‘k’, ‘s’, ‘f’, ‘o’, ‘r’, ‘g’, ’e’, ‘e’, ‘k’, ‘s’, ‘’};
   • char string[20] = “geeksforgeeks”;
   • char string [] = “geeksforgeeks”;

   Difference between above declarations are:

   1. when we declare char as “string[20]”, 20 bytes of memory space is allocated for holding the string value.
   2. When we declare char as “string[]”, memory space will be allocated as per the requirement during execution of the program.
5. Special Symbols: The following special symbols are used in C having some special meaning and thus, cannot be used for some other purpose.[] () {}, ; * = #
   • Brackets[]: Opening and closing brackets are used as array element reference. These indicate single and multidimensional subscripts.
   • Parentheses(): These special symbols are used to indicate function calls and function parameters.
   • Braces{}: These opening and ending curly braces marks the start and end of a block of code containing more than one executable statement.
   • comma (, ): It is used to separate more than one statements like for separating parameters in function calls.
   • semi colon : It is an operator that essentially invokes something called an initialization list.
   • asterick (*): It is used to create pointer variable.
   • assignment operator: It is used to assign values.
   • pre processor(#): The preprocessor is a macro processor that is used automatically by the compiler to transform your program before actual compilation.
6. Operators: Operators are symbols that triggers an action when applied to C variables and other objects. The data items on which operators act upon are called operands.
   Depending on the number of operands that an operator can act upon, operators can be classified as follows:
   • Unary Operators: Those operators that require only single operand to act upon are known as unary operators.For Example increment and decrement operators
   • Binary Operators: Those operators that require two operands to act upon are called binary operators. Binary operators are classified into :
     1. Arithmetic operators
     2. Relational Operators
     3. Logical Operators
     4. Assignment Operators
     5. Conditional Operators
     6. Bitwise Operators

     Ternary Operators: These operators requires three operands to act upon. For Example Conditional operator(?:).

     For more information about operators click